Image classification method and device

Abstract

The present application discloses an image classification method, comprising: encrypting an image to be classified; simplifying the attributes of the encrypted image, wherein the number of attributes of the simplified image changes from a first number to a second number, the first number is greater than the second number, and the attributes of the image are the basis for classifying the image; inputting the encrypted image after attribute simplification into a model to obtain an output result, wherein the model is a machine learning model for classification trained using multiple sets of training data; and decrypting the output result of the model to obtain the classification result of the image. The present application solves the problem of inefficiency caused by privacy protection when classifying images using machine learning technology, thereby making image classification safe and efficient.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence, and specifically, to an image classification method based on machine learning. Background Art

[0002] In recent years, artificial intelligence technology has gradually been applied to various fields, and the application of existing machine learning technology in the field of image classification has made certain breakthroughs. The main algorithms involved are neural networks, support vector machines, decision trees, Bayesian networks, and various improved algorithms.

[0003] However, since digital images contain a lot of private information, they are vulnerable to theft, tampering or destruction. Once the data is leaked, it will cause irreparable losses. At present, the commonly used method to protect the privacy of images is image encryption. For complex image information, encryption and decryption takes too long, resulting in inefficiency of the entire classification process. Summary of the invention

[0004] The embodiment of the present application provides an image classification method to at least solve the problem of inefficiency caused by privacy protection when classifying images using machine learning technology.

[0005] According to one aspect of the present application, there is provided an image classification method, comprising:

[0006] Encrypt the image to be classified;

[0007] Simplifying the attributes of the encrypted image, wherein the number of the attributes of the simplified image changes from a first number to a second number, the first number is greater than the second number, and the attributes of the image are the basis for classifying the image;

[0008] Inputting the encrypted image after attribute reduction into the model to obtain an output result, wherein the model is a machine learning model for classification trained using multiple sets of training data;

[0009] The classification result of the image is obtained after decrypting the output result of the model.

[0010] Furthermore, simplifying the attributes of the encrypted image includes:

[0011] The encrypted image is simplified using an encryption attribute reduction algorithm, wherein the attribute importance standard of the encryption attribute reduction algorithm is the secure Manhattan distance.

[0012] Further, encrypting the image to be classified includes:

[0013] The image to be classified is encrypted using the Paillier encryption algorithm.

[0014] Furthermore, decrypting the output result of the model to obtain the classification result of the image includes:

[0015] The Paillier decryption algorithm is used to decrypt the output result of the model to obtain the classification result of the image.

[0016] Furthermore, the encrypted image after attribute reduction is input into the model to obtain output results including:

[0017] Encrypting the parameters of the model using an encryption algorithm, wherein the model is an SVM model, and the parameters are w parameters and b parameters in the SVM model;

[0018] The encrypted image after attribute reduction is input into the encrypted model to obtain an output result.

[0019] Furthermore, before encrypting the image to be classified, the method further includes:

[0020] Processing the image, wherein the processing includes: denoising, enhancing and removing background;

[0021] Image features are extracted from the processed image using a gray level co-occurrence matrix.

[0022] Further, performing image feature extraction on the processed image using a gray level co-occurrence matrix method includes:

[0023] Divide each image into 4 parts, and perform the following operations on each part:

[0024] The gray-level co-occurrence matrix at 0°, 45°, 90°, and 135° is obtained, and the four statistics of contrast, correlation, energy, and angular second moment are solved through the gray-level co-occurrence matrix. The mean and variance of each statistic are then calculated to obtain an 8-dimensional feature value.

[0025] Another aspect of the present invention provides an image classification device, comprising: at least one module, wherein each module in the at least one module is used to execute the steps in the above method.

[0026] According to a third aspect of the present invention, a processor is provided for executing software, wherein the software is used to execute the above method.

[0027] A fourth aspect of the present invention provides a memory for storing software, wherein the software is used to execute the above method.

[0028] In an embodiment of the present application, the image to be classified is encrypted; the attributes of the encrypted image are simplified, wherein the number of attributes of the simplified image changes from a first number to a second number, the first number is greater than the second number, and the attributes of the image are the basis for classifying the image; the encrypted image after attribute simplification is input into the model to obtain an output result, wherein the model is a machine learning model for classification trained using multiple sets of training data; the output result of the model is decrypted to obtain the classification result of the image. The present application solves the problem of inefficiency caused by privacy protection when classifying images using machine learning technology, thereby making image classification safe and efficient. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] The drawings constituting a part of the present application are used to provide a further understanding of the present application. The illustrative embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0030] Figure 1 It is a flowchart of an image classification method according to an embodiment of the present application. DETAILED DESCRIPTION

[0031] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0032] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0033] In the prior art, it is impossible to take into account both privacy protection and rapid classification of images, resulting in low efficiency of image classification. In accordance with the above requirements, the embodiments of the present invention integrate privacy protection and image classification technology, making image classification work both safe and efficient.

[0034] In order to solve this problem, an image classification method is provided in this embodiment. Figure 1 FIG. 1 is a flow chart of an image classification method according to an embodiment of the present application, and the flow chart includes the following steps:

[0035] Step S102, encrypting the image to be classified;

[0036] Step S104, simplifying the attributes of the encrypted image, wherein the number of attributes of the simplified image changes from a first number to a second number, the first number is greater than the second number, and the attributes of the image are the basis for classifying the image;

[0037] Step S106, inputting the encrypted image after attribute reduction into the model to obtain an output result, wherein the model is a machine learning model for classification trained using multiple sets of training data;

[0038] Step S108: Decrypt the output result of the model to obtain the classification result of the image.

[0039] Through the above method, the image to be classified is encrypted and the image attributes are simplified before entering the machine learning model. The encryption algorithm first provides privacy protection for the image from the beginning of the execution of this method. The simplification process is set after the encryption process, and simplification is performed directly on the encrypted image. The image simplification process is simple and fast, and provides a guarantee for efficiently obtaining the overall classification result. Therefore, the above method can solve the needs of both image privacy protection and efficient classification, making image classification both safe and efficient.

[0040] There are many ways of simplification. In this embodiment, a more optimal processing method is provided. In this method, simplifying the attributes of the encrypted image may include: simplifying the encrypted image using an encrypted attribute simplification algorithm, wherein the attribute importance standard of the encrypted attribute simplification algorithm is the secure Manhattan distance.

[0041] In traditional reduction algorithms, Manhattan distance is usually used as the standard of attribute importance, but this standard cannot directly apply the reduction algorithm to the ciphertext. If encryption and reduction are to be considered at the same time, it is necessary to perform simplification before encryption, which will undoubtedly lead to the risk of privacy leakage during the simplification process. The secure Manhattan distance used in this preferred embodiment as the attribute importance of the reduction algorithm allows the reduction algorithm to directly calculate the distance on the ciphertext, ensuring security. At the same time, due to its outstanding feature of extremely simple calculation process, it effectively saves time and greatly improves classification efficiency.

[0042] There are many encryption algorithms. For example, in an optional implementation, encrypting the image to be classified may include: encrypting the image to be classified using a Paillier encryption algorithm. The Paillier encryption algorithm is a probabilistic public key encryption system, which is a homomorphic encryption that satisfies addition and multiplication homomorphism.

[0043] After encryption, the corresponding decryption algorithm may be used for decryption, that is, the classification result of the image obtained after decrypting the output result of the model may include:

[0044] The Paillier decryption algorithm is used to decrypt the output result of the model to obtain the classification result of the image.

[0045] In order to enhance the privacy protection of the system, the embodiment of the present invention further encrypts the machine learning model, that is, double encryption of data and model. Specifically, the encrypted image after attribute simplification is input into the model to obtain the output result including:

[0046] Encrypting the parameters of the model using an encryption algorithm, wherein the model is an SVM model, and the parameters are w parameters and b parameters in the SVM model;

[0047] The encrypted image after attribute reduction is input into the encrypted model to obtain an output result.

[0048] The encryption algorithm for the model here is also the Paillier encryption algorithm, so that the model parameters are not easy to be cracked. The above model adopts the SVM model, which has the characteristics of few parameters and fast calculation, and is particularly suitable for the requirements of the embodiment of the present invention.

[0049] Preferably, before encrypting the image to be classified, the method further comprises:

[0050] Processing the image, wherein the processing includes: denoising, enhancing and removing background;

[0051] Image features are extracted from the processed image using a gray level co-occurrence matrix.

[0052] There are many ways to extract features. For example, in an optional method, using the gray level co-occurrence matrix method to extract image features from the processed image can include: dividing each image into 4 parts on average, and performing the following operations on each part: obtaining its gray level co-occurrence matrix in the directions of 0°, 45°, 90°, and 135°, and solving the four statistics of contrast, correlation, energy, and angular second moment through the gray level co-occurrence matrix, and then calculating the mean and variance of each statistic to obtain an 8-dimensional feature value.

[0053] The above embodiment can be applied to a variety of image classifications. For example, the above embodiment can be specifically applied to classify images in the MIAS breast cancer image database, and the classification result is whether the image in the MIAS breast cancer database is an image of breast cancer. The classification of breast cancer images is used as an example for explanation.

[0054] In this embodiment, there are 322 breast X-ray images in the database, which can be divided into three categories: normal (208 images), benign (63 images), and malignant (51 images). The benign and malignant categories are classified as abnormal, that is, normal (208 images) and abnormal (114 images). Each image is 1024×1024 pixels. The abnormal symptoms of all abnormal images include calcification, mass, deformation, etc.

[0055] The above image is processed as follows:

[0056] (1) Image preprocessing: Cropping the images to a uniform size of 500 × 750, denoising the images using the median filter, and enhancing the images using histogram equalization to remove background tissue, enhance texture features, improve image quality, and enhance image usability.

[0057] (2) The gray-level co-occurrence matrix method is used to extract features from the preprocessed image: first set the directions of the gray-level co-occurrence matrix to 0°, 45°, 90°, and 135°, the distance between pixels to 1, and the four extracted statistics are contrast, correlation, energy, and angular second-order moment. Specifically, the image is first divided into four equal parts, and one of them is taken. The gray-level co-occurrence matrix in each direction is first obtained. The above four statistics are solved by the gray-level co-occurrence matrix to obtain a 4×4 matrix, where the rows represent the statistics and the columns represent the directions. Then the mean and variance of the statistics in each direction are calculated to eliminate the influence of the direction and obtain an 8-dimensional feature quantity. Similarly, each image can obtain an 8-dimensional feature quantity, and a complete image can obtain a 32-dimensional feature quantity.

[0058] (3) The feature quantities of all images are put into the database as the feature library for classification, as shown in Table 1. The first 32 dimensions are the feature quantities, and the last dimension is the classification label (1 represents diseased, -1 represents normal).

[0059] Table 1

[0060]

[0061] (4) The feature library is randomly divided into 10 parts, and 90% of them are selected as the training set to train the SVM model.

[0062] (5) Generate the public key and private key of the Paillier encryption algorithm, and encrypt the image test set corresponding to the remaining 10% of the feature library and the parameters of the SVM model (i.e., parameter w and parameter b).

[0063] Specifically, the principle of the Paillier encryption algorithm is as follows: first randomly select two large prime numbers a and b and is less than n 2A set of positive integers. Let n = a*b, λ = (a-1)(b-1), and let the function And g, n satisfy: gcd(l(g λ modn 2 ),n)=1, then the public key is pk=(n,g), and the private key is sk=λ. Then randomly select an integer For plaintext m∈Z n , the encrypted ciphertext c is c=g m *r n mod n 2 ,in, is less than n 2 And with n 2 The set of mutually prime positive integers.

[0064] In the above process, c = g m *r n mod n 2 Each element in the feature vector of the image test set and the parameter (w, b) in the SVM model are encrypted to obtain encrypted vectors and parameters.

[0065] In the above principle, a and b are randomly selected large prime numbers, and g is A positive integer randomly selected from is a set of positive integers less than n2, r is A positive integer randomly selected from is a set of positive integers less than n2 and coprime to n2, c is the ciphertext, m is the plaintext, and mod is the modulus symbol.

[0066] (6) The attributes of the image test set are reduced using an encrypted attribute reduction algorithm based on secure Manhattan distance to obtain an encrypted reduced set. The original classification feature library has 32 feature attributes. After simplification, the final feature library has 4 feature attributes, namely the 1st column, the 7th column, the 18th column and the 25th column.

[0067] Specifically, the principle of the reduction algorithm is as follows: If ed(c i ,c j ) is greater than max(ed(c i ,D),ed(c i , D)), we can think that there is a high possibility of redundancy between the two attributes ci and cj, so min(ed(c i , D), ed(c i , D)) The corresponding conditional attributes are put into the redundant attribute set C t , the remaining conditional attributes are put into the attribute set to be tested C v Calculate ed(C v, D) and ed(C, D), if the two are equal, then Cv is the new conditional attribute set, and the redundant attributes are deleted correctly. If they are not equal, the deletion fails and the deleted attributes need to be restored. Among them, C is the conditional attribute set, c i ∈C,c i (i≠j)∈C, D is the decision attribute set.

[0068] Calculate the safe Manhattan distance between feature vectors. The safe Manhattan distance is denoted as ed(|XY|) and is defined as: where x i ,y i Represent the elements in the feature vector, and then perform attribute reduction on the feature set based on the safe Manhattan distance.

[0069] In the above principle, max represents the larger of the two, and min represents the smaller of the two. I , c j ) is the conditional attribute c i and c j The safe Manhattan distance between i and c j They represent the i-th and j-th attributes in the conditional attributes, ed(c i , D) is the conditional attribute c i The safe Manhattan distance between the decision attribute D, Ct is the redundant attribute set, and Cv is the attribute set to be tested. |XY| represents the distance between two feature vectors, X and Y are two feature vectors. i and i They respectively represent the feature vectors corresponding to two attributes in the feature vector of the i-th row in the feature library (i.e., the i-th sample), n represents the number of samples, and E(X) represents the result of X after Paillier encryption.

[0070] (7) The encrypted reduced set is input into the encrypted SVM model to obtain the encrypted classification results, as shown in Table 2 below.

[0071] Table 2

[0072]

[0073] (8) Use the private key of the Paillier encryption algorithm to decrypt the above ciphertext results, and SVM outputs the plaintext classification results.

[0074] Specifically, based on the characteristics of the Paillier encryption algorithm, in this embodiment, the decryption formula encrypted with Paillier is The classification result is obtained, where c is the ciphertext and m is the plaintext. The encrypted parameters are recorded as E(w) and E(b), and the encrypted data set is recorded as x. According to the additive homomorphism of homomorphic encryption, Enc(m1)×Enc(m2)=Enc(m1+m2)modn 2 , through y = wx + b, the final result is calculated by safe multiplication, that is, E(y) = E(wx + b) = E(wx) * E(b). The safe multiplication is that given E(w) and E(x), the formula w*x = (wr w )*(xr x )+w*r w +x*r x -r w *r x Calculate E(w*x).

[0075] Take the first record in Table 2 as an example to explain it in detail.<phe.paillier.EncryptedNumberobjectat0x00000294CF91D940> The decryption result is -0.9999296269252584, which is less than 0. According to the definition of the classification label in the aforementioned SVM model, y is -1, which means normal.

[0076] The above image classification method was tested on the ionosphere and heart data sets in UCI, and the classification accuracy and encryption and decryption time of SVM and the embodiment of the present invention were compared. The experiment shows that sacrificing at most 3.7% of the accuracy or even keeping the accuracy unchanged can reduce the encryption and decryption time by several times or even dozens of times.

[0077] In response to security issues in the image classification process, the data to be classified in the present invention is used as the input of the classifier in ciphertext form, which ensures the security of the data during transmission. When the classifier performs classification, according to the additive homomorphism of Paillier encryption, the ciphertext can be directly calculated to obtain the encryption result, and then decrypted with a private key. The decrypted result obtained is consistent with the result obtained by directly calculating with plaintext.

[0078] The privacy-preserving image classification method provided by the above-mentioned embodiment has the following advantages over the prior art: 1. The present invention takes into account the efficiency of traditional classifiers in the classification process. It first uses an encrypted attribute reduction algorithm based on the secure Manhattan distance to reduce the attributes of the encrypted image test set to obtain a reduced set. Unlike the traditional Manhattan distance, the secure Manhattan distance is used as a standard to measure the importance of the attribute. The distance can be calculated directly on the ciphertext. The biggest advantage is that the calculation is simple, time-saving, and the security is good, which effectively improves the efficiency of the classifier. 2. The present invention takes into account the security of the image. When using SVM to classify the image, the Paillier encryption algorithm is first used to encrypt the image set and the model parameters. The homomorphism and secure multiplication protocol of Paillier encryption can enable the SVM to directly classify the encrypted image set, ensuring the security of the image.

[0079] Compared with image databases with large data volumes, the technical solution in this embodiment is more suitable for situations where the data volume is small and the possibility of attribute feature redundancy is high. An encrypted attribute reduction algorithm based on secure Manhattan distance is provided to simplify the attributes of the feature library to solve the problem of the classification process taking too long. At the same time, the additive homomorphism of Paillier encryption is used to solve the privacy leakage problem existing in the existing image classification method.

[0080] In this embodiment, an electronic device is further provided, including a memory and / or a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the image classification method in the above embodiment.

[0081] These computer programs can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps of the functions specified in one or more boxes can be implemented by different modules corresponding to different steps. In this embodiment, an image classification device is also provided, including: at least one module, wherein each module in the at least one module is used to execute the steps in the above-mentioned image classification method.

[0082] The above program can be run in the processor, or it can be stored in the memory (or computer-readable medium), which includes permanent and non-permanent, removable and non-removable media. Information storage can be achieved by any method or technology. Information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined in this article, computer-readable media does not include temporary computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0083] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included within the scope of the claims of the present application.

Claims

1. An image classification method, characterized in that: include: The image to be classified is encrypted using the Paillier encryption algorithm; wherein, before encrypting the image to be classified, the method further includes: Processing the image, wherein the processing includes: denoising, enhancing and removing the background; extracting image features from the processed image using a gray-level co-occurrence matrix; wherein extracting image features from the processed image using a gray-level co-occurrence matrix method includes: dividing each image into 4 parts on average, performing the following operations on each part: obtaining its gray-level co-occurrence matrix in the directions of 0°, 45°, 90° and 135°, and solving four statistics of contrast, correlation, energy and angular second-order moment through the gray-level co-occurrence matrix, and then calculating the mean and variance of each statistic to obtain an 8-dimensional feature quantity; Simplifying the attributes of the encrypted image, wherein the number of attributes of the simplified image changes from a first number to a second number, the first number is greater than the second number, and the attributes of the image are the basis for classifying the image; wherein simplifying the attributes of the encrypted image comprises: simplifying the encrypted image using an encrypted attribute simplification algorithm, wherein the attribute importance standard of the encrypted attribute simplification algorithm is the secure Manhattan distance; The reduction algorithm is as follows: if ed(c i ,c j ) is greater than max(ed(c i ,D),ed(c i ,D)), it is believed that there is a high probability of redundancy between the two attributes ci and cj, so min(ed(c i ,D),ed(c i ,D)) The corresponding conditional attributes are put into the redundant attribute set C t , the remaining conditional attributes are put into the attribute set to be tested C v ; Calculate ed(C v ,D) and ed(C,D), if the two are equal, then Cv is the new conditional attribute set, and the redundant attributes are deleted correctly. If they are not equal, the deletion fails and the deleted attributes need to be restored; where C is the conditional attribute set, c i ∈C, c i (i≠j)∈C, D is the set of decision attributes; Calculate the safe Manhattan distance between feature vectors. The safe Manhattan distance is denoted as ed(|XY|) and is defined as: where x i ,y i Represent the elements in the feature vector, and then perform attribute reduction on the feature set based on the safe Manhattan distance; Among them, max represents the larger of the two, and min represents the smaller of the two; ed(c I ,c j ) is the conditional attribute c i and c j The safe Manhattan distance between i and c j They represent the i-th and j-th attributes in the conditional attributes, ed(c i ,D) is the conditional attribute c i The safe Manhattan distance between the decision attribute D, Ct is the redundant attribute set, Cv is the attribute set to be tested; |XY| represents the distance between two feature vectors, X and Y are two feature vectors respectively; x i and i They represent the feature vectors corresponding to two attributes in the feature vector of the i-th row in the feature library, n represents the number of samples, and E(X) represents the result of X after Paillier encryption; Inputting the encrypted image after attribute reduction into the encrypted model to obtain an output result, wherein the model is a machine learning model for classification trained using multiple sets of training data; wherein the encrypted model is obtained by encrypting the parameters of the model using an encryption algorithm, wherein the model is an SVM model, and the parameters are w parameters and b parameters in the SVM model; The Paillier decryption algorithm is used to decrypt the output result of the model to obtain the classification result of the image.

2. An image classification device, characterized in that: include: At least one module, wherein each module in the at least one module is used to execute the steps in the method of claim 1.

3. A processor for executing software, characterized in that: The software is used to execute the method of claim 1.

4. A memory for storing software, characterized in that: The software is used to execute the method of claim 1.

Images