A privacy-protected outsourced image feature extraction and classification method

Abstract

In view of the problems that some existing outsourcing feature extraction methods cannot prevent face information leakage or the face recognition operation after encrypting the face is very time-consuming, a privacy protection outsourcing calculation method using cloud for image feature extraction and feature classification is provided, including the following steps: step one, converting the RGB image to YCbCr color space at the client end to obtain the pixel value matrix of R, B, Cb and Cr color channels; step two, using a one-time pad key to encrypt the matrix in step one, and sending the encrypted matrix to the cloud; step three, the cloud extracts the encrypted matrix in step two, inputs a neural network or an integrated classifier model for training, and obtains a result; and step four, the cloud returns the classification result to the client end.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of computer encryption, and particularly relates to a privacy-protected outsourced image feature extraction and classification method. BACKGROUND

[0002] Computer vision technology helps people to deal with many automatic visual recognition tasks, such as identification of real and fake images, which is often achieved by feature extraction of images and then training of a classifier model using machine learning technology. The feature extraction algorithm needs to perform a large number of statistical calculations and construct high-dimensional features to improve the detection performance. For a large number of images, due to the limited storage space and computing capacity of the client, it is difficult to extract features, train and classify on the local computer. Therefore, it is very important for the local computer to reduce the computing load without reducing the detection performance of feature extraction.

[0003] The huge storage space and powerful computing capacity of the cloud server can enable the devices with limited computing capacity to complete complex calculations. For users and enterprises, the server can not only store various text files, but also store various multimedia files (images, voice, video, etc.). The image often contains a lot of privacy or sensitive information. A large number of raw images uploaded to the cloud or sent to the third party for analysis and identification services also face the risk of privacy leakage. If the data privacy is not protected, these highly sensitive information will be illegally used by the third party or attacker.

[0004] The data owner should encrypt the data before uploading it to the cloud for calculation, but the encrypted data often makes it difficult to use. Therefore, a safe outsourced image feature extraction and classification scheme is needed to solve the problem of image owners needing to use the cloud for image classification.

[0005] Some existing outsourcing feature extraction methods often have some problems: the privacy protection face recognition scheme based on deformation can only protect the visual content of the face to provide visual security, but cannot prevent the leakage of facial information, and will cause the accuracy of face recognition to be reduced; due to heavy cryptographic techniques such as homomorphic encryption algorithm, it is very time-consuming to use them to encrypt the face and perform face recognition operation, and the outsourcing method based on homomorphic encryption often needs large communication complexity and calculation complexity, and the calculation cost and communication cost are high, and the encryption efficiency is low; the method based on matrix calculation and feature decomposition is not safe, and the malicious server can obtain the eigenvalue and eigenvector by calculating the greatest common divisor of the encrypted data, and the client and the cloud often need to interact three times, that is, the client needs to perform three times of encryption and decryption calculation to realize it; the outsourcing method based on a certain classical image feature is often used to solve a specific problem and cannot be applied to other aspects; some outsourcing computing schemes discard part of the features or reduce their precision, resulting in that the extracted features are not exactly the same as the original features. SUMMARY

[0006] In view of the problems that some existing outsourcing feature extraction methods cannot prevent the leakage of facial information or the face recognition operation after encrypting the face is very time-consuming, a privacy protection outsourcing computing method using cloud for image feature extraction and feature classification is proposed.

[0007] The method of the application comprises the following steps:

[0008] Step one, converting the RGB image to YCbCr color space on the client side to obtain the pixel value matrix of its R, B, Cb and Cr color channels;

[0009] Step two, encrypting the matrix of step one using a one-time pad key, and sending the encrypted matrix to the cloud;

[0010] Step three, the cloud extracts the encrypted matrix of step two, inputs a neural network or an integrated classifier model for training, and obtains the result;

[0011] Step four, the cloud returns the classification result to the client.

[0012] Through the response of the cloud, the client can use less work than directly calculating the function to verify whether the calculation is correctly performed, and since the extracted features and the features of the unencrypted image remain consistent, the accuracy of image classification remains unchanged, and the communication efficiency is greatly improved.

[0013] As a preferred, in the image encryption of step two, the RGB image is first preprocessed and converted to YCbCr color space, then the last two channels of the RGB image and the last two channels of the YCbCr image are extracted separately, and combined into an image matrix ORI_IMG to be encrypted{C} , where C ∈ {R, B, Cb, Cr}.

[0014] Let the image size be a x b, and the block size be h x h, so the image of each channel is divided into s = (a x b) / (h x h) different small blocks where C ∈ {R, B, Cb, Cr}, i ∈ [1, a / h], j ∈ [1, b / h], h must be the common factor of a and b. Since the matrix size of each color channel of the image is the same, they all have the same number of small blocks, and a sequence matrix KEY_1 of the same size as the block is randomly generated for each small block at the same index position i,j,m,n As the key, where i ∈ [1, a / h], j ∈ [1, b / h], m, n ∈ [1, h], the matrix is a random permutation of 1 to the size of the small block, that is, a random permutation sequence matrix of 1 to h x h, which has:

[0015]

[0016]

[0017]

[0018] where, indicates down evidence, mod indicates the remainder operation, indicates the encrypted result of the (m, n) position in the matrix of the (i, j) position of the c color channel after this round of encryption, and p, q are used as encryption matrix indexes, the same key is used for small blocks at the same index position in each color channel, and different keys are used for small blocks at different index positions, and each key is a randomly generated sequence, and the generated small block is

[0019] Then, for the s small blocks of each color channel a sequence matrix key KEY_2 of size s is generated i,j , where i ∈ [1, a / h], j ∈ [1, b / h], the matrix is a random permutation of 1 to s, and the generation method is the same as KEY_1 i,j,m,n , where i ∈ [1, a / h], j ∈ [1, b / h], the matrix is a random permutation of 1 to s, and the generation method is the same as KEY_1

[0020]

[0021]

[0022]

[0023] where, The encryption result of the matrix representing the (i, j) position of the c color channel, taking r, s as the index of the encryption block, each color channel uses the same key, and the size is a x b. Finally, the encrypted image ENC IMG with a size of 4 x a x b is obtained {C} , wherein c ∈ {R, B, Cb, Cr}.

[0024] The encryption method proposed in the application is applicable to images of different sizes, and different block sizes can be set for image encryption. The size of the block can be evenly divided by the size of the image, that is, the larger the block, the faster the encryption speed. Finally, the encrypted image ENC IMG c is sent to the cloud by the client.

[0025] As a preferred embodiment, the feature extraction in step three is described as follows: after receiving the encrypted image ENC IMG c from the client, the cloud can extract the features therefrom. For an encrypted image with a size of 4 x a x b, the following is calculated for each two different channels:

[0026]

[0027] wherein C1 and C2 represent the encrypted images of different color channels, because the pixel size of the image is between 0 and 255, therefore m, n ∈ [0, 255], and the size of each ENC F is 256 x 256. There are four color channels, so six encrypted feature matrices can be extracted, and the six different feature matrices are combined to obtain the final image feature ENC MATRIX, which has a size of 6 x 256 x 256.

[0028] The image is encrypted by the client and sent to the cloud. The key is one-time pad and does not need to be stored. The cloud and malicious attackers cannot obtain any information of the image by analysis, and the cloud can only extract the features according to the specified algorithm. The size of the feature is 393216 dimensions, which is difficult to store and calculate locally. Transferring it to the cloud for calculation will not expose any information.

[0029] As a preferred embodiment, the image classification in step three has the following characteristics: since the encryption only changes the positional relationship between the pixel points of each color channel of the image, and does not change the pixel values, the features extracted by the cloud after encryption by the client are the same as the features directly extracted from the original image, so the image classification process is also performed by the cloud. The features extracted by the application are suitable for most machine learning classifiers, including SVM, Bayesian classifier, Fisher linear discriminator, and neural network classifier, all of which can achieve very high accuracy. Therefore, the features can be input into any classifier for training and detection, and the classification result of the image can be obtained.

[0030] Finally, the cloud returns the classification result of the image to the client, only needs to send "0" or "1", wherein "0" represents that the cloud judges the result to be a false image, and "1" represents that the cloud judges it to be a real image. Therefore, for the client, only needs to send the image to the cloud after encryption to obtain the image classification result, without consuming too many local resources, the outsourcing extraction and classification of image features

[0031] The substantial feature of the present application is that the present application realizes a simple and efficient privacy protection outsourcing image feature extraction and classification method, which uses a one-time one-key encryption method to encrypt the image. The one-time one-key encryption is theoretically unbreakable, so the encryption method is safe. The encryption method is used for images of any size, and the size of the block can also be changed at will. In the scheme proposed by the present application, the client only needs to encrypt the image, send the encrypted image to the cloud, and the cloud will return the classification result. In this process, the client only needs to perform a little calculation, and the encryption does not occupy extra space; the size of the block used in the encryption can be changed, and there is no need to store the key and decrypt, which greatly improves the efficiency of the client and reduces the consumption of the client; the cloud extracts features from the encrypted image, and the features are the same as those extracted from the unencrypted image, so the training and classification can be directly performed in the cloud, and the classification accuracy is the same as that of direct classification without encryption, which is different from the feature extraction of other outsourcing computing schemes but cannot perform feature training; in addition, the client only needs to send the encrypted image, and the cloud only needs to return the classification result, so the communication cost is greatly reduced. BRIEF DESCRIPTION OF DRAWINGS

[0032] Figure 1 Basic flowchart of the present application

[0033] Figure 2 Original R channel image and encrypted R channel image DETAILED DESCRIPTION

[0034] The technical solutions of the present application will be further specifically described below by means of specific embodiments and in combination with the drawings.

[0035] Example 1

[0036] The application is a privacy protection outsourcing computing method for image feature extraction and feature classification using cloud, and the main process is as follows: first, the RGB image is converted to YCbCr color space on the client side to obtain the pixel value matrix of R, B, Cb and Cr color channels, then the matrix is encrypted respectively, the key is one-time one-key, the encrypted matrix is sent to the cloud, the cloud extracts the features, then inputs the neural network or integrated classifier model for training, the trained model can directly classify the new image sent by the client without decryption, finally, the cloud returns the classification result to the client. Through the response of the cloud, the client can use less work than directly calculating the function to verify whether the calculation is correctly executed, and since the extracted features and the features of the unencrypted image remain consistent, the accuracy of image classification remains unchanged, and the communication efficiency is greatly improved. The basic process of the application is shown in Figure 1 .

[0037] The image encryption, feature extraction and image classification modules, wherein the image encryption is performed on the client side, the feature extraction and image classification are performed on the cloud side, and the specific description is as follows

[0038] 1) Image encryption

[0039] Most of the images we can see in life are based on RGB color space RGB images, so the RGB image is first preprocessed and converted to YCbCr color space, then the last two channels of the RGB image and the last two channels of the YCbCr image are extracted separately to form the encrypted image matrix ORI_IMG {C} , wherein C is in {R, B, Cb, Cr}.

[0040] Assuming that the image size is a x b and the block size is h x h, the image of each channel is divided into s = (a x b) / (h x h) different small blocks , wherein C is in {R, B, Cb, Cr}, i is in [1, a / h], j is in [1, b / h], and h must be a common factor of a and b. Since the matrix size of each color channel of the image is the same, they all have the same number of small blocks, and a sequence matrix KEY_1 i,j,m,n of the same size as the block is randomly generated as the key, wherein i is in [1, a / h], j is in [1, b / h], m and n are in [1, h], the matrix is a random arrangement of 1 to the size of the small block, that is, a random arrangement sequence matrix of 1 to h x h, and has

[0041]

[0042]

[0043]

[0044] wherein, represents down trace, mod represents remainder operation, represents the encrypted result of the (m, n) position in the matrix of the (i, j) position of the c color channel after the round of encryption, p and q are used as the index of the encryption matrix, the same key is used for the small block of the same index position in each color channel, and different keys are used for the small block of different index positions, each key is a randomly generated sequence, and the generated small block is Then, for the s small blocks of each color channel A sequence matrix key KEY_2 with a size of s is generated i,j , wherein i is in [1, a / h], j is in [1, b / h], the matrix is a random arrangement of 1 to s, and the generation manner is the same as KEY_1 i,j,m,n , and there is:

[0045]

[0046]

[0047]

[0048] wherein, represents the encryption result of the matrix of the (i, j) position of the c color channel, r and s are used as the index of the encrypted small block, the same key is used for each color channel, and the size is a*b. Finally, an encrypted image ENC_IMG with a size of 4*a*b is obtained {C} , wherein c is in {R, B, Cb, Cr}.

[0049] The encryption method provided by the application is suitable for images of different specifications and sizes, different block sizes can be set for image encryption, the block size can be evenly divided by the size of the image, and the larger the block, the faster the encryption speed. As shown in the following formula, the image size is taken as 1024*1024, the block size is taken as 64*64, and the R channel image is encrypted Figure 2 , the left side is the original R channel image, and the right side is the encrypted R channel image. The encrypted image does not expose any information of the original image. Finally, the encrypted image ENC_IMG c is sent to the cloud by the client.

[0050] 1) Feature extraction

[0051] The cloud receives the encrypted image ENC_IMG c sent by the client, and can extract features therefrom. For an encrypted image with a size of 4*a*b, the following is calculated for each two different channels:

[0052]

[0053] wherein C1, C2 represent the encrypted images of different color channels, since the pixel size of the image is between 0 and 255, thus m, n ∈ [0, 255], the size of each ENC_F is 256 x 256. There are four color channels in total, thus six encrypted feature matrices can be extracted, and the six different feature matrices are combined to obtain the final image feature ENC_MATRIX, which has a size of 6 x 256 x 256.

[0054] The image is sent to the cloud after being encrypted by the client, the key is one-time and does not need to be stored, the cloud and malicious attackers cannot obtain any information of the image by analysis, and the cloud can only extract features according to the specified algorithm. The size of the feature is 393216 dimensions, which is difficult to store and calculate locally, and transferring it to the cloud for calculation will not expose any information.

[0055] 2) Image classification Since the encryption only changes the positional relationship between the pixel points of each color channel of the image, and does not change the pixel values, the features extracted by the cloud after being encrypted by the client are the same as the features directly extracted from the original image, thus the image classification process is also performed by the cloud. The features extracted by the present application are suitable for most machine learning classifiers, such as SVM, Bayesian classifier, Fisher linear discriminator, neural network classifier (Xception, Capsule, etc.), which can achieve very high accuracy. Therefore, the features can be input into any classifier for training and detection, and the classification result of the image can be obtained. Finally, the cloud returns the classification result of the image to the client, only needs to send "0" or "1", wherein "0" represents that the result judged by the cloud is a fake image, and "1" represents that the cloud judges it as a real image. Therefore, for the client, only needs to encrypt the image and send it to the cloud to obtain the image classification result, without consuming too many local resources, thus the outsourcing extraction and classification of image features can be achieved.

Claims

1. A privacy-preserving outsourced image feature extraction and classification method, characterized in that, The method comprises the following steps: Step one, convert the RGB image to YCbCr color space on the client side to obtain the pixel value matrix of its R, B, Cb, Cr color channels; Step two, encrypt the matrix of step one using a one-time pad key, and send the encrypted matrix to the cloud; Image encryption includes the following sub-steps: Sub-step two, in step one, the RGB image is converted to YCbCr color space, the last two channels of the RGB image and the last two channels of the YCbCr image are extracted separately, and combined into an image matrix ORI IMG to be encrypted {C} , wherein C ∈ {R, B, Cb, Cr}; Sub-step two, assuming the image size is a x b, the block size is h x h, and s = (a x b) / (h x h) different small blocks are obtained wherein C ∈ {R, B, Cb, Cr}, i ∈ [1, a / h], j ∈ [1, b / h], and h is a common factor of a and b; Sub-step two, randomly generate a sequence matrix KEY_1 with the same size as the block for each small block at the same index position i,j,m,n As the key, where i∈[1, a / h], j∈[1, b / h], m, n∈[1, h], then have wherein, denotes a floor function, mod denotes a modulo operation, denotes the encrypted result in the (m, n) position of the matrix of the (i, j) position of the c color channel after encryption; Sub-step two, use p and q as the index of the encrypted matrix, use the same key for each color channel corresponding to the small block at the same index position, and use different keys for the small block at different index positions, and each key is random; Sub-step 25, for each color channel, s small blocks Generate a sequence matrix key KEY_2 of size s i,j , where i ∈ [1, a / h], j ∈ [1, b / h], KEY_2 i,j is a random permutation of 1 to s, and is generated in the same way as KEY_1 i,j,m,n Then, we have: wherein, represents the encryption result of the matrix of the (i,j) position of the c color channel; Sub-step two six, take r, s as the index of the encrypted small block, each color channel uses the same key, and the size is a x b; finally get the encrypted image ENC IMG with the size of 4 x a x b {C} where c e {R, B, Cb, Cr}, the generated small block is Step three, the cloud extracts the encrypted matrix of step two, inputs the neural network or integrated classifier model for training, and obtains the result; The extraction of the encrypted matrix of step two includes: Calculate for each of the two different channels: Where C1, C2 represent the encrypted images of different color channels, because the pixel size of the image is between 0 and 255, so m, n ∈ [0, 255], the size of each ENC_F is 256 × 256; There are four color channels, so six encrypted feature matrices can be extracted, and the six different feature matrices are combined to obtain the final image feature ENC_MATRIX, which has a size of 6 × 256 × 256; Step four, the cloud returns the classification result to the client.

Images