A privacy-preserving image classification method and system supporting a trusted execution environment

Abstract

The application provides a privacy protection image classification method and system supporting a trusted execution environment, and comprises the following steps: a model provider is authenticated with a trusted execution environment (TEE) and shares a first secret key and model parameters; the TEE generates a system public key (PK) and a private key (SK) by using a full homomorphic encryption algorithm; the model provider encrypts the model parameters by using the PK obtained from the TEE and uploads the model parameters to a cloud server; an image data provider is authenticated with the TEE and shares a second secret key, the image data provider encrypts image data by using the PK obtained from the TEE and uploads the image data to the cloud server; the cloud server performs a ciphertext image classification prediction by using the model parameters and obtains an encrypted image classification prediction result; the cloud server feeds back the encrypted result to the TEE, the TEE decrypts the result by using the SK and obtains the result; the TEE encrypts the result by using the second secret key and sends the result to the image data provider, and the image data provider decrypts the result and obtains an original image prediction result. The application provides core technical support for realizing the availability and invisibility of privacy data.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of privacy protection of machine learning, in particular to a privacy protection image classification method and system supporting a trusted execution environment. BACKGROUND

[0002] In machine learning as a service (MLaaS), a model provider provides a trained model, such as a convolutional neural network model, to an image data provider through a cloud server, and the image data provider needs to share its image data with the cloud server, and the model provider needs to share its model with the cloud server. Traditional machine learning algorithms need to be calculated on plaintext data, and face the risk of data leakage and privacy infringement.

[0003] An activation layer is an important component of a convolutional neural network model, and its non-linear characteristics allow the model to capture complex patterns from input data. Depending on the application, various typical activation functions can be used, such as the formula of a ReLU activation function: f(x) = max(0, x).

[0004] Fully homomorphic encryption (FHE) belongs to the field of cryptography. Since fully homomorphic encryption supports arbitrary calculations on ciphertext without decryption, it can immediately solve the problem of data privacy and security, and has great application demand. For example, in a cloud environment, a user encrypts data and stores it in the cloud. Since the data is encrypted, the cloud cannot obtain the content of the data, thereby ensuring the privacy of the data. In addition, since it is fully homomorphic encryption, the cloud can perform arbitrary calculations on the ciphertext data. In summary, fully homomorphic encryption not only protects data through encryption, but also does not lose computing power.

[0005] However, since fully homomorphic encryption algorithms, such as the CKKS encryption algorithm, only support homomorphic addition and homomorphic multiplication operations, but the ReLU activation function described above is a non-linear operation, the fully homomorphic encryption scheme cannot be directly applied to the convolutional neural network model. Therefore, how to apply fully homomorphic encryption to machine learning while ensuring the performance of the convolutional neural network model and improving the security of image data classification is a difficult problem that needs to be solved, and the fully homomorphic encryption algorithm requires a large number of homomorphic addition and homomorphic multiplication operations, which is also a problem that cannot be ignored due to its low computational efficiency. SUMMARY

[0006] The present application provides a privacy protection image classification method and system supporting a trusted execution environment to solve the problems existing in the prior art. The technical solutions provided by the present application are as follows:

[0007] In one aspect, a privacy-protected image classification method supporting a trusted execution environment is provided, and the method comprises:

[0008] S1, a model provider authenticates a trusted execution environment (TEE), and after successful authentication, the model provider shares a first secret key and model parameters of a pre-trained convolutional neural network model with the TEE. The TEE sets security parameters and homomorphic capacity of a fully homomorphic encryption algorithm according to the model parameters, and randomly generates an asymmetric pair of a system public key (PK) and a system private key (SK) by using the fully homomorphic encryption algorithm.

[0009] S2, the TEE securely sends the PK to the model provider by using the first secret key. The model provider encrypts the model parameters by using the PK and the fully homomorphic encryption algorithm, and uploads them to a cloud server.

[0010] S3, an image data provider authenticates the TEE, and after successful authentication, the image data provider shares a second secret key with the TEE. The TEE securely sends the PK to the image data provider by using the second secret key. The image data provider encrypts image data by using the PK and the fully homomorphic encryption algorithm, and uploads them to the cloud server.

[0011] S4, the cloud server performs image classification prediction in an encrypted domain on the encrypted image data by using a pre-trained convolutional neural network model according to the model parameters, to obtain encrypted image classification prediction results.

[0012] S5, the cloud server feeds back the encrypted image classification prediction results to the TEE. The TEE decrypts the original image prediction results by using the SK and the fully homomorphic encryption algorithm.

[0013] S6, the TEE encrypts the original image prediction results by using the second secret key, and sends them to the image data provider. The image data provider decrypts the original image prediction results by using the second secret key.

[0014] Optionally, the fully homomorphic encryption algorithm is a CKKS encryption algorithm, and in S3, the image data provider encrypts the image data by using the PK and the fully homomorphic encryption algorithm, specifically comprising:

[0015] The same pixel point of multiple original images is packed into the ciphertext in the corresponding position of the ciphertext matrix to encrypt the original image matrix img[channels][length][width] in the floating-point number field to obtain a ciphertext image matrix encrypted_img[channels][length][width] of the same size as the original image matrix.

[0016] Optionally, the S4 specifically comprises:

[0017] The object processed by the convolutional neural network model is an image matrix of the floating-point number type, and the encrypted ciphertext image is an image matrix of the ciphertext polynomial ring type, which cannot be directly used as the input of the convolutional neural network model. The network layer of the convolutional neural network model is adjusted to apply the image classification prediction in the ciphertext field. The network layer of the convolutional neural network model is divided into a polynomial layer and a non-polynomial layer. The operation of the polynomial layer on data is only addition and multiplication, and the polynomial layer can be expressed in the form of a polynomial. The network layer that cannot be expressed in the form of a polynomial is referred to as a non-polynomial layer.

[0018] The adjustment of the polynomial layer comprises:

[0019] It is assumed that the plaintext image or the intermediate result is represented as IMG, and the ciphertext form corresponding thereto is represented as ENC_IMG. The operation of the polynomial layer of the convolutional neural network model on the image in the floating-point number field comprises linear operation and Nth power IMG N in two forms.

[0020] For the form of linear operation: the product of the element IMG i in the corresponding vector or matrix and the corresponding parameter filter i is added to the bias value bias, and in the prediction in the ciphertext field, the expected result is the encrypted value of the result in the floating-point number field. According to the properties of homomorphic addition and homomorphic multiplication in homomorphic encryption, the operation in the ciphertext field is evolved as follows:

[0021]

[0022] The parameter IMG N in the floating-point number field to be multiplied by the pixel point corresponding to the image is encrypted first, and then multiplied with the pixel point of the ciphertext image by using homomorphic multiplication. Finally, the values are added by using homomorphic addition to obtain the encrypted value of the corresponding result in the floating-point number field, while ensuring that the information of the original image is not leaked.

[0023] For the form of Nth power: the calculation of IMG NThen, the transformation takes the following form:

[0024]

[0025] The encrypted ciphertext image matrix only needs to be multiplied N times using homomorphic multiplication to obtain the corresponding encrypted value;

[0026] For adjustments to the non-polynomial layer:

[0027] The ReLU activation function f(x) = max(0,x) of the activation layer in a convolutional neural network model can be approximated by a polynomial as f(x) = 0.1500 + 0.5012X + 0.2981X. 2 -0.0004X 3 -0.0388X 4 Then, adjustments are made using a polynomial layer adjustment method.

[0028] Optionally, for consecutive multiplications of ciphertext, a symmetrical multiplication method is used to reduce the cost of multiplication levels. The symmetrical multiplication method is as follows:

[0029] Suppose we need to calculate about the ciphertext x i The function f(x1,x2,…,x) n )=x1·x2·...·x n (where n is a power of 2), first calculate x1·x2, x3·x4, ..., x n-1 ·x n Then calculate (x1·x2)·(x3·x4),...,(x n-2 ·x n-1 )·(x n-1 ·x n The calculation is performed recursively, and finally (x1·x2)·(x3·x4)...·(x n-1 ·x n It requires a total of log2n multiplication levels, which greatly reduces the cost of multiplication levels.

[0030] Optionally, for each polynomial layer that only requires linear operations, multiple parameter multiplications can be optimized into a single matrix multiplication. Based on this, the prediction stage can use parameter merging to fuse consecutive linear polynomial layers for optimization, thereby reducing the overhead of multiplication levels, including:

[0031] 1) Convolutional layer - batch normalized layer:

[0032] Output of convolutional layer It is the input of the batch normalization layer, and the output of each batch normalization layer. The following forms of transformation exist:

[0033]

[0034] In processing the network layer, the corresponding and are encoded, and the convolution layer and the batch normalization layer are merged into one layer, which only needs one multiplication level consumption;

[0035] 2) Convolution layer-average pooling layer:

[0036] The output of the convolution layer is the input of the average pooling layer, and the output of each average pooling layer has the following form of transformation:

[0037]

[0038] In processing the network layer, the corresponding parameters of the convolution layer are divided by the number N of average pooling in advance for encoding, that is, the and are encoded, and only one multiplication is performed, which only needs one multiplication level consumption, and the merged layer is equivalent to calculating times of the results of the convolution layer before merging, and adding the corresponding elements for the average pooling layer to obtain the result.

[0039] Optionally, the function containing an N-th polynomial in the polynomial layer can be expressed in the form of , corresponding to N=2 k -1(k∈Z∩[0,+∞)) when the highest order a N x N can be split into log2(N+1) multiplication level consumptions by using a symmetric calculation form, and when N is exactly 2 k (k∈Z∩[0,+∞)) because the coefficient a N introduces an additional multiplication level consumption, for this form of problem, the parameter fusion method is adopted, and the coefficient a N is transferred to other polynomial layers of linear operation for merged calculation, thereby reducing the multiplication level consumption, and the polynomial approximation function of the activation layer ReLU activation function is f(x) = ax 4 +bx 3 +cx 2 +dx+e, which is the case of N=4, the output of the full connection layer The output of the activation layer is brought into the output of the full connection layer for calculation, and the following form is obtained:

[0040]

[0041] When processing the activation layer, the highest order coefficient of the approximation function is normalized:

[0042]

[0043] The fully connected layer is multiplied by the filter i The consumption of one multiplication level is reduced by using symmetric multiplication in the activation layer and only one ciphertext-ciphertext multiplication in the fully connected layer.

[0044] Alternatively, for ciphertext matrix multiplication, the ciphertext matrices to be multiplied are multiplied by row and column masks respectively to obtain ciphertext matrices containing only the i-th row and i-th column elements, and then the ciphertext homomorphic addition operation is performed by rotating by row and by column respectively to perform row duplication and column duplication operations, and the ciphertext matrix is filled, and finally the slots of the matrix are multiplied and added to reduce the number of ciphertext multiplications with the largest calculation overhead.

[0045] Alternatively, the ciphertext matrix multiplication specifically comprises:

[0046] The ciphertext matrices to be multiplied are multiplied by row and column masks respectively to obtain ciphertext matrices containing only the i-th row and i-th column elements, comprising:

[0047] Two binary vectors Π and Ψ are defined, which are plaintext vectors containing only {0,1} elements, used to manage the rows and columns of the matrix using scalar multiplication, referred to as row and column masks. For an integer l, the binary vectors In the slot with index k1+l·k2, it is equal to 1, and In the slots with index from k1 to k1+k2-1, it is equal to 1, and in all other slots, both vectors are zero, which is represented as follows:

[0048]

[0049]

[0050] Suppose that two n×n-dimensional image matrices are encoded and encrypted into ciphertext matrices ct A , ct B , respectively, for 0≤i Return a ciphertext where all slots are equal to 0 except the slot in position (i+l·n) which is equal to the corresponding ct A , and for 0≤i Return a ciphertext where all slots are equal to 0 except the slots i·n to (i+1)·n-1 of the i-th row which are equal to the corresponding ctB The rest of the slots are equal to 0, and through row and column masks, the 2 ciphertext matrices are expanded to 2n ciphertext matrices after masking;

[0051] Then, by rotating respectively by row and by column, and performing ciphertext fully homomorphic addition, the row and column replication operations are performed, including:

[0052] Using algorithm Respectively on the n matrix ciphertexts after column masking Respectively along the column, the matrix is filled with this column by rotating left by column and performing ciphertext fully homomorphic addition, to obtain matrix Using algorithm Respectively on the n matrix ciphertexts after row masking Respectively along the row, the matrix is filled with this row by rotating up by row and performing ciphertext fully homomorphic addition, to obtain matrix Finally, the slots of the matrix are multiplied and added, including:

[0053] The calculation of the ciphertext of the multiplication of the two encrypted matrices is converted to point-to-point matrix multiplication And, Obtaining matrix C as the calculation result.

[0054] On the other hand, a system for image classification using the above privacy protection image classification method supporting the trusted execution environment is provided, and the system comprises a trusted execution environment TEE, a model provider, an image data provider and a cloud server;

[0055] The model provider authenticates the TEE, and after successful authentication, the model provider shares a first secret key and model parameters of a pre-trained convolutional neural network model with the TEE. The TEE sets security parameters and homomorphic capacity of a fully homomorphic encryption algorithm according to the model parameters, and randomly generates an asymmetric pair of system public key PK and system private key SK using the fully homomorphic encryption algorithm.

[0056] The TEE securely sends the PK to the model provider using the first secret key, and the model provider encrypts the model parameters using the PK using the fully homomorphic encryption algorithm, and uploads to the cloud server;

[0057] The image data provider authenticates the TEE, and after successful authentication, the image data provider shares a second secret key with the TEE. The TEE securely sends the PK to the image data provider using the second secret key, and the image data provider encrypts image data using the PK using the fully homomorphic encryption algorithm, and uploads to the cloud server;

[0058] The cloud server uses a pre-trained convolutional neural network model to perform image classification prediction in an encrypted domain on the encrypted image data according to the model parameters, and obtains an encrypted image classification prediction result.

[0059] The cloud server feeds back the encrypted image classification prediction result to the TEE, and the TEE uses the SK to obtain an original image prediction result by using a fully homomorphic encryption algorithm.

[0060] The TEE encrypts the original image prediction result using the second secret key and sends it to the image data provider, and the image data provider uses the second secret key to obtain the original image prediction result.

[0061] In another aspect, a cloud server is also provided for using a pre-trained convolutional neural network model to perform image classification prediction in an encrypted domain on encrypted image data uploaded by an image data provider, using the encrypted model parameters uploaded by a model provider, by using the above-mentioned privacy protection image classification method supporting a trusted execution environment.

[0062] Compared with the prior art, the above technical solution has at least the following beneficial effects:

[0063] The present application realizes end-to-end image ciphertext input and ciphertext output, provides core technical support for realizing privacy data usability invisibility, and can greatly improve the calculation efficiency of fully homomorphic encryption. BRIEF DESCRIPTION OF DRAWINGS

[0064] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0065] Figure 1 A flow chart of a privacy protection image classification method supporting a trusted execution environment is provided for the embodiments of the present application.

[0066] Figure 2 An encrypted image data schematic diagram is provided for the embodiments of the present application.

[0067] Figure 3 Symmetric multiplication (3(b)) and existing technology sequential multiplication (3(a)) schematic diagrams are provided for the embodiments of the present application.

[0068] Figure 4 A system block diagram of a privacy protection image classification method supporting a trusted execution environment is provided for the embodiments of the present application. DETAILED DESCRIPTION

[0069] In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions of the embodiments of the present application will be described clearly and completely below with reference to the drawings of the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the described embodiments of the present application, all other embodiments obtained by a person of ordinary skill in the art without any inventive effort fall within the scope of protection of the present application.

[0070] As shown in Figure 1 The embodiments of the present application provide a privacy protection image classification method supporting a trusted execution environment, which comprises the following steps:

[0071] S1, a model provider authenticates a trusted execution environment (TEE), and after successful authentication, the model provider shares a first secret key and model parameters of a pre-trained convolutional neural network model with the TEE. The TEE sets security parameters and homomorphic capacity of a fully homomorphic encryption algorithm according to the model parameters, and randomly generates a pair of asymmetric system public key PK and system private key SK by using the fully homomorphic encryption algorithm.

[0072] S2, the TEE securely sends the PK to the model provider by using the first secret key. The model provider encrypts the model parameters by using the PK and the fully homomorphic encryption algorithm, and uploads them to a cloud server.

[0073] S3, an image data provider authenticates the TEE, and after successful authentication, the image data provider shares a second secret key with the TEE. The TEE securely sends the PK to the image data provider by using the second secret key. The image data provider encrypts image data by using the PK and the fully homomorphic encryption algorithm, and uploads them to the cloud server.

[0074] S4, the cloud server performs image classification prediction in an encrypted domain on the encrypted image data by using a pre-trained convolutional neural network model according to the model parameters, and obtains an encrypted image classification prediction result.

[0075] S5, the cloud server feeds back the encrypted image classification prediction result to the TEE. The TEE decrypts the original image prediction result by using the SK and the fully homomorphic encryption algorithm.

[0076] S6, the TEE encrypts the original image prediction result by using the second secret key, and sends it to the image data provider. The image data provider decrypts the original image prediction result by using the second secret key.

[0077] The application will be described below in conjunction with Figures 2-3 A privacy protection image classification method supporting a trusted execution environment is provided, and the method comprises the following steps:

[0078] S1, a model provider authenticates a trusted execution environment (TEE), and after successful authentication, the model provider shares a first secret key and model parameters of a pre-trained convolutional neural network model with the TEE; the TEE sets security parameters and homomorphic capacity of a fully homomorphic encryption algorithm according to the model parameters, and randomly generates an asymmetric pair of a system public key PK and a system private key SK by using the fully homomorphic encryption algorithm.

[0079] The specific process of authenticating the TEE by the model provider and the specific process of sharing the first secret key and the model parameters of the pre-trained convolutional neural network model with the TEE are both prior art, and will not be described here.

[0080] The homomorphic encryption of approximate number algorithm (Cheon-Kim-Kim-Song, CKKS) is an encryption algorithm based on fully homomorphic encryption, which was proposed by Jung Hee Cheon et al. in 2017. It can perform fully homomorphic addition and multiplication operations on ciphertext, which means that data can be calculated and analyzed in an encrypted state, greatly improving the flexibility and efficiency of calculation. The advantages of the CKKS algorithm mainly lie in its scalability, security and wide application. The CKKS algorithm is based on the ring learning with errors problem (Ring Learning with Errors, RLWE) and adopts operations on polynomial rings, which can be used for large-scale data calculation and privacy protection, and its security is based on mathematical difficult problems, providing strong security protection against attacks and cracking. In addition, the CKKS algorithm has wide application prospects in the fields of machine learning, data mining and privacy protection, and is an important algorithm in the field of homomorphic encryption.

[0081] The Convolutional Neural Networks (CNN) model of the embodiment of the present application is a deep neural network mainly used for analyzing image data, which is composed of multiple layers of stacking, such as an input layer, a convolution layer, an activation layer, a pooling layer, a full connection layer, and an output layer. The specific structure of the CNN may not be the same (for example, the convolution layer can be followed by a batch normalization layer, or the convolution layer can be followed by an average pooling layer), and the specific structure of the CNN is not limited by the embodiment of the present application, which is within the protection scope of the embodiment of the present application. The CNN is first trained on a data set, and then can be used for image classification prediction on a prediction image data. The CNN converts the image data of the input layer into the score of each image classification label in the output layer through layer-by-layer calculation. The model parameters of the CNN include the number and type of layers, the size of neurons in each layer, etc. Different layers may need different encryption parameters to maintain the correctness of the calculation. More complex or deep topology usually requires a higher encryption level to ensure that the noise during the operation of the entire model remains within a controllable range. The Trusted Execution Environment (TEE) of the embodiment of the present application determines the highest encryption level L (which defines the growth limit of the noise in the homomorphic encryption scheme, and represents the tolerable calculation depth, i.e., the number of homomorphic operations that can be safely executed without Bootstrapping self-lifting operation) and the security parameter λ as 128 bits according to the topology structure of the model.

[0082] In addition to generating the PK and the SK, the TEE also generates a system calculation key EVK using the fully homomorphic encryption algorithm, which is subsequently sent to the cloud server together with the PK, and needs to be used by the cloud server in the process of image classification prediction in the encrypted domain on the encrypted image data.

[0083] The specific process of the TEE for randomly generating the PK, the SK, and the EVK using the fully homomorphic encryption algorithm is a prior art, which will not be repeated here.

[0084] S2, the TEE securely sends the PK to the model provider using the first secret key, and the model provider uses the PK to encrypt the model parameters using the fully homomorphic encryption algorithm, and uploads to the cloud server;

[0085] The specific process of S2 is also a prior art, which will not be repeated here.

[0086] S3, the image data provider authenticates the TEE, after successful authentication, the image data provider shares a second secret key with the TEE, the TEE securely sends the PK to the image data provider using the second secret key, and the image data provider uses the PK to encrypt image data using a fully homomorphic encryption algorithm and uploads it to the cloud server;

[0087] Optionally, the fully homomorphic encryption algorithm is a CKKS encryption algorithm, and the image data provider in S3 uses the PK to encrypt image data using a fully homomorphic encryption algorithm, specifically including:

[0088] The same pixel point of the plurality of original images is packed into the ciphertext in the corresponding position of the ciphertext matrix, so that the original image matrix img[channels][length][width] in the floating-point number domain is encoded and encrypted to obtain a ciphertext image matrix encrypted_img[channels][length][width] with the same size as the original image matrix.

[0089] The encryption of image data is essentially the conversion of image data from the plaintext domain to the ciphertext domain. Before using the CKKS encryption algorithm to encrypt the image data, since the space corresponding to the elements of the original image is the entire floating-point number, the CKKS needs to pre-encode the floating-point number to obtain a polynomial ring, and the ciphertext can be obtained after encrypting the polynomial ring. Therefore, the plaintext space referred to in the embodiment of the application refers to the polynomial ring space after encoding the pixel points of the original image, and the plaintext refers to the encoding result of the polynomial ring type, and the space corresponding to the original image is uniformly referred to as the floating-point number domain.

[0090] In order to directly use the convolutional neural network model trained by the original image set in the floating-point number domain, it is necessary to encode and encrypt the image matrix img[channels][length][width] in the floating-point number domain to obtain a ciphertext image matrix encrypted_img[channels][length][width] with the same size as the original image matrix. The ciphertext image matrix is set to have the same size as the original image matrix, and each element is encrypted by the corresponding pixel point of the original image. This form of ciphertext image can be directly calculated according to the calculation method in the floating-point number domain in subsequent convolutional neural network calculation, and the subsequent results can also be kept consistent with the calculation results in the floating-point number domain, which is convenient for the implementation of the specific scheme.

[0091] Since the plaintext obtained by the encoding mode in the CKKS encryption algorithm is a cyclotomic polynomial, it supports packing multiple values into different "slots" of a polynomial. This packing mechanism allows multiple sets of data to be encoded into a polynomial for parallel operations such as addition and multiplication in actual operations. Since each image is independent of each other during inference in the ciphertext domain and only depends on the pre-trained network model, based on the packing mechanism of plaintext encoding, the embodiment of the present application packs the pixel points at the same position of multiple original images into one ciphertext for parallel calculation, thereby realizing inference in the ciphertext domain while processing a batch of images, rather than repeatedly processing a single image, as shown in Figure 2 Due to the redundancy of plaintext encoding and the efficiency problem of homomorphic encryption, the overhead of memory and communication for image classification prediction in the encryption domain is greatly increased compared to the floating-point number domain, and it is more time-consuming. Parallel processing can evenly distribute the overhead of a batch of images to be processed. For scenarios that require simultaneous inference in the encryption domain for a group of images, the overhead of calculation, storage, and transmission is greatly reduced.

[0092] When using the encoding packing mechanism, the first is to pack the pixel points at the same position of multiple original images into one ciphertext, and the second is to pack different pixel points of an original image into one ciphertext. The first packing scheme uses the redundancy of plaintext encoding to integrate the same type of addition and multiplication of multiple images into one calculation, which can greatly improve the efficiency of processing a large amount of data and can allocate the number of "slots" to the upper limit to achieve the reduction of average time consumption for a large number of image calculations. However, when processing a small number of images, the required overhead is the same as when processing a full load of "slots", which is not ideal for scenarios that require processing a small amount of data. The second packing scheme can use the redundancy of plaintext encoding to integrate many multiplication or addition calculations of an image into one ciphertext multiplication or ciphertext addition, thereby greatly improving the speed of processing a single image. Some schemes can place a maximum number of images whose data quantity does not exceed the number of "slots" in the number of "slots" that the plaintext can accommodate, thereby achieving a certain parallel processing. However, this scheme also has some problems. Packing different pixel points of an original image into one ciphertext often requires a large number of rotation operations when subsequent operations are performed on these ciphertext values. The rotation operation of the ciphertext also consumes a lot of time, which does not reduce the time overhead in large-scale processing. Since the amount of data required to be embedded in an original image is often not a factor of the number of "slots", the number of "slots" in the plaintext will be wasted, which further affects the time consumption of the overall scheme compared to the first scheme.

[0093] After comparison and comprehensive analysis, the packing scheme adopted in the embodiment of the present application is to pack the same pixel points of different original images into the ciphertext in the corresponding position of the ciphertext matrix.

[0094] S4, the cloud server uses a pre-trained convolutional neural network model to perform image classification prediction in the encrypted domain according to the model parameters, and obtains an encrypted image classification prediction result;

[0095] Optionally, the S4 specifically includes:

[0096] The object processed by the convolutional neural network model is a floating-point number type image matrix, and the encrypted ciphertext image is a ciphertext polynomial ring type image matrix, which cannot be directly used as the input of the convolutional neural network model. The network layer of the convolutional neural network model is adjusted to apply it to image classification prediction in the ciphertext domain. The network layer of the convolutional neural network model is divided into a polynomial layer and a non-polynomial layer. The operation of the polynomial layer on data only has addition and multiplication, and can be expressed in the form of a polynomial (such as a convolution layer). The network layer that cannot be expressed in the form of a polynomial is referred to as a non-polynomial layer (such as an activation layer).

[0097] The adjustment of the polynomial layer includes:

[0098] Suppose that the plaintext image or intermediate result is represented as IMG, and the corresponding ciphertext form is represented as ENC_IMG. The operation of the polynomial layer of the convolutional neural network model on the image in the floating-point number domain includes linear operation and Nth power IMG N Two forms.

[0099] For the form of linear operation: the product of the element IMG i in the corresponding vector or matrix and the corresponding parameter filter i is added to the bias value bias, and in the prediction in the ciphertext domain, the expected result is the encrypted value of the result in the floating-point number domain. According to the properties of homomorphic addition and homomorphic multiplication in homomorphic encryption, the operation in the ciphertext domain is evolved as follows:

[0100]

[0101] The parameter IMG corresponding to the pixel point of the image in the floating-point number domain is first encrypted, and then multiplied with the pixel point of the ciphertext image by using homomorphic multiplication. Finally, the values are added together by using homomorphic addition to obtain the encrypted value of the corresponding result in the floating-point number domain, while ensuring that the information of the original image is not leaked.

[0102] For the form of Nth power: the IMG in the floating-point number domain NThe calculation of the ciphertext domain is expected to obtain the result Enc(IMG N ), and the transformation is as follows:

[0103]

[0104] Only need to multiply the encrypted ciphertext image matrix according to the homomorphic multiplication N times to obtain the encrypted value of the corresponding result;

[0105] For non-polynomial layer adjustment:

[0106] The Relu activation function f(x) = max(0, x) of the activation layer of the convolutional neural network model is approximated to f(x) = 0.1500 + 0.5012X + 0.2981X 2 -0.0004X 3 -0.0388X 4 , and then adjusted by the adjustment method of the polynomial layer.

[0107] In the CKKS encryption algorithm, rescale is required after each multiplication, which reduces the multiplication level of the ciphertext after multiplication. The algorithm needs to set the corresponding multiplication level at the beginning, and the polynomial degree determines the upper limit of the multiplication level. If the multiplication level is too large, the polynomial degree will also increase when setting the parameters. Larger polynomial degree and multiplication level will bring larger time overhead.

[0108] Optionally, for the continuous multiplication of the ciphertext, symmetric multiplication is used to reduce the consumption of the multiplication level, and the symmetric multiplication is as follows:

[0109] Suppose that the function f(x1, x2,..., x i ) = x1·x2·...·x n needs to be calculated about the ciphertext x n (where n takes the power of 2), first calculate x1·x2, x3·x4,..., x n-1 ·x n , then calculate (x1·x2)·(x3·x4),..., (x n-2 ·x n-1 )·(x n-1 ·x n ), and finally calculate (x1·x2)·(x3·x4)...·(x n-1 ·x n ), a total of log2n multiplication levels are required, greatly reducing the consumption of the multiplication level.

[0110] For example, the function f(x1, x2,..., x ifunction f(x1, x2, x3, x4) = x1 x2 x3 x4, if the calculation is in order, i.e. x1 x2, (x1 x2) x3, ((x1 x2) x3) x4, a total of three multiplications are needed. Since x3 is at level 3, which is different from the multiplication level of level 2 where x1 x2 is located, x3 needs to be switched to the same level as x1 x2, i.e. level 2, before multiplication. After multiplication, rescaling is needed to reduce the multiplication level, so each multiplication consumes one multiplication level, and a total of three multiplication levels are consumed, as shown in Figure 3 (a). If the multiplication is in the form of symmetric multiplication, i.e. x1 x2, x3 x4, (x1 x2) (x3 x4), a total of three multiplications are needed. However, since x1 x2 and x3 x4 are at the same level, i.e. level 2, they can be directly multiplied and then rescaled, consuming only two multiplication levels, which reduces the consumption of one multiplication level compared to the sequential method, as shown in Figure 3 (b).

[0111] Optionally, for each polynomial layer that only needs to perform linear operation, multiple parameter multiplication can be optimized to one matrix multiplication. Based on this, the parameter merging method can be used in the prediction stage to fuse and optimize the continuous linear operation polynomial layers, thereby reducing the consumption of multiplication levels, including:

[0112] 1) Convolution layer - batch normalization layer:

[0113] The output of the convolution layer is the input of the batch normalization layer, and the output of each batch normalization layer has the following form of transformation:

[0114]

[0115] When processing the network layer, the corresponding and are encoded in advance, and the convolution layer and the batch normalization layer are merged into one layer, which only needs to consume one multiplication level;

[0116] 2) Convolution layer - average pooling layer:

[0117] The output of the convolution layer is the input of the average pooling layer, and the output of each average pooling layer has the following form of transformation:

[0118]

[0119] When processing the network layer, the parameters of the corresponding convolution layer are divided by the number N of average pooling in advance, that is, and Encoding, only one multiplication is performed, only one multiplication level is needed, and the merging layer is equivalent to calculating the result of the convolution layer before merging times, and adding the corresponding elements for the average pooling layer to obtain the result.

[0120] Optionally, for the function containing an N-th polynomial in the polynomial layer, it can be expressed in the form of , corresponding to N taking 2 k -1 (k∈Z∩[0,+∞)), the highest order a N x N can be split into log2(N+1) multiplication levels of consumption by using a symmetric calculation form, and when N is exactly 2 k (k∈Z∩[0,+∞)), because the coefficient a N The introduction of one multiplication leads to the need for an additional multiplication level of consumption. For such a form of problem, the parameter fusion method is used to transfer the coefficient a N to other polynomial layers of linear operation for merging calculation, thereby reducing the consumption of the multiplication level, and the polynomial approximation function of the activation layer ReLU activation function is f(x) = ax 4 +bx 3 +cx 2 +dx+e, which is the case of N=4. The output of the full connection layer is The output of the activation layer is brought into the output of the full connection layer for calculation, and the following form is obtained:

[0121]

[0122] In processing the activation layer, the highest order coefficient of the approximate function is normalized:

[0123]

[0124] The full connection layer encodes the filter i ·a and bias. In the activation layer, the symmetric multiplication form is used for multiplication, which only needs to consume two multiplication levels, and the full connection layer only needs to do one ciphertext-ciphertext multiplication, which only needs to consume one multiplication level, so the consumption of the multiplication level can be reduced.

[0125] Optionally, for ciphertext matrix multiplication, the ciphertext matrices that need to be multiplied are multiplied by row and column masks respectively to obtain ciphertext matrices that only contain the i-th row and i-th column elements, and then the row and column replication operations are performed by rotating and performing ciphertext homomorphic addition operations respectively to fill the ciphertext matrices. Finally, the slots of the matrices are multiplied and added, which reduces the number of ciphertext multiplications with the largest calculation overhead.

[0126] Optionally, the ciphertext matrix multiplication, in particular comprises:

[0127] Respectively multiplying the ciphertext matrices that need to be multiplied by row and column masks to obtain ciphertext matrices that contain only the i-th row and i-th column elements, including:

[0128] Define two binary vectors Π and Ψ, which are plaintext vectors containing only {0, 1} elements, used to manage the rows and columns of the matrix using scalar multiplication, called row and column masks, for an integer l, the binary vectors Equal to 1 in the slot of index form k1+l·k2, and Equal to 1 in the slot of index from k1 to k1+k2-1, and both vectors are zero in all other slots, represented as follows:

[0129]

[0130]

[0131] Suppose two n×n-dimensional image matrices are encoded and encrypted into ciphertext matrices ct A , ct B , respectively, for 0≤i Return a ciphertext where all slots are equal to 0 except the slot in position (i+l·n) is equal to the corresponding ct A ; for 0≤i Return a ciphertext where all slots are equal to 0 except the slots i·n to (i+1)·n-1 of the i-th row are equal to the corresponding ct B ; through row and column masks, the two ciphertext matrices are expanded into 2n masked ciphertext matrices;

[0132] For example:

[0133]

[0134] When Π 0,3 = [1, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0],

[0135] Similarly,

[0136] When Ψ 0，3 = [1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0],

[0137] Similarly,

[0138] Further, by rotating respectively in row and in column, and ciphertext fully homomorphic addition, the row and column replication operations are performed, including:

[0139] Using algorithm Respectively on the n matrix ciphertexts after column masking Rotate left in column, ciphertext fully homomorphic addition, respectively along column replication, fill the matrix with this column, get the matrix Using algorithm Respectively on the n matrix ciphertexts after row masking Rotate up in row, ciphertext fully homomorphic addition, respectively along row replication, fill the matrix with this row, get the matrix

[0140] The replication algorithm for row and column has no dependency, and can be calculated in parallel, saving operation time.

[0141] 1) Column replication

[0142] The ciphertext matrix Rotate left in column, ciphertext fully homomorphic addition, respectively along column replication, fill the matrix with this column. If simply by rotating and adding, t2-1 rotations and homomorphic additions are needed, while using algorithm Only O(logn) times of fully homomorphic addition operations and O(logn) times of rotation operations are needed.

[0143] As shown in Algorithm 1, wherein Rotate represents rotating the matrix in column, lenBits(t) represents the binary bit number of t, numBits i (t) represents the i-th bit after t is converted into binary.

[0144]

[0145]

[0146] Continuing the above example,

[0147] 2) Row replication

[0148] The ciphertext matrix Rotate up in row, ciphertext fully homomorphic addition, respectively along row replication, fill the matrix with this row. Using algorithm Only O(logn) times of fully homomorphic addition operations and O(logn) times of rotation operations are needed.

[0149] As shown in Algorithm 2, wherein Rotate represents rotating the matrix in row, lenBits(d) represents the binary bit number of d, numBits​i (d) represents the i-th bit after d is converted into binary.

[0150]

[0151] Following the above example,

[0152] Due to the limitation of homomorphic encryption, efficient matrix multiplication has been proven to be challenging. A simple and direct method is to use different ciphertexts for each matrix element, but for a square matrix of dimension n, n 2 ciphertexts are needed to perform multiplication, then n 3 times of ciphertext multiplication.

[0153] The embodiment of the application finally corresponds to the multiplication and addition of the slots of the matrix, comprising:

[0154] The calculation of the ciphertext of the multiplication of the two encrypted matrices is converted into the sum of the point-to-point matrix multiplication , that is The matrix C is obtained as the calculation result.

[0155] The matrix multiplication of the embodiment of the application needs n times of ciphertext multiplication and n times of ciphertext addition, only needs less homomorphic operation and smaller multiplication depth, and through parallel calculation, the performance of the algorithm can be amortized for each multiplication, equivalent to 1 time of multiplication depth, which greatly improves the implementation speed.

[0156] As shown in Algorithm 3:

[0157]

[0158] Following the above example,

[0159]

[0160] S5, the cloud server feeds back the encrypted image classification prediction result to the TEE, and the TEE uses the SK to decrypt the original image prediction result using a fully homomorphic encryption algorithm;

[0161] S6, the TEE encrypts the original image prediction result using the second secret key and sends it to the image data provider, and the image data provider uses the second secret key to decrypt the original image prediction result.

[0162] As shown in Algorithm 3: Figure 4 The embodiment of the application also provides a system for image classification using the above privacy protection image classification method supporting a trusted execution environment, the system comprising: a trusted execution environment TEE, a model provider, an image data provider and a cloud server.

[0163] The model provider authenticates the TEE, and after successful authentication, the model provider shares a first secret key and model parameters of a pre-trained convolutional neural network model with the TEE, and the TEE sets security parameters and homomorphic capacity of a fully homomorphic encryption algorithm according to the model parameters, and randomly generates an asymmetric pair of a system public key PK and a system private key SK by using the fully homomorphic encryption algorithm;

[0164] The TEE securely sends the PK to the model provider by using the first secret key, and the model provider encrypts the model parameters by using the PK and the fully homomorphic encryption algorithm, and uploads to the cloud server;

[0165] The image data provider authenticates the TEE, and after successful authentication, the image data provider shares a second secret key with the TEE, the TEE securely sends the PK to the image data provider by using the second secret key, and the image data provider encrypts image data by using the PK and the fully homomorphic encryption algorithm, and uploads to the cloud server;

[0166] The cloud server performs image classification prediction in an encrypted domain on the encrypted image data by using the pre-trained convolutional neural network model according to the model parameters, and obtains an encrypted image classification prediction result;

[0167] The cloud server feeds back the encrypted image classification prediction result to the TEE, and the TEE decrypts to obtain an original image prediction result by using the SK and the fully homomorphic encryption algorithm;

[0168] The TEE encrypts the original image prediction result by using the second secret key, and sends to the image data provider, and the image data provider decrypts to obtain the original image prediction result by using the second secret key.

[0169] The image classification system provided by the embodiment of the application corresponds to the image classification method provided by the embodiment of the application, and the function structure of the image classification system is not repeated here.

[0170] The embodiment of the application further provides a cloud server for performing image classification prediction in an encrypted domain on encrypted image data uploaded by an image data provider by using the above-mentioned image classification method supporting a trusted execution environment, and obtaining an encrypted image classification prediction result by using a pre-trained convolutional neural network model according to encrypted model parameters uploaded by a model provider.

[0171] The above merely describes preferred embodiments of the present application and is not used to limit the present application, and any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A privacy-preserving image classification method supporting a trusted execution environment, characterized by, The method comprises: S1, the model provider authenticates the trusted execution environment (TEE), and after successful authentication, the model provider shares a first secret key and model parameters of a pre-trained convolutional neural network model with the TEE; the TEE sets security parameters and homomorphic capacity of a fully homomorphic encryption algorithm according to the model parameters, and randomly generates an asymmetric pair of system public key PK and system private key SK by using the fully homomorphic encryption algorithm; S2, the TEE securely sends the PK to the model provider by using the first secret key, the model provider encrypts the model parameters by using the PK and the fully homomorphic encryption algorithm, and uploads to a cloud server; S3, the image data provider authenticates the TEE, and after successful authentication, the image data provider shares a second secret key with the TEE; the TEE securely sends the PK to the image data provider by using the second secret key, the image data provider encrypts image data by using the PK and the fully homomorphic encryption algorithm, and uploads to the cloud server; S4, the cloud server performs image classification prediction in the encrypted domain on the encrypted image data by using the pre-trained convolutional neural network model according to the model parameters, and obtains an encrypted image classification prediction result; S5, the cloud server feeds back the encrypted image classification prediction result to the TEE, and the TEE decrypts the original image prediction result by using the SK and the fully homomorphic encryption algorithm; S6, the TEE encrypts the original image prediction result by using the second secret key and sends it to the image data provider, and the image data provider decrypts the original image prediction result by using the second secret key.

2. The method of claim 1, wherein, The fully homomorphic encryption algorithm is a CKKS encryption algorithm, and in S3, the image data provider encrypts the image data by using the PK and the fully homomorphic encryption algorithm, specifically comprising: Pack the same pixel points of multiple original images into the ciphertext in the corresponding position of the ciphertext matrix, so as to encode and encrypt the original image matrix img[channels][length][width] in the floating-point number domain to obtain a ciphertext image matrix encrypted_img[channels][length][width] with the same size as the original image matrix.

3. The method of claim 2, wherein, In S4, specifically comprising: The object processed by the convolutional neural network model is an image matrix of the floating-point number type, and the encrypted ciphertext image is an image matrix of the ciphertext polynomial ring type, which cannot be directly used as the input of the convolutional neural network model. Adjust the network layer of the convolutional neural network model to a certain extent to apply it to image classification prediction in the ciphertext domain. The network layer of the convolutional neural network model is divided into a polynomial layer and a non-polynomial layer. The operation of the polynomial layer on data is only addition and multiplication, which can be expressed in the form of a polynomial. The network layer that cannot be expressed in the form of a polynomial is referred to as a non-polynomial layer; wherein the adjustment of the polynomial layer comprises: Assuming a plaintext image or intermediate result is represented as IMG, its ciphertext form is represented as ENC IMG, the operation of a polynomial layer of a convolutional neural network model on an image over the field of floating-point numbers includes linear operation and Nth power IMG N in two forms; For the form of linear operation: the product of the corresponding element in the vector or matrix IMG i and the corresponding parameter filter i is superimposed after adding the bias value bias, and the expected result in the prediction under the ciphertext domain is the encrypted value of the result in the floating-point number domain According to the properties of homomorphic addition and homomorphic multiplication in homomorphic encryption, the operation in the ciphertext domain is evolved as follows: The parameters to be multiplied with the pixel points corresponding to the image on the floating-point number field are encrypted first, then multiplied with the ciphertext image pixel points by using homomorphic multiplication, and finally added together by using homomorphic addition to obtain the encrypted value of the floating-point number field corresponding result, while ensuring that the information of the original image is not leaked; For the form of Nth power: IMG N under the floating-point number field, the result expected under the ciphertext field is Enc(IMG N ), then there is a transformation in the following form: Only the encrypted ciphertext image matrix needs to be multiplied N times according to the homomorphic multiplication to obtain the encrypted value of the corresponding result; Adjustment for non-polynomial layers: The Relu activation function f(x) = max(0, x) of the activation layer of the convolutional neural network model is approximated as f(x) = 0.1500 + 0.5012X + 0.2981X 2 -0.0004X 3 -0.0388X 4 and then adjusted by using the adjustment mode of the polynomial layer.

4. The method of claim 3, wherein, For the continuous multiplication of ciphertext, symmetric multiplication is used to reduce the consumption of multiplication levels, and the symmetric multiplication is as follows: Suppose we need to calculate about the ciphertext x i The function f(x1,x2,…,x) n )=x1·x2·...·x n Where n is a power of 2, first calculate x1·x2, x3·x4, ..., x n-1 ·x n Then calculate (x1·x2)·(x3·x4),...,(x n-2 ·x n-1 )·(x n-1 ·x n The calculation is performed recursively, and finally (x1·x2)·(x3·x4)...·(x n-1 ·x n It requires a total of log2n multiplication levels, which greatly reduces the overhead of multiplication levels.

5. The method of claim 4, wherein, For each polynomial layer that only needs to perform linear operation, multiple parameter multiplication can be optimized to once matrix multiplication. Based on this, the parameter merging method can be used in the prediction stage to fuse the continuous linear operation polynomial layers to optimize and reduce the consumption of multiplication levels, including: 1) Convolution layer-batch normalization layer: the output of the convolutional layer is the input to the batch normalization layer, and the output of each batch normalization layer is transformed as follows: When processing the network layer, the corresponding and Encoding, the convolution layer and batch normalization layer into a layer, only one multiplication level of consumption; 2) Convolution layer-average pooling layer: The output of the convolutional layer is the input to the average pooling layer, for each output of the average pooling layer has a transformation of the form: In processing the network layer, the parameters of the corresponding convolution layer are encoded by dividing by the number N of average pooling, that is, the multiplication is only performed once, and only one multiplication level is required. The merging layer is equivalent to calculating the result of the convolution layer before merging and times, and adding the corresponding elements for the average pooling layer to obtain the result. ​ 6. The method of claim 5, wherein, For the function containing N polynomial in the polynomial layer can be expressed in the form of When N takes 2 k -1, k∈Z∩[0,+∞), the highest order a N x N can be split into log2(N+1) multiplication level consumption by symmetric calculation form, and when N takes 2 k exactly, k∈Z∩[0,+∞), because the coefficient a N The introduction of a multiplication leads to the need for an additional multiplication level consumption, for this form of problem, the method of parameter fusion is adopted, and the coefficient a N is transferred to other linear operation polynomial layer for combined calculation, thereby reducing the multiplication level consumption, and the function of the polynomial approximation of the activation layer ReLU activation function is f(x) = ax 4 +bx 3 +cx 2 +dx+e, which is the case of N=4, the output of the full connection layer The output of the activation layer is brought into the output of the full connection layer for calculation, and the following form is obtained: When processing the activation layer, the highest order coefficient of the approximation function is normalized: The fully connected layer is then applied to the filter i The multiplication of a and bias encoding, which is in the form of symmetric multiplication in the activation layer, only consumes two multiplication levels, and the fully connected layer only needs to do one ciphertext-ciphertext multiplication, which only consumes one multiplication level, so the consumption of one multiplication level can be reduced.

7. The method of claim 6, wherein, For ciphertext matrix multiplication, the ciphertext matrix to be multiplied is multiplied by row and column masks respectively to obtain a ciphertext matrix containing only the i-th row and i-th column elements, and then row and column rotation and ciphertext homomorphic addition operations are performed to perform row and column replication operations to fill the ciphertext matrix, and finally the slots of the matrix are multiplied and added to reduce the number of ciphertext multiplications with the largest calculation overhead.

8. The method of claim 7, wherein, The ciphertext matrix multiplication specifically includes: The ciphertext matrix to be multiplied is multiplied by row and column masks respectively to obtain a ciphertext matrix containing only the i-th row and i-th column elements, including: Two binary vectors Π and Ψ are defined, which are plaintext vectors containing only {0,1} elements, to manage the rows and columns of the matrix using scalar multiplication, called row and column masks, for an integer l, the binary vectors equal 1 in the slots of index form k1+l·k2, while equal 1 in the slots of index from k1to k1+k2-1, and both vectors are zero in all other slots, denoted as follows: Suppose two n x n dimensional image matrices are encoded encrypted ciphertext matrices ct A , ct B , respectively, compute the full homomorphic multiplication Return a ciphertext where all slots are equal to 0 except the slots in positions (i + l-n) which are equal to the corresponding ct A , compute the full homomorphic multiplication Return a ciphertext where all slots are equal to 0 except the slots i-n to (i+1)-n-1 of the i-th row which are equal to the corresponding ct B , by row and column masking, the 2 ciphertext matrices are expanded to 2n masked ciphertext matrices; Then row and column rotation and ciphertext homomorphic addition operations are performed to perform row and column replication operations, including: Using algorithm Respectively to the column mask after n matrix ciphertext Rotate by column left, ciphertext homomorphic addition, respectively along the column, fill the matrix with this column, get the matrix Using algorithm Respectively to the row mask after n matrix ciphertext Rotate by row up, ciphertext homomorphic addition, respectively along the row, fill the matrix with this row, get the matrix ct Bi ; Finally, the slots of the matrix are multiplied and added, including: The calculation of the ciphertext of the multiplication of the two encrypted matrices is converted to point-to-point matrix multiplication the sum of, The matrix C is obtained as the result of the calculation.

9. A system for image classification using the privacy-preserving image classification method supporting a trusted execution environment according to any one of claims 1-8, characterized in that, The system includes: a trusted execution environment TEE, a model provider, an image data provider and a cloud server; The model provider authenticates the TEE, and after successful authentication, the model provider shares a first secret key and model parameters of a pre-trained convolutional neural network model with the TEE. The TEE sets security parameters and homomorphic capacity of a fully homomorphic encryption algorithm according to the model parameters, and randomly generates an asymmetric pair of system public key PK and system private key SK by using the fully homomorphic encryption algorithm; The TEE securely sends the PK to the model provider using the first secret key, and the model provider encrypts the model parameters using the PK by using the fully homomorphic encryption algorithm, and uploads to the cloud server; The image data provider authenticates the TEE, and after successful authentication, the image data provider shares a second secret key with the TEE. The TEE securely sends the PK to the image data provider using the second secret key, and the image data provider encrypts image data using the PK by using the fully homomorphic encryption algorithm, and uploads to the cloud server; The cloud server uses a pre-trained convolutional neural network model to perform image classification prediction in the encrypted domain on the encrypted image data according to the model parameters, to obtain an encrypted image classification prediction result; The cloud server feeds back the encrypted image classification prediction result to the TEE, and the TEE uses the SK to obtain an original image prediction result by using a fully homomorphic encryption algorithm; The TEE encrypts the original image prediction result using the second secret key and sends it to the image data provider, and the image data provider uses the second secret key to obtain the original image prediction result.

10. A cloud server, characterized by, A privacy protection image classification method using a trusted execution environment according to any one of claims 1-8 is used to perform image classification prediction in the encrypted domain on encrypted image data uploaded by an image data provider using a pre-trained convolutional neural network model according to encrypted model parameters uploaded by a model provider, to obtain an encrypted image classification prediction result.

Images