Privacy protection image classification method and system supporting trusted execution environment

Abstract

The invention provides a privacy protection image classification method and system supporting a trusted execution environment (TEE). The method comprises the steps that a model provider and the TEE authenticate and share a first secret key and model parameters; the TEE generates a system public key PK and a private key SK by using a fully homomorphic encryption algorithm; the model provider uses a PK encryption model parameter obtained from the TEE and uploads the PK encryption model parameter to the cloud server; the image data provider authenticates and shares a second secret key with the TEE, and the image data provider uploads PK encrypted image data obtained from the TEE to the cloud server; the cloud server performs ciphertext image classification prediction by using the model parameters to obtain an encrypted image classification prediction result; the cloud server feeds back an encryption result to the TEE, and the TEE obtains a result by using SK decryption; and the TEE encrypts the result by using the second secret key and sends the result to an image data provider, and the image data provider decrypts the result to obtain an original image prediction result. The method provides core technical support for realizing the availability and invisibility of the privacy data.

Description

Technical Field

[0001] The present invention relates to the technical field of privacy - protected machine learning, and in particular, to a privacy - protected image classification method and system supporting a trusted execution environment. Background Art

[0002] In Machine Learning as a Service (MLaaS), the model provider provides a trained model, such as a convolutional neural network model, to the image data provider through a cloud server. The image data provider needs to share its image data with the cloud server, and the model provider needs to share its model with the cloud server. Traditional machine learning algorithms need to perform calculations on plaintext data, facing the risks of data leakage and privacy infringement.

[0003] The activation layer is an important part of the convolutional neural network model. Due to its non - linear characteristics, it allows the model to capture complex patterns from the input data. According to different applications, various typical activation functions can be used. For example, the formula of the ReLU activation function is: f(x)=max(0,x).

[0004] Fully Homomorphic Encryption (FHE) belongs to the field of cryptography. Since FHE supports performing arbitrary calculations on ciphertexts without decryption, it can immediately solve the problem of data privacy and security, and there is a great demand for applications. For example, in a cloud environment, users encrypt their data and store it in the cloud. Since the data is encrypted, the cloud cannot obtain the content of the data, thus ensuring data privacy. In addition, due to full - homomorphic encryption, the cloud can perform arbitrary calculations on the ciphertext data. In short, full - homomorphic encryption not only protects data through encryption but also does not lose its computability.

[0005] However, due to fully homomorphic encryption algorithms, such as the CKKS encryption algorithm, which only support homomorphic addition and homomorphic multiplication operations, and the above - mentioned ReLU activation function is a non - linear operation, the fully homomorphic encryption scheme cannot be directly applied to the convolutional neural network model. Therefore, how to apply the fully homomorphic scheme to machine learning while ensuring the performance of the convolutional neural network model and improving the security of image data classification is an urgent problem to be solved. Moreover, the fully homomorphic encryption algorithm requires a large number of homomorphic addition and homomorphic multiplication operations, and its low computational efficiency is also a problem that cannot be ignored. Summary of the Invention

[0006] The present invention provides a privacy - protected image classification method and system supporting a trusted execution environment to solve the problems existing in the prior art. The technical solutions provided by the present invention are as follows:

[0007] On the one hand, a privacy-preserving image classification method supporting a trusted execution environment is provided, and the method includes:

[0008] S1. The model provider authenticates the trusted execution environment (TEE). After successful authentication, the model provider shares a first secret key and the model parameters of a pre-trained convolutional neural network model with the TEE. The TEE sets the security parameters and homomorphic capacity of the fully homomorphic encryption algorithm according to the model parameters, and randomly generates an asymmetric pair of system public key PK and system private key SK using the fully homomorphic encryption algorithm.

[0009] S2. The TEE securely sends the PK to the model provider using the first secret key. The model provider encrypts the model parameters using the PK and the fully homomorphic encryption algorithm, and uploads them to the cloud server.

[0010] S3. The image data provider authenticates the TEE. After successful authentication, the image data provider shares a second secret key with the TEE. The TEE securely sends the PK to the image data provider using the second secret key. The image data provider encrypts the image data using the PK and the fully homomorphic encryption algorithm, and uploads them to the cloud server.

[0011] S4. The cloud server performs image classification prediction in the encrypted domain on the encrypted image data according to the model parameters using the pre-trained convolutional neural network model, and obtains an encrypted image classification prediction result.

[0012] S5. The cloud server feeds back the encrypted image classification prediction result to the TEE. The TEE decrypts it using the SK and the fully homomorphic encryption algorithm to obtain the original image prediction result.

[0013] S6. The TEE encrypts the original image prediction result using the second secret key and sends it to the image data provider. The image data provider decrypts it using the second secret key to obtain the original image prediction result.

[0014] Optionally, the fully homomorphic encryption algorithm is the CKKS encryption algorithm. When the image data provider in S3 encrypts the image data using the PK and the fully homomorphic encryption algorithm, it specifically includes:

[0015] Pack the same pixel points of multiple original images into the ciphertexts at the corresponding positions of the ciphertext matrix, so as to encode and encrypt the original image matrix img[channels][length][width] in the floating-point domain to obtain a ciphertext image matrix encrypted_img[channels][length][width] with the same size as the original image matrix.

[0016] Optionally, S4 specifically includes:

[0017] The object processed by the convolutional neural network model is an image matrix of floating-point type, while the encrypted ciphertext image is an image matrix of ciphertext polynomial ring type and cannot be directly used as the input of the convolutional neural network model. Make certain adjustments to the network layers of the convolutional neural network model to apply it to image classification prediction in the ciphertext domain. Divide the network layers of the convolutional neural network model into polynomial layers and non-polynomial layers. The operations of the polynomial layers on data are only addition and multiplication and can be represented in polynomial form. The network layers that cannot be expressed in polynomial form are called non-polynomial layers;

[0018] Among them, the adjustments to the polynomial layers include:

[0019] Assume that the plaintext image or intermediate result is represented as IMG, and its ciphertext form is represented as ENC_IMG. The operations of the polynomial layers of the convolutional neural network model on the image in the floating-point domain include linear operations and the Nth power of IMG N in two forms;

[0020] For the form of linear operation: the elements IMG in the corresponding vector or matrix i are multiplied by the corresponding parameter filter i and then the bias value bias is added after superposition. In the prediction in the ciphertext domain, the expected result is the encrypted value of the result in the floating-point domain According to the properties of homomorphic addition and homomorphic multiplication in homomorphic encryption, the operations in the ciphertext domain are evolved as follows:

[0021]

[0022] First encrypt the parameters to be multiplied by the pixel points corresponding to the image in the floating-point domain, then multiply them with the ciphertext image pixels using homomorphic multiplication, and finally add these values using homomorphic addition to obtain the encrypted value of the corresponding result in the floating-point domain, while ensuring that the information of the original image is not leaked;

[0023] For the form of the Nth power: the calculation of IMG N in the floating-point domain, the expected result in the ciphertext domain is Enc(IMG N), there is a transformation in the following form:

[0024]

[0025] It is only necessary to multiply the encrypted ciphertext image matrix N times according to homomorphic multiplication to obtain the encrypted value of the corresponding result;

[0026] For the adjustment of non-polynomial layers:

[0027] The Relu activation function f(x)=max(0,x) of the activation layer of the convolutional neural network model is approximated by a polynomial as f(x)=0.1500 + 0.5012X + 0.2981X 2 -0.0004X 3 -0.0388X 4 , and then it is adjusted by the adjustment method of the polynomial layer.

[0028] Optionally, for the continuous multiplication of ciphertext, the symmetric multiplication form is used for multiplication to reduce the consumption of multiplication levels. The symmetric multiplication form is multiplied as follows:

[0029] Suppose it is necessary to calculate the function f(x1,x2,…,x i ) = x1·x2·...·x n (where n takes the power of 2). First, calculate x1·x2, x3·x4, …, x n ·x n-1 , then calculate (x1·x2)·(x3·x4),...,(x n ·x n-2 ), and calculate (x n-1 )·(x n-1 ·x n ) in a loop and recursively. Finally, calculate (x1·x2)·(x3·x4)...·(x n-1 ·x n ). A total of log2n multiplication levels are required, greatly reducing the consumption of multiplication levels.

[0030] Optionally, for each polynomial layer that only needs to perform linear operations, the multiplication of multiple parameters can be optimized into a single matrix multiplication. Based on this, in the prediction stage, the method of parameter merging can be used to fuse continuous linear operation polynomial layers for optimization, thereby reducing the consumption of multiplication levels, including:

[0031] 1) Convolutional layer - Batch Normalization layer:

[0032] The output of the convolutional layer is the input of the Batch Normalization layer. For the output of each Batch Normalization layer there is a transformation in the following form:

[0033]

[0034] When processing the network layer, the corresponding and are encoded, and the convolutional layer and the batch normalization layer are merged into one layer, which only requires the consumption of one multiplication level;

[0035] 2) Convolutional layer - Average pooling layer:

[0036] The output of the convolutional layer is the input of the average pooling layer. For the output of each average pooling layer there is a transformation in the following form:

[0037]

[0038] When processing the network layer, the parameters of the corresponding convolutional layer are encoded by dividing by the number N of average poolings, that is, for and are encoded, then only one multiplication is performed, which only requires the consumption of one multiplication level. The merged layer is equivalent to calculating times the result of the convolutional layer before merging, and adding the corresponding elements for the average pooling layer to obtain the result.

[0039] Optionally, for a function containing an N - degree polynomial in the polynomial layer, it can be represented in the form of . Corresponding to N taking 2 k -1 (k ∈ Z ∩ [0, +∞)), the highest - degree term a N x N can be split into the consumption of log2(N + 1) multiplication levels in a symmetric calculation form. And when N exactly takes 2 k (k ∈ Z ∩ [0, +∞)), due to the coefficient a N introducing one more multiplication, an additional multiplication level of consumption is required. For this form of problem, the method of parameter fusion is adopted to transfer the coefficient a N to other polynomial layers of linear operations for combined calculation, thereby reducing one multiplication level of consumption. The polynomial - approximation function of the ReLU activation function in the activation layer is f(x) = ax 4 + bx 3 + cx 2 + dx + e, which is the case of N = 4. The output of the fully - connected layer brings the output of the activation layer into the output of the fully - connected layer for calculation, obtaining the following form:

[0040]

[0041] When processing the activation layer, normalize the coefficient of the highest-degree term of the approximation function:

[0042]

[0043] The fully-connected layer encodes the filter i ·a and the bias. In the activation layer, multiplying in the form of symmetric multiplication only consumes two multiplication levels, and the fully-connected layer only needs to perform one ciphertext-ciphertext multiplication, consuming only one multiplication level, thus reducing the consumption of one multiplication level.

[0044] Optionally, for ciphertext matrix multiplication, multiply the ciphertext matrices to be multiplied by row and column masks respectively to obtain ciphertext matrices separately containing the elements of the i-th row and the i-th column. Then, through row and column rotations and ciphertext homomorphic addition operations respectively, perform row and column replication operations to fill the ciphertext matrices. Finally, multiply and add the corresponding slots of the matrices, reducing the number of ciphertext multiplications with the largest computational overhead.

[0045] Optionally, the ciphertext matrix multiplication specifically includes:

[0046] Multiply the ciphertext matrices to be multiplied by row and column masks respectively to obtain ciphertext matrices separately containing the elements of the i-th row and the i-th column, including:

[0047] Define two binary vectors Π and Ψ. The binary vectors Π and Ψ are plaintext vectors containing only {0, 1} elements and are used to manage the rows and columns of the matrix using scalar multiplication, called row and column masks. For an integer l, the binary vector is equal to 1 in the slot of index form k1 + l·k2, while is equal to 1 in the slots with indices from k1 to k1 + k2 - 1. In all other slots, these two vectors are zero, represented as follows:

[0048]

[0049]

[0050] Suppose the ciphertext matrices ct A and ct B after encoding and encrypting two n×n-dimensional image matrices. For 0 ≤ i < n, calculate the homomorphic multiplication Return a ciphertext where all slots except the slot at position (i + l·n) are equal to 0 and the slot at position (i + l·n) is equal to the corresponding ct A ; for 0 ≤ i < n, calculate the homomorphic multiplication Return a ciphertext where the slots from i·n to (i + 1)·n - 1 in the i-th row are equal to the corresponding ctB All other slots are equal to 0. Through the row-column masks, the two ciphertext matrices are expanded into 2n masked ciphertext matrices;

[0051] Then, through row-wise and column-wise rotations and homomorphic addition operations on ciphertexts, row replication and column replication operations are performed, including:

[0052] Using the algorithm For the n matrix ciphertexts after column masking respectively Rotate left by column and perform homomorphic addition on ciphertexts, and replicate along columns respectively to fill the matrix with this column, obtaining the matrix Using the algorithm For the n matrix ciphertexts after row masking respectively Rotate up by row and perform homomorphic addition on ciphertexts, and replicate along rows respectively to fill the matrix with this row, obtaining the matrix Finally, multiply and add the corresponding slots of the matrices, including:

[0053] The calculation of the ciphertext of multiplying the two encrypted matrices is transformed into point-to-point matrix multiplication and to obtain matrix C as the calculation result.

[0054] On the other hand, a system for image classification using the above privacy-preserving image classification method supporting a trusted execution environment is provided. The system includes: a trusted execution environment TEE, a model provider, an image data provider, and a cloud server;

[0055] The model provider authenticates the TEE. After successful authentication, the model provider shares a first secret key and the model parameters of a pre-trained convolutional neural network model with the TEE. The TEE sets the security parameters and homomorphic capacity of the fully homomorphic encryption algorithm according to the model parameters, and randomly generates an asymmetric pair of system public key PK and system private key SK using the fully homomorphic encryption algorithm;

[0056] The TEE securely sends the PK to the model provider using the first secret key. The model provider encrypts the model parameters using the PK with the fully homomorphic encryption algorithm and uploads them to the cloud server;

[0057] The image data provider authenticates the TEE. After successful authentication, the image data provider shares a second secret key with the TEE. The TEE securely sends the PK to the image data provider using the second secret key. The image data provider encrypts the image data using the PK with the fully homomorphic encryption algorithm and uploads them to the cloud server;

[0058] The cloud server uses the pre-trained convolutional neural network model according to the model parameters to perform image classification prediction in the encrypted domain on the encrypted image data, and obtains an encrypted image classification prediction result;

[0059] The cloud server feeds back the encrypted image classification prediction result to the TEE, and the TEE uses the SK to decrypt it using the fully homomorphic encryption algorithm to obtain the original image prediction result;

[0060] The TEE encrypts the original image prediction result using the second secret key and sends it to the image data provider, and the image data provider decrypts it using the second secret key to obtain the original image prediction result.

[0061] On the other hand, a cloud server is also provided, which is used to perform image classification prediction in the encrypted domain on the encrypted image data uploaded by the image data provider using the above privacy protection image classification method supporting the trusted execution environment, according to the encrypted model parameters uploaded by the model provider, and obtains an encrypted image classification prediction result.

[0062] The above technical solution has at least the following beneficial effects compared with the prior art:

[0063] The present invention realizes end-to-end encrypted image input and encrypted image output, provides core technical support for realizing the availability but invisibility of privacy data, and can greatly improve the computing efficiency of fully homomorphic encryption. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present invention, and those of ordinary skill in the art can also obtain other drawings based on these drawings without creative efforts.

[0065] Figure 1 It is a flowchart of a privacy protection image classification method supporting a trusted execution environment provided by an embodiment of the present invention;

[0066] Figure 2 It is a schematic diagram of encrypted image data provided by an embodiment of the present invention;

[0067] Figure 3 It is a schematic diagram of symmetric multiplication (3(b)) and sequential multiplication (3(a)) of the prior art provided by an embodiment of the present invention;

[0068] Figure 4 It is a block diagram of a privacy protection image classification system supporting a trusted execution environment provided by an embodiment of the present invention. Detailed implementation manners

[0069] To make the objectives, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Apparently, the described embodiments are some but not all of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the described embodiments of the present invention without creative efforts shall fall within the scope of protection of the present invention.

[0070] As Figure 1 shown, the embodiments of the present invention provide a privacy protection image classification method supporting a trusted execution environment, and the method includes:

[0071] S1. A model provider authenticates a trusted execution environment (TEE). After successful authentication, the model provider shares a first secret key and model parameters of a pre-trained convolutional neural network model with the TEE. The TEE sets security parameters and a homomorphic capacity of a fully homomorphic encryption algorithm according to the model parameters, and randomly generates a pair of asymmetric system public key (PK) and system private key (SK) by using the fully homomorphic encryption algorithm;

[0072] S2. The TEE securely sends the PK to the model provider by using the first secret key. The model provider encrypts the model parameters by using the PK and the fully homomorphic encryption algorithm, and uploads the encrypted model parameters to a cloud server;

[0073] S3. An image data provider authenticates the TEE. After successful authentication, the image data provider shares a second secret key with the TEE. The TEE securely sends the PK to the image data provider by using the second secret key. The image data provider encrypts image data by using the PK and the fully homomorphic encryption algorithm, and uploads the encrypted image data to the cloud server;

[0074] S4. The cloud server performs encrypted domain image classification prediction on the encrypted image data by using the pre-trained convolutional neural network model according to the model parameters, and obtains an encrypted image classification prediction result;

[0075] S5. The cloud server feeds back the encrypted image classification prediction result to the TEE. The TEE decrypts the encrypted image classification prediction result by using the SK and the fully homomorphic encryption algorithm to obtain an original image prediction result;

[0076] S6. The TEE encrypts the original image prediction result by using the second secret key, and sends the encrypted result to the image data provider. The image data provider decrypts the encrypted result by using the second secret key to obtain the original image prediction result.

[0077] The following combines Figures 2 - 3 , and details a privacy - protected image classification method provided by an embodiment of the present invention that supports a trusted execution environment. The method includes:

[0078] S1. The model provider authenticates the trusted execution environment (TEE). After successful authentication, the model provider shares a first secret key and the model parameters of a pre - trained convolutional neural network model with the TEE. The TEE sets the security parameters and homomorphic capacity of the fully homomorphic encryption algorithm according to the model parameters, and randomly generates an asymmetric pair of system public key PK and system private key SK using the fully homomorphic encryption algorithm;

[0079] The specific process of the model provider authenticating the TEE and the specific process of sharing the first secret key and the model parameters of the pre - trained convolutional neural network model with the TEE are both prior arts and will not be elaborated here.

[0080] The homomorphic encryption of approximate number algorithms (Cheon - Kim - Kim - Song, CKKS) is an encryption algorithm based on fully homomorphic encryption, proposed by JungHeeCheon et al. in 2017. It can perform fully homomorphic addition and fully homomorphic multiplication operations on ciphertexts, which means that data can be calculated and analyzed in the encrypted state, greatly improving the flexibility and efficiency of calculations. The advantages of the CKKS algorithm are mainly reflected in its scalability, security, and wide application range. The CKKS algorithm is based on the Ring Learning with Errors (RLWE) problem, uses operations on polynomial rings, can be used for the calculation and privacy protection of large - scale data, and at the same time, its basis on mathematical difficult problems provides strong security guarantees for attacks and cracking. In addition, the CKKS algorithm has broad application prospects in the fields of machine learning, data mining, privacy protection, etc., and is an important algorithm in the field of homomorphic encryption.

[0081] The Convolutional Neural Networks (CNN) model in the embodiments of the present invention is a deep neural network mainly used for analyzing image data, which consists of multiple layers stacked, such as an input layer, a convolutional layer, an activation layer, a pooling layer, a fully connected layer, and an output layer. The specific structure of the CNN may be different (for example, a batch normalization layer may follow the convolutional layer, or an average pooling layer may follow the convolutional layer). The embodiments of the present invention do not limit the specific structure of the CNN, and all are within the protection scope of the embodiments of the present invention. The CNN is first trained on a dataset and then can be used to perform image classification prediction on the image data to be predicted. The CNN transforms the image data in the input layer into scores for each image classification label in the output layer through layer-by-layer calculations. The model parameters of the CNN include the number and type of layers, the size of neurons in each layer, etc. Different layers may require different encryption parameters to maintain the correctness of the calculation. More complex or deeper topologies usually require a higher encryption level to ensure that the noise remains within a controllable range during the operation of the entire model. The Trusted Execution Environment (TEE) in the embodiments of the present invention determines the highest encryption level L (defining the growth limit of noise in the homomorphic encryption scheme, representing the tolerable computational depth, that is, the number of homomorphic operations that can be safely executed without performing the Bootstrapping operation) and the security parameter λ of 128 bits according to the topology of the model.

[0082] In addition to generating the PK and SK, the TEE also generates the system computing key EVK using the fully homomorphic encryption algorithm, which is subsequently sent to the cloud server together with the PK and is required to be used by the cloud server during the process of performing image classification prediction on the encrypted image data in the encrypted domain.

[0083] The specific process of the TEE randomly generating the PK, SK, and EVK using the fully homomorphic encryption algorithm is prior art and will not be elaborated here.

[0084] S2. The TEE securely sends the PK to the model provider using the first secret key. The model provider encrypts the model parameters using the PK with the fully homomorphic encryption algorithm and uploads them to the cloud server.

[0085] The specific process of S2 is also prior art and will not be elaborated here.

[0086] S3. The image data provider authenticates the TEE. After successful authentication, the image data provider and the TEE share a second secret key. The TEE securely sends the PK to the image data provider using the second secret key. The image data provider encrypts the image data using the PK with the fully homomorphic encryption algorithm and uploads it to the cloud server.

[0087] Optionally, the fully homomorphic encryption algorithm is the CKKS encryption algorithm. In S3, the image data provider encrypts the image data using the PK with the fully homomorphic encryption algorithm, which specifically includes:

[0088] Pack the same pixel points of multiple original images into the ciphertexts at the corresponding positions of the ciphertext matrix, so as to encode and encrypt the original image matrix img[channels][length][width] in the floating-point domain to obtain a ciphertext image matrix encrypted_img[channels][length][width] of the same size as the original image matrix.

[0089] The encryption of image data is essentially the transformation of image data from the plaintext domain to the ciphertext domain. Before encrypting the image data using the CKKS encryption algorithm, since the space corresponding to the elements of the original image is the set of all floating-point numbers, and CKKS needs to encode the floating-point numbers in advance to obtain a polynomial ring and encrypt the polynomial ring to obtain the ciphertext, the plaintext space referred to in the embodiments of the present invention all refers to the polynomial ring space after encoding the pixel points of the original image, the plaintexts all refer to the encoding results of the polynomial ring type, and the space corresponding to the original image is uniformly referred to as the floating-point domain.

[0090] For image classification prediction in the encrypted domain of the encrypted image data, it is first necessary to encode and encrypt the image data. In order to be able to directly use the convolutional neural network model trained with the original image set in the floating-point domain, it is necessary to encode and encrypt the image matrix img[channels][length][width] in the floating-point domain to obtain a ciphertext image matrix encrypted_img[channels][length][width] of the same size as the original image matrix. The ciphertext image matrix is set to be the same size as the original image matrix, and each element is encrypted from the corresponding pixel point of the original image. This form of ciphertext image can be directly calculated according to the calculation method in the floating-point domain in the subsequent convolutional neural network calculation, and the subsequent results can also retain the same size as the calculation results in the floating-point domain, which is convenient for the implementation of the specific scheme.

[0091] Since the plaintext obtained by the encoding method in the CKKS encryption algorithm is a cyclotomic polynomial, which supports packing multiple numerical values into different "slots" of a polynomial, this packing mechanism enables multiple sets of data to be encoded into a polynomial in actual operations, thereby performing parallel operations such as addition and multiplication on it. Since each image is independent and irrelevant when performing inference in the ciphertext domain and only depends on the pre-trained network model, therefore, based on the packing mechanism of plaintext encoding, the embodiments of the present invention adopt packing the pixel points at the same position of multiple original images into a ciphertext and performing parallel calculations on them, so as to realize the inference in the ciphertext domain for processing a batch of images simultaneously, rather than repeatedly processing a single image, such as Figure 2 shown. Due to the redundancy of plaintext encoding and the efficiency problem of homomorphic encryption, the memory and communication overheads of image classification prediction in the encrypted domain are much greater than those in the floating-point domain, and it is also slower in terms of time. Through parallel processing, the overheads can be evenly distributed among a batch of images to be processed. For scenarios where it is necessary to perform inference in the encrypted domain on a group of images simultaneously, the overheads in aspects such as computing, storage, and transmission are greatly reduced.

[0092] When applying the encoding packing mechanism, the first one is the method adopted by the embodiments of the present invention, which packs the pixel points at the same position of multiple sets of original images into a ciphertext, and the second one is to pack different pixel points of an original image into a ciphertext. For the first packing scheme, by utilizing the redundancy of plaintext encoding, the addition and multiplication of the same type of multiple images are incorporated into one calculation, which can greatly improve the efficiency of processing a large amount of data and can allocate the number of "slots" to the upper limit to achieve a reduction in the average time-consuming for calculating a large number of images. However, when processing a small number of images, the overhead required is the same as that when the number of "slots" is full, which is not ideal for scenarios with a small amount of data to be processed. For the second packing scheme, by utilizing the redundancy of plaintext encoding, many multiplications or additions of an image can be incorporated into one ciphertext multiplication or ciphertext addition, thereby greatly improving the speed of processing a single image. And some schemes can place the maximum number of images whose data quantity does not exceed the number of "slots" within the number of "slots" that can be accommodated by the plaintext, so as to achieve a certain degree of parallel processing. However, this scheme also has some problems. When packing different pixel points of an original image into a ciphertext, a large number of rotation operations are often required when operating on these ciphertext values subsequently, and the rotation operation of ciphertext also consumes a lot of time, which does not reduce the time overhead in large-scale processing. Since the number of data required to be embedded in an original image is often not a factor of the number of "slots", there will be vacancies and waste in the number of "slots" in the plaintext, and the reduction in the utilization rate of the number of "slots" will further affect the overall time-consuming of the scheme compared to the previous one.

[0093] After comparison and comprehensive analysis, the packing scheme adopted in the embodiments of the present invention is to pack the same pixel points of different original images into the ciphertexts at the corresponding positions of the ciphertext matrix.

[0094] S4. The cloud server uses the pre-trained convolutional neural network model according to the model parameters to perform image classification prediction in the encrypted domain on the encrypted image data, and obtains an encrypted image classification prediction result.

[0095] Optionally, the S4 specifically includes:

[0096] The object processed by the convolutional neural network model is an image matrix of floating-point type, while the encrypted ciphertext image is an image matrix of ciphertext polynomial ring type and cannot be directly used as the input of the convolutional neural network model. Make certain adjustments to the network layer of the convolutional neural network model to apply it to image classification prediction in the ciphertext domain. The network layer of the convolutional neural network model is divided into a polynomial layer and a non-polynomial layer. The operations of the polynomial layer on data are only addition and multiplication and can be represented in polynomial form (such as the convolutional layer), and the network layer that cannot be expressed in polynomial form is called the non-polynomial layer (such as the activation layer).

[0097] Among them, the adjustments for the polynomial layer include:

[0098] Assume that the plaintext image or intermediate result is represented as IMG, and its ciphertext form is represented as ENC_IMG. The operations of the polynomial layer of the convolutional neural network model on the image in the floating-point domain include linear operations and the Nth power of IMG N in two forms;

[0099] For the form of linear operation: the elements IMG in the corresponding vector or matrix i are multiplied by the corresponding parameter filter i and then the bias value bias is added after superposition. In the prediction in the ciphertext domain, it is expected that the obtained result is the encrypted value of the result in the floating-point domain According to the properties of homomorphic addition and homomorphic multiplication in homomorphic encryption, the operations in the ciphertext domain are evolved as follows:

[0100]

[0101] First encrypt the parameters to be multiplied by the pixel points of the image in the floating-point domain, then multiply them with the pixel points of the ciphertext image using homomorphic multiplication, and finally add these values using homomorphic addition to obtain the encrypted value of the corresponding result in the floating-point domain, while ensuring that the information of the original image is not leaked.

[0102] For the form of the Nth power: IMG in the floating-point domain NFor the calculation, the expected result in the ciphertext domain is Enc(IMG N ), then there is a transformation in the following form:

[0103]

[0104] It is only necessary to multiply the encrypted ciphertext image matrix N times according to homomorphic multiplication to obtain the encrypted value of the corresponding result;

[0105] For the adjustment of non-polynomial layers:

[0106] Approximate the Relu activation function f(x) = max(0, x) of the activation layer of the convolutional neural network model with a polynomial as f(x) = 0.1500 + 0.5012X + 0.2981X 2 - 0.0004X 3 - 0.0388X 4 , and then adjust it using the adjustment method of the polynomial layer.

[0107] In the CKKS encryption algorithm, rescale is required after each multiplication, which reduces the multiplication level of the ciphertext after the multiplication operation. The algorithm needs to set the corresponding multiplication level at the beginning, and the polynomial degree determines the upper limit of the multiplication level. If the multiplication level is too large, the polynomial degree will also increase accordingly when setting parameters. Larger polynomial degrees and multiplication levels will bring greater time overhead.

[0108] Optionally, for the continuous multiplication of ciphertexts, multiply them in the form of symmetric multiplication to reduce the consumption of multiplication levels. The multiplication in the form of symmetric multiplication is as follows:

[0109] Suppose it is necessary to calculate the function f(x1, x2,..., x i ) = x1·x2·...·x n (where n takes a power of 2). First, calculate x1·x2, x3·x4,..., x n ·x n-1 ·x n , then calculate (x1·x2)·(x3·x4),...,(x n-2 ·x n-1 )·(x n-1 ·x n ), calculate recursively in a loop, and finally calculate (x1·x2)·(x3·x4)...·(x n-1 ·x n ). A total of log2n multiplication levels are required, greatly reducing the consumption of multiplication levels.

[0110] For example, it is necessary to calculate the ciphertext x iThe function f(x1, x2, x3, x4) = x1·x2·x3·x4. If calculated in sequential order, i.e., x1·x2, (x1·x2)·x3, ((x1·x2)·x3)·x4, a total of three multiplications are required. Since x3 is at level 3, which is different from the multiplication level of x1·x2 at level 2, when multiplying, x3 needs to be switched to the same level as x1·x2, level 2, before multiplication, and after multiplication, it is necessary to rescale to reduce the multiplication level. Therefore, each multiplication consumes one multiplication level, and a total of three multiplication levels are consumed, as shown in Figure 3 (a). If multiplied in symmetric form, i.e., x1·x2, x3·x4, (x1·x2)·(x3·x4), a total of three multiplications are required. However, since x1·x2 and x3·x4 are at the same level, level 2, they can be directly multiplied and then rescaled, consuming only two multiplication levels, which can reduce the consumption of one multiplication level compared to the sequential method, as shown in Figure 3 (b).

[0111] Optionally, for each polynomial layer that only requires linear operations, the multiple parameter multiplications can be optimized into a single matrix multiplication. Based on this, in the prediction stage, the method of parameter merging can be used to fuse consecutive linear operation polynomial layers for optimization, thereby reducing the consumption of multiplication levels, including:

[0112] 1) Convolution layer - Batch Normalization layer:

[0113] The output of the convolution layer is the input of the Batch Normalization layer. For the output of each Batch Normalization layer there is a transformation in the following form:

[0114]

[0115] When processing the network layer, the corresponding and are pre - encoded, and the convolution layer and the Batch Normalization layer are combined into one layer, only requiring one multiplication level of consumption;

[0116] 2) Convolution layer - Average Pooling layer:

[0117] The output of the convolution layer is the input of the Average Pooling layer. For the output of each Average Pooling layer there is a transformation in the following form:

[0118]

[0119] When processing the network layer, the parameters of the corresponding convolution layer are pre - encoded by dividing by the number N of average pooling, that is, for and When encoding, only one multiplication is performed, and only the consumption at the multiplication level is required. The merging layer is equivalent to calculating the result of the convolutional layer before merging by a factor of , and adding the corresponding elements for the average pooling layer to obtain the result.

[0120] Optionally, for a function containing an N-degree polynomial in the polynomial layer, it can be represented in the form of . Corresponding to N taking 2 k -1 (k ∈ Z ∩ [0, +∞)), the highest-degree term a N x N can be split into the consumption at the multiplication level of log2(N + 1) times using a symmetric calculation form. And when N exactly takes 2 k (k ∈ Z ∩ [0, +∞)), due to the coefficient a N introducing one more multiplication, an additional consumption at the multiplication level is required. For this form of problem, the method of parameter fusion is adopted to transfer the coefficient a N to other polynomial layers of linear operations for combined calculation, thereby reducing the consumption at the multiplication level by one time. The polynomial approximation function of the ReLU activation function in the activation layer is f(x) = ax 4 + bx 3 + cx 2 + dx + e, which is the case of N = 4. The output of the fully connected layer Substitute the output of the activation layer into the output of the fully connected layer for calculation to obtain the following form:

[0121]

[0122] When processing the activation layer, normalize the coefficient of the highest-degree term of the approximation function:

[0123]

[0124] The fully connected layer then encodes filter i ·a and bias. In the activation layer, multiplying in the form of symmetric multiplication only requires two multiplications at the multiplication level, and the fully connected layer only needs to perform one ciphertext-ciphertext multiplication, only consuming one multiplication level, so the consumption at the multiplication level can be reduced by one time.

[0125] Optionally, for ciphertext matrix multiplication, multiply the ciphertext matrices to be multiplied by row and column masks respectively to obtain ciphertext matrices separately containing the elements of the i-th row and the i-th column, and then perform row replication and column replication operations through row and column rotations and ciphertext homomorphic addition operations respectively, fill the ciphertext matrix, and finally multiply and add the corresponding slots of the matrix, reducing the number of ciphertext multiplications with the largest computational overhead.

[0126] Optionally, the multiplication of the ciphertext matrices specifically includes:

[0127] Multiply the ciphertext matrices to be multiplied by row and column masks respectively to obtain ciphertext matrices separately containing the elements of the i-th row and the i-th column, including:

[0128] Define two binary vectors Π and Ψ. The binary vectors Π and Ψ are plaintext vectors containing only {0, 1} elements and are used to manage the rows and columns of the matrix using scalar multiplication. They are called row and column masks. For an integer l, the binary vector is equal to 1 in the slot of index form k1 + l·k2, while is equal to 1 in the slots with indices from k1 to k1 + k2 - 1. In all other slots, these two vectors are zero, which is expressed as follows:

[0129]

[0130]

[0131] Suppose the ciphertext matrices ct A and ct B after encoding and encrypting two n×n - dimensional image matrices. For 0 ≤ i < n, calculate the fully homomorphic multiplication Return a ciphertext where all slots except the slot at position (i + l·n) are equal to 0 and the slot at position (i + l·n) is equal to the corresponding ct A ; For 0 ≤ i < n, calculate the fully homomorphic multiplication Return a ciphertext where all slots except the slots from i·n to (i + 1)·n - 1 in the i - th row are equal to 0 and the slots from i·n to (i + 1)·n - 1 in the i - th row are equal to the corresponding ct B . Through the row and column masks, the two ciphertext matrices are expanded into 2n masked ciphertext matrices;

[0132] For example:

[0133]

[0134] When Π 0,3 = [1, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0],

[0135] Similarly,

[0136] When Ψ 0，3 = [1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0],

[0137] Similarly,

[0138] Then, by rotating row - by - row and column - by - column respectively and performing homomorphic addition on ciphertexts, row replication and column replication operations are performed, including:

[0139] Using the algorithm For the n matrix ciphertexts after column masking respectively Rotate left by column and perform homomorphic addition on ciphertexts, and replicate along the column respectively to fill the matrix with this column, obtaining the matrix Using the algorithm For the n matrix ciphertexts after row masking respectively Rotate up by row and perform homomorphic addition on ciphertexts, and replicate along the row respectively to fill the matrix with this row, obtaining the matrix

[0140] Regarding the replication algorithms for rows and columns, there is no dependency between them, and they can be calculated in parallel to save operation time.

[0141] 1) Column replication

[0142] For the ciphertext matrix Rotate left by column and perform homomorphic addition on ciphertexts, and replicate along the column respectively to fill the matrix with this column. If simply adding after rotation is done, t2 - 1 rotations and homomorphic additions are required, while using the algorithm only requires O(logn) homomorphic addition operations and O(logn) rotation operations.

[0143] As shown in Algorithm 1, where Rotate represents rotating the matrix left by column, lenBits(t) represents the number of bits of t in binary, and numBits i (t) represents the i - th bit of t after being converted to binary.

[0144]

[0145]

[0146] Continuing with the above example,

[0147] 2) Row replication

[0148] For Rotate up by row and perform homomorphic addition on ciphertexts, and replicate along the row respectively to fill the matrix with this row. Using the algorithm requires O(logn) homomorphic addition operations and O(logn) rotation operations.

[0149] As shown in Algorithm 2, where Rotate represents rotating the matrix up by row, lenBits(d) represents the number of bits of d in binary, and numBitsi (d) represents the i-th bit after converting d into binary.

[0150]

[0151] Continuing with the above example,

[0152] Due to the limitations of fully homomorphic encryption, efficient matrix multiplication has been proven to be challenging. A simple and straightforward method is to use different ciphertexts for each matrix element. However, for a square matrix of dimension n, n 2 ciphertexts are required to perform the multiplication, and n 3 ciphertext multiplications are needed.

[0153] Finally, the embodiments of the present invention multiply and add the slots of the matrices correspondingly, including:

[0154] transforming the calculation of the ciphertext of multiplying two encrypted matrices into the sum of point-to-point matrix multiplications i.e., obtaining matrix C as the calculation result.

[0155] The matrix multiplication in the embodiments of the present invention requires n ciphertext multiplications and n ciphertext additions, only requires fewer homomorphic operations and a smaller multiplication depth, and through parallel computing, the performance of the algorithm can amortize each multiplication, equivalent to 1 multiplication depth, greatly improving the implementation speed.

[0156] As shown in Algorithm 3:

[0157]

[0158] Continuing with the above example,

[0159]

[0160] S5. The cloud server feeds back the encrypted image classification prediction result to the TEE, and the TEE uses the SK and the fully homomorphic encryption algorithm to decrypt and obtain the original image prediction result;

[0161] S6. The TEE encrypts the original image prediction result using the second secret key and sends it to the image data provider, and the image data provider decrypts it using the second secret key to obtain the original image prediction result.

[0162] As Figure 4 shown, the embodiments of the present invention also provide a system for image classification using the above privacy protection image classification method supporting a trusted execution environment. The system includes: a trusted execution environment TEE, a model provider, an image data provider, and a cloud server;

[0163] The model provider authenticates the TEE. After successful authentication, the model provider shares a first secret key and the model parameters of a pre-trained convolutional neural network model with the TEE. The TEE sets the security parameters and homomorphic capacity of the fully homomorphic encryption algorithm according to the model parameters, and randomly generates an asymmetric pair of system public key PK and system private key SK using the fully homomorphic encryption algorithm;

[0164] The TEE securely sends the PK to the model provider using the first secret key. The model provider encrypts the model parameters using the PK with the fully homomorphic encryption algorithm and uploads them to the cloud server;

[0165] The image data provider authenticates the TEE. After successful authentication, the image data provider shares a second secret key with the TEE. The TEE securely sends the PK to the image data provider using the second secret key. The image data provider encrypts the image data using the PK with the fully homomorphic encryption algorithm and uploads it to the cloud server;

[0166] The cloud server performs image classification prediction in the encrypted domain on the encrypted image data using the pre-trained convolutional neural network model according to the model parameters, and obtains an encrypted image classification prediction result;

[0167] The cloud server feeds back the encrypted image classification prediction result to the TEE. The TEE decrypts it using the SK with the fully homomorphic encryption algorithm to obtain the original image prediction result;

[0168] The TEE encrypts the original image prediction result using the second secret key and sends it to the image data provider. The image data provider decrypts it using the second secret key to obtain the original image prediction result.

[0169] A privacy-preserving image classification system supporting a trusted execution environment provided by an embodiment of the present invention has a functional structure corresponding to a privacy-preserving image classification method supporting a trusted execution environment provided by an embodiment of the present invention, which will not be elaborated here.

[0170] An embodiment of the present invention further provides a cloud server, which is used to perform image classification prediction in the encrypted domain on the encrypted image data uploaded by the image data provider using the pre-trained convolutional neural network model according to the encrypted model parameters uploaded by the model provider, and obtain an encrypted image classification prediction result using the above privacy-preserving image classification method supporting a trusted execution environment.

[0171] The above are only the preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention shall be included within the protection scope of the present invention.

Claims

1. A privacy - protected image classification method supporting a trusted execution environment, characterized in that, The method includes: S1. The model provider authenticates the trusted execution environment (TEE). After successful authentication, the model provider shares a first secret key and the model parameters of a pre-trained convolutional neural network model with the TEE. The TEE sets the security parameters and homomorphic capacity of the fully homomorphic encryption algorithm according to the model parameters, and randomly generates an asymmetric pair of system public key PK and system private key SK using the fully homomorphic encryption algorithm. S2. The TEE securely sends the PK to the model provider using the first secret key. The model provider encrypts the model parameters using the PK with the fully homomorphic encryption algorithm and uploads them to the cloud server. S3. The image data provider authenticates the TEE. After successful authentication, the image data provider shares a second secret key with the TEE. The TEE securely sends the PK to the image data provider using the second secret key. The image data provider encrypts the image data using the PK with the fully homomorphic encryption algorithm and uploads it to the cloud server. S4. The cloud server performs image classification prediction in the encrypted domain on the encrypted image data using the pre-trained convolutional neural network model according to the model parameters, and obtains an encrypted image classification prediction result. S5. The cloud server feeds back the encrypted image classification prediction result to the TEE. The TEE decrypts it using the SK with the fully homomorphic encryption algorithm to obtain the original image prediction result. S6. The TEE encrypts the original image prediction result using the second secret key and sends it to the image data provider. The image data provider decrypts it using the second secret key to obtain the original image prediction result.

2. The method according to claim 1, characterized in that, The fully homomorphic encryption algorithm is the CKKS encryption algorithm. In S3, when the image data provider encrypts the image data using the PK with the fully homomorphic encryption algorithm, it specifically includes: Packing the same pixel points of multiple original images into the ciphertexts at the corresponding positions of the ciphertext matrix, so as to encode and encrypt the original image matrix img[channels][length][width] in the floating-point domain to obtain a ciphertext image matrix encrypted_img[channels][length][width] of the same size as the original image matrix.

3. The method according to claim 2, characterized in that, S4 specifically includes: The object processed by the convolutional neural network model is an image matrix of floating-point type, while the encrypted ciphertext image is an image matrix of ciphertext polynomial ring type and cannot be directly used as the input of the convolutional neural network model. Make certain adjustments to the network layers of the convolutional neural network model to apply it to image classification prediction in the ciphertext domain. Divide the network layers of the convolutional neural network model into polynomial layers and non-polynomial layers. The operations on data in the polynomial layers are only addition and multiplication and can be represented in polynomial form. The network layers that cannot be expressed in polynomial form are called non-polynomial layers. Among them, the adjustments for the polynomial layers include: Assume that the plaintext image or intermediate result is represented as IMG, and its corresponding ciphertext form is represented as ENC_IMG. The operations of the polynomial layer of the convolutional neural network model on the image in the floating-point domain include linear operations and the Nth power of IMG N in two forms; For the form of linear operations: the elements in the corresponding vectors or matrices IMG i and the corresponding parameter filter i After the product superposition, add the bias value bias. In the prediction in the ciphertext domain, the expected result is the encrypted value of the result in the floating-point domain According to the properties of homomorphic addition and homomorphic multiplication in homomorphic encryption, the operations in the ciphertext domain are evolved as follows: The parameters to be multiplied with the pixel points corresponding to the image in the floating-point domain are first encrypted, and then multiplied with the ciphertext image pixel points using homomorphic multiplication. Finally, these values are added together using homomorphic addition to obtain the encrypted value of the corresponding result in the floating-point domain, while ensuring that the information of the original image is not leaked; For the form of the Nth power: IMG in the floating-point domain N For the calculation, the expected result in the ciphertext domain is Enc(IMG N ), then there is a transformation in the following form: Only need to multiply the encrypted ciphertext image matrix N times using homomorphic multiplication to obtain the encrypted value of the corresponding result; Adjustment for non-polynomial layers: The ReLU activation function f(x) = max(0, x) of the activation layer of the convolutional neural network model is approximated by a polynomial as f(x) = 0.1500 + 0.5012X + 0.2981X 2 - 0.0004X 3 - 0.0388X 4 , and then adjusted by the adjustment method of the polynomial layer.

4. The method according to claim 3, wherein For the continuous multiplication of ciphertexts, symmetric multiplication is used to reduce the consumption of multiplication levels. The symmetric multiplication is performed as follows: Suppose it is necessary to calculate a function f(x1, x2, …, x i ) = x1·x2·...·x n (where n is a power of 2). First, calculate x1·x2, x3·x4, …, x n ·x n-1 ·x n . Then, calculate (x1·x2)·(x3·x4),...,(x n-2 ·x n-1 )·(x n-1 ·x n ). Calculate recursively in a loop. Finally, calculate (x1·x2)·(x3·x4)...·(x n-1 ·x n ). A total of log2n multiplication levels are required, greatly reducing the consumption of multiplication levels.

5. The method according to claim 4, wherein For each polynomial layer that only needs to perform linear operations, the multiple parameter multiplications can be optimized into a single matrix multiplication. Based on this, in the prediction stage, the method of parameter merging can be used to fuse the continuous linear operation polynomial layers for optimization, thereby reducing the consumption of multiplication levels, including: 1) Convolution layer - Batch Normalization layer: Output of the convolutional layer is the input to the batch normalization layer. For the output of each batch normalization layer there is a transformation of the following form: When processing the network layer, the corresponding and are pre-encoded, and the convolutional layer and the batch normalization layer are merged into one layer, which only requires the consumption of one multiplication level; 2) Convolution layer - Average Pooling layer: Output of the convolutional layer is the input to the average pooling layer, and for the output of each average pooling layer there is a transformation in the following form: When processing the network layer, the parameters of the corresponding convolutional layer are pre-encoded by dividing them by the number N of average pooling, that is, for and are encoded, then only one multiplication is performed, and only the consumption at the multiplication level is required. The merging layer is equivalent to calculating the times the result of the convolutional layer before merging, and adding the corresponding elements for the average pooling layer to obtain the result.

6. The method according to claim 5, characterized in that, For a function with an N - degree polynomial in a polynomial layer, it can be represented in the form of When corresponding to N taking 2 k -1 (k ∈ Z ∩ [0, +∞)), the highest - degree term a N x N can be split into the consumption of log2(N + 1) - degree multiplication in a symmetric calculation form. And when N exactly takes 2 k (k ∈ Z ∩ [0, +∞)), due to the coefficient a N introducing one more multiplication, an additional multiplication - level consumption is required. For this form of problem, the method of parameter fusion is adopted to transfer the coefficient a N to other polynomial layers of linear operations for combined calculation, thus reducing one multiplication - level consumption. The function approximated by the polynomial of the ReLU activation function in the activation layer is f(x)=ax 4 +bx 3 +cx 2 +dx + e, which is the case of N = 4. The output of the fully - connected layer Substitute the output of the activation layer into the output of the fully - connected layer for calculation to obtain the following form: When processing the activation layer, normalize the coefficient of the highest-degree term of the approximation function: The fully connected layer encodes filter i ·a and bias. In the activation layer, multiplying in the form of symmetric multiplication only requires two multiplication levels, and the fully connected layer only needs to perform one ciphertext-ciphertext multiplication, which only requires one multiplication level, thus reducing the consumption of one multiplication level.

7. The method according to claim 6, wherein For the multiplication of ciphertext matrices, multiply the ciphertext matrices to be multiplied by row and column masks respectively to obtain ciphertext matrices that separately contain the elements of the i-th row and the i-th column. Then, through row and column rotations and ciphertext fully homomorphic addition operations respectively, perform row replication and column replication operations to fill the ciphertext matrix. Finally, multiply and add the corresponding slots of the matrix, reducing the number of ciphertext multiplications with the largest computational overhead.

8. The method according to claim 7, characterized in that The multiplication of the ciphertext matrices specifically includes: Multiply the ciphertext matrices to be multiplied by row and column masks respectively to obtain ciphertext matrices that separately contain the elements of the i-th row and the i-th column, including: Define two binary vectors Π and Ψ, which are plaintext vectors containing only {0, 1} elements and are used to manage the rows and columns of a matrix using scalar multiplication. They are called row and column masks. For an integer l, the binary vector is equal to 1 in the slot at index form k1 + l·k2, while is equal to 1 in the slots with indices ranging from k1 to k1 + k2 - 1. In all other slots, both vectors are zero, which is represented as follows: Suppose the ciphertext matrices ct A and ct B after encoding and encrypting two n×n - dimensional image matrices. For 0 ≤ i < n, calculate the fully homomorphic multiplication Return a ciphertext where all slots except the slot at position (i + l·n) are equal to 0, and the slot at position (i + l·n) is equal to the corresponding ct A ; For 0 ≤ i < n, calculate the fully homomorphic multiplication Return a ciphertext where all slots except the slots from i·n to (i + 1)·n - 1 in the i - th row are equal to 0, and the slots from i·n to (i + 1)·n - 1 in the i - th row are equal to the corresponding ct B ; Through row - column masking, the two ciphertext matrices are expanded into 2n masked ciphertext matrices; Then, through row and column rotations and ciphertext fully homomorphic addition operations respectively, perform row replication and column replication operations, including: Using the algorithm For the n matrix ciphertexts after column masking respectively Rotate left by column, perform homomorphic addition of ciphertexts, copy along columns respectively, and fill the matrix with this column to obtain the matrix Using the algorithm For the n matrix ciphertexts after row masking respectively Rotate up by row, perform homomorphic addition of ciphertexts, copy along rows respectively, and fill the matrix with this row to obtain the matrix Finally, multiply and add the corresponding slots of the matrix, including: The calculation of the ciphertext obtained by multiplying two encrypted matrices is transformed into a point-to-point matrix multiplication and matrix C is obtained as the calculation result.

9. A system for classifying images using the privacy - protected image classification method according to any one of claims 1 - 8 to support a trusted execution environment, characterized in that, The system includes: a Trusted Execution Environment TEE, a model provider, an image data provider, and a cloud server; The model provider authenticates the TEE. After successful authentication, the model provider shares the first secret key and the model parameters of the pre-trained convolutional neural network model with the TEE. The TEE sets the security parameters and homomorphic capacity of the fully homomorphic encryption algorithm according to the model parameters, and randomly generates an asymmetric pair of system public key PK and system private key SK using the fully homomorphic encryption algorithm; The TEE securely sends the PK to the model provider using the first secret key. The model provider uses the PK to encrypt the model parameters using the fully homomorphic encryption algorithm and uploads them to the cloud server; The image data provider authenticates the TEE. After successful authentication, the image data provider shares the second secret key with the TEE. The TEE securely sends the PK to the image data provider using the second secret key. The image data provider uses the PK to encrypt the image data using the fully homomorphic encryption algorithm and uploads it to the cloud server; The cloud server uses the pre-trained convolutional neural network model according to the model parameters to perform image classification prediction in the encrypted domain on the encrypted image data, and obtains an encrypted image classification prediction result; The cloud server feeds back the encrypted image classification prediction result to the TEE, and the TEE uses the SK to decrypt it by using the fully homomorphic encryption algorithm to obtain the original image prediction result; The TEE encrypts the original image prediction result by using the second secret key and sends it to the image data provider, and the image data provider decrypts it by using the second secret key to obtain the original image prediction result.

10. A cloud server, characterized in that, It is used to perform image classification prediction in the encrypted domain on the encrypted image data uploaded by the image data provider by using the pre-trained convolutional neural network model according to the encrypted model parameters uploaded by the model provider, and obtain an encrypted image classification prediction result by using the privacy protection image classification method for supporting the trusted execution environment according to any one of claims 1-8.

Images