Deep learning homomorphic model based on homomorphic encryption and image processing system

Abstract

The invention relates to the technical field of data encryption, in particular to a deep learning homomorphic model based on homomorphic encryption and an image processing system. According to the method, batch data is firstly encoded, then finite-level homomorphic encryption is carried out, and deep learning homomorphic model reasoning is carried out on the data under homomorphic encryption. According to the method, the data of the user only needs to be encrypted once, all subsequent cloud computing is in a ciphertext state, the data of any user cannot be leaked, and a correct reasoning result can be obtained only by decrypting the model reasoning result. Compared with a fully homomorphic encryption technology, the method has the advantages that the efficiency of reasoning calculation can be ensured while a correct reasoning result is provided, and delay caused by large calculation burden is avoided.

Description

Technical Field

[0001] The present invention relates to the field of data encryption technology, and in particular to a deep learning homomorphic model and image processing system based on homomorphic encryption. Background Art

[0002] As an advanced machine learning technology, deep learning technology has become very popular in the fields of pattern recognition, medical prediction, computer vision, natural language processing, speech recognition and intrusion detection because it overcomes the characteristics of artificially designed sample features. In particular, joint model training supported by cloud computing has made deep learning technology develop particularly rapidly. However, as deep learning technology develops rapidly, privacy and security issues arise. In the process of model training and reasoning, a large amount of user sensitive data will be processed. In particular, in the field of medical prediction, the model training and reasoning data used come from a large number of cases, which seriously affects the medical data privacy of patient users. What's more, with the support of cloud computing technology, model training and reasoning are carried out in the cloud, all data will be processed in the cloud, and deep learning models will be shared by all parties, which makes deep learning privacy protection an urgent need.

[0003] At present, deep learning privacy protection technology can be divided into differential privacy (DP) and secure multiparty computing (SMC) according to the protection technology classification. In deep learning differential privacy protection, the input, intermediate parameters and output can be perturbed to protect the privacy data while completing the training and reasoning of the deep learning model. A differential privacy model must meet the following conditions:

[0004] Pr[M(d)∈S]≤e ε Pr[M(d′)∈S]+δ

[0005] Among them, M:D→R is a random algorithm, is any adjacent data set. M(d) and M(d′) represent the output on the data set; Pr[.] is the output probability of the algorithm; ε is the privacy budget, which is used to control the level of privacy protection. The smaller ε is, the stronger the privacy protection provided to the data; δ is another privacy budget, which represents the probability that the privacy budget can be allowed to exceed ε. If a score-checking privacy model meets the above conditions, M is said to satisfy ε-differential privacy. From the perspective of this model, the differential privacy model requires artificial perturbation of the sample data to keep it within a probability range, which will seriously affect the training quality and inference results of the deep learning model. Secure multi-party computing aims to solve the problem of maintaining privacy collaborative computing between a group of mutually distrusting participants. It assumes that there are m participants P1, P2,…, P m , each of them has its own data set d1, d2, …, d m , in the case of an untrusted third party, solve how to safely calculate a contract function y = (d1, d2, ..., d m ). This technology can be used in the joint training mode of deep learning because it involves multiple parties to perform calculations together. However, in the common reasoning process, only user data and cloud computing are involved, and there is no third party. Therefore, secure multi-party computing is more in the training stage of the model.

[0006] In view of the above defects, a deep learning homomorphic model and image processing system based on homomorphic encryption is proposed. Summary of the invention

[0007] In view of the deficiencies in the prior art, the present invention provides a deep learning homomorphic model and image processing system based on homomorphic encryption, which performs deep learning homomorphic model reasoning on data under homomorphic encryption to provide correct reasoning results while ensuring the efficiency of reasoning calculations.

[0008] The present invention discloses a deep learning homomorphic model based on homomorphic encryption through an embodiment, and the deep learning homomorphic model includes:

[0009] A user encrypted transmission layer and a deep neural network, wherein the user encrypted transmission layer includes a homomorphic encryption layer and a homomorphic decryption layer; the homomorphic encryption layer is used to perform batch homomorphic encryption on the input user-side image data to obtain the user-side image data ciphertext; the deep neural network is used to infer the user-side image data ciphertext output by the homomorphic encryption layer to obtain the ciphertext inference data; the homomorphic decryption layer is used to perform homomorphic decryption operation on the ciphertext inference data output by the deep neural network to obtain the inference result of the user-side image data.

[0010] Furthermore, the homomorphic encryption layer includes a plaintext encoding layer and a plaintext encoding encryption layer: the plaintext encoding layer is used to perform batch plaintext encoding on the input user-side image data to obtain encoded data; the plaintext encoding layer encryption layer is used to perform limited-level homomorphic encryption on the encoded data to obtain the user-side image data ciphertext.

[0011] Furthermore, the input user-side image data is batch-encoded in plain text, and the encoded data includes:

[0012] Among K user-side image data of size CH×H×W, select the data at the c*H*W+h*W+wth position of each user-side image data to form a plaintext vector in, 0≤i≤K-1, 0≤c≤CH-1, 0≤h≤H-1, 0≤w≤W-1, It is a plaintext space consisting of K user-side image data of size CH×H×W;

[0013] use For the plaintext vector Encode the plain text;

[0014] Among them, CH is the number of color channels of the user-side image data, H is the height of the user-side image data, and W is the width of the user-side image data. Indicates that in the plaintext encoding process, for a (N / 2)-dimensional complex vector First expand it to After multiplying it by a scaling factor Δ, calculate its discretization The encoding process returns its corresponding integral polynomial The scaling factor Δ is a constant.

[0015] Furthermore, the encoded data is subjected to limited-level homomorphic encryption to obtain the user-side image data ciphertext including:

[0016] Set security parameters, generate a key group (pk, sk, evk) based on the security parameters, and Perform limited-level homomorphic encryption to obtain the ciphertext of the user-side image data

[0017] in, i=c*H*W+h*W+w, 0≤c≤CH-1, 0≤h≤H-1, 0≤w≤W-1;

[0018] Expressed as an encryption algorithm, using the public key pk to encrypt data Encryption, generate ciphertext C i , specifically, it is expressed as: encrypted plaintext for Output v·pk+(m+e0,e1)(modq L ).

[0019] Furthermore, the deep neural network includes: convolutional layers, pooling layers, activation layers and / or fully connected layers.

[0020] The convolution layer, pooling layer, activation layer and / or fully connected layer are used to sequentially encrypt the user-side image data. Perform operations to obtain ciphertext inference data.

[0021] Furthermore, the convolutional layer encrypts the user-side image data The operations include:

[0022] Use convolution kernel to cipher the user-side image data From left to right and from top to bottom, perform homomorphic convolution calculations in sequence to obtain the convolution data. Specifically, it can be expressed as:

[0023]

[0024] Among them, the convolution kernel size is K h ×K w , the convolution step size is S h ×S w , the convolution uses a padding size of P h ×P w , the number of output channels of the convolutional layer is O, and the weight parameter size of the convolutional layer is O×K h ×K w Vector The bias vector of the convolutional layer is a vector of length O x1=max(0,h),y1=max(0,w),x2=min(H-1,h+K h +1), y2=min(W-1,w+K w -1), 0≤h <H+P h -K h +1, 0≤w <W+P w -K w +1, p c =c*H*W+h*W+w,p w =o*K h *K w +(xh)*K w +(yw), 0≤o <O,0≤i<O×n H ×n W , Mult evk It means using the evk key to perform homomorphic calculation on the ciphertext.

[0025] After convolution, the data Perform the first scaling operation to convert the ciphertext data at layer j into the ciphertext data at layer j-1, which is specifically expressed as:

[0026] Then the bias parameter of the convolution kernel Add to scaled data In the above formula, it is specifically expressed as:

[0027] Finally, get the first scaled data

[0028] Furthermore, the pooling layer processes the first scaled data The operations include:

[0029] After the first scaling, Perform pooling operation on each element in and get the pooled data:

[0030] Use a window size of K h ×K w , the step length is S h ×S w , the padding size is P h ×P w The pooling window is in the order from left to right and from top to bottom. Perform homomorphic pooling calculation, assuming First

[0031] Compute the homomorphic accumulation of a pooling window:

[0032]

[0033] Among them, pos=c*H*W+h*W+w, 0≤h <H+P h -K h +1, 0≤w <W+P w -K w +1, x1=max(0,h), y1=max(0,w), x2=min(H-1,h+K h +1), y2=min(W-1,w+K w -1);

[0034] Calculate the final homomorphic pooling result:

[0035]

[0036] After pooling, the data The second scaling operation is performed to convert the ciphertext data at layer j into the ciphertext data at layer j-1, which is specifically expressed as:

[0037] Then the bias parameter of the convolution kernel Add to scaled data In the above formula, it is specifically expressed as:

[0038] Finally, the second scaled data is obtained

[0039] Furthermore, the activation layer performs operation on the user-side image data ciphertext, including: converting the second scaled data located at the jth layer The size is CH×H×W. After the activation calculation of the homomorphic activation layer, the ciphertext at the j-1 layer is obtained. Its size is CH×H×W;

[0040] Specifically, it can be expressed as performing homomorphic multiplication on each ciphertext to obtain: Then, the third scaling operation is used to convert the ciphertext data at layer j into the ciphertext data at layer j-1, which is specifically expressed as:

[0041] Then the bias parameter of the convolution kernel Add to scaled data In the above formula, it is specifically expressed as:

[0042] Finally, the third scaled data is obtained

[0043] Where 0≤i <O×N H ×n W , pos=c*H*W+h*W+w, 0≤c <CH,0≤h≤H-1,0≤w≤W-1。

[0044] Furthermore, the fully connected layer performs operations on the encrypted image data on the user side including:

[0045] Assume that the input of the fully connected layer is the ciphertext at the jth layer Its length is I; let the weight matrix of the homomorphic fully connected layer be Its length is l×I; weight bias vector The length is l, then the output of the homomorphic fully connected layer is the ciphertext at the j-1th layer Its length is l;

[0046] Calculate the homomorphic multiplication of the ciphertext and the weight matrix,

[0047]

[0048] Where 0≤j <l,pos=o*I+j,0≤o<l;

[0049] Perform a homomorphic addition operation on and the bias vector to obtain the ciphertext: Finally, obtain the ciphertext of the homomorphic fully connected layer

[0050] Furthermore, the homomorphic decryption layer performs a homomorphic decryption operation on the ciphertext inference data output by the deep neural network to obtain the inference result of the user-side image data, including:

[0051] Perform plaintext decryption on the ciphertext of the homomorphic fully connected layer to obtain the decrypted data; perform plaintext decoding on the decrypted data to obtain the inference result;

[0052] Specifically, it can be expressed as: use the decryption algorithm in the limited-level homomorphic encryption scheme to decrypt

[0053]

[0054] where 0 ≤ l < l;

[0055] Then use the decoding algorithm in the limited-level homomorphic encryption scheme to perform a decoding operation on the decrypted plaintext M l ’ to obtain the final inference result The length of this vector is K; then use l decoded data to form the inference results of K user data: where pos = k * l + l, 0 ≤ k < K, 0 ≤ l < l; finally, through the decoding of the homomorphic decryption layer, obtain K inference results: Dec sk represents the decryption algorithm, and decrypt the ciphertext using the private key sk.

[0056] In the above algorithms, a key group (pk, sk, evk) is generated according to the security parameters, and at the same time, the public key pk is used to encrypt the data to generate the ciphertext C i The encryption process is prior art and can be implemented relying on prior art; performing homomorphic calculation on the ciphertext using the evk key is prior art and can be implemented relying on prior art; decrypting the ciphertext using the private key sk is prior art and can be implemented relying on prior art.

[0057] To achieve the above object, the present invention also provides an image processing system based on homomorphic encryption, which includes:

[0058] A client is deployed with a user encryption transport layer in the above-mentioned deep learning homomorphic model; a server is deployed with a deep neural network in the above-mentioned deep learning homomorphic model.

[0059] The technical principle of the present invention is: due to homomorphic encryption, the user's data only needs to be encrypted once in the whole process, and all subsequent cloud computing is in a ciphertext state, and no user's data will be leaked. The correct reasoning result can be obtained by decrypting the model reasoning result.

[0060] Compared with the prior art, the present invention has the following beneficial effects: the present invention uses homomorphic encryption technology that supports batch data processing to provide privacy protection, and while avoiding differential privacy to modify data, it can also avoid the multi-party burden of secure multi-party computing. This solution uses limited-level homomorphic encryption calculations to perform model reasoning. Compared with full homomorphic encryption technology, this solution supports batch data processing while providing correct reasoning results, which can ensure the efficiency of reasoning calculations and will not cause delays due to heavy calculation burdens. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] In order to more clearly illustrate the technical solution of the present invention and facilitate further understanding of the technical effects, technical features and purposes of the present invention, the present invention is described in detail below in conjunction with the accompanying drawings. The accompanying drawings constitute an essential part of the specification and are used together with the embodiments of the present invention to illustrate the technical solution of the present invention, but do not constitute a limitation to the present invention.

[0062] Figure 1 A schematic diagram of the model described in the present invention;

[0063] Figure 2 A schematic diagram of the deployment of the model described in the present invention;

[0064] Figure 3 Schematic diagram of batch processing of image data by the model described in the present invention. DETAILED DESCRIPTION

[0065] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. Of course, the specific embodiments described below are only intended to explain the technical solutions of the present invention, rather than to limit the present invention. In addition, the parts described in the embodiments or the accompanying drawings are only examples of the relevant parts of the present invention, rather than the entirety of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art should naturally belong to the protection scope of the present invention.

[0066] At the same time, before introducing the proposed finite-level homomorphic encryption scheme in detail, it is necessary to introduce some symbolic descriptions and operation rules. The subsequent scheme relies on these default symbolic descriptions and operation rules to be implemented.

[0067] All logarithms have base 2; for a vector, use Indicates that each vector is a column vector by default; use <.,.> to represent the dot product of two vectors; for a real number r, represents the nearest integer to r, rounded up; for an integer q, use As and [z] q represents the operation of integer z modulo q into the interval; x←D is used to represent sampling according to sample distribution D. When D is a finite set, it represents sampling from a uniform distribution on D; λ is a security parameter; for a positive integer M, let Φ M (X) is the Mth cyclotomic polynomial of degree N = φ(M); let A number field The ring of integers over ; express The residual ring of operations modulo q; the ring of any real polynomial You can use polynomials of degree less than N Represented by, and the coefficient vector can be used to identify; use for The unit multiplication group on Embedded in the circle The process is that a is in Φ M The evaluation vector at the root of (X) extends it to the set of real polynomials Therefore, for any σ(a) will be represented as where ζ M =exp(-2πi / M) represents the original mth unified root, and l of σ(a) ∞ The norm is called the canonical embedding norm of a, and is expressed as to express; through

[0068]

[0069] Define a Isomorphic inner multiplication space

[0070]

[0071] Where I is the same matrix of size N / 2, and J is its inverse matrix; for r>0, define a Gaussian function ρ r : in Using Γ r represents a continuous Gaussian probability distribution with density r -N ·ρ r(z); Ring Learning with Errors (RLWE) is a problem Indicates that it is for Distribution on Randomly selected from from Among the uniformly random and independent samples, distinguish the The problem of selecting arbitrarily many independent samples.

[0072] like Figure 1 As shown, in order to achieve the above-mentioned purpose, the present invention provides a deep learning homomorphic model based on homomorphic encryption, and the deep learning homomorphic model includes: a user encryption transmission layer and a deep neural network, and the user encryption transmission layer includes a homomorphic encryption layer and a homomorphic decryption layer; the homomorphic encryption layer is used to perform batch homomorphic encryption on the input user-side image data to obtain the user-side image data ciphertext; the deep neural network is used to infer the user-side image data ciphertext output by the homomorphic encryption layer to obtain the ciphertext inference data; the homomorphic decryption layer is used to perform homomorphic decryption operation on the ciphertext inference data output by the deep neural network to obtain the inference result of the user-side image data.

[0073] It is worth noting that if Figure 2 As shown in the figure, when this model is actually applied, the user encryption transport layer is deployed on the user side, and the deep neural network is provided by the provider of the network model, which is deployed in the cloud server provided by the cloud service provider. When the entire model is running, the user encrypts or decrypts data on the user encryption transport layer through his own user side, and the deep neural network performs homomorphic operations on the cloud server. Data under homomorphic encryption is transmitted between the user side and the cloud server through the network. Therefore, when this model is applied, there are only the user side and the cloud server, which avoids the possibility of third-party leakage and has a high privacy protection capability.

[0074] In this embodiment, the homomorphic encryption layer includes a plaintext encoding layer and a plaintext encoding encryption layer. The plaintext encoding layer is used to perform batch plaintext encoding on the input user-side image data to obtain encoded data; the plaintext encoding layer encryption layer is used to perform limited-level homomorphic encryption on the encoded data to obtain the user-side image data ciphertext.

[0075] In this embodiment, if Figure 3 As shown, the input user-side image data is batch-encoded in plain text, and the encoded data includes:

[0076] Among K user-side image data of size CH×H×W, select the data at the c*H*W+h*W+wth position of each user-side image data to form a plaintext vector in, 0≤i≤K-1, 0≤c≤CH-1, 0≤h≤H-1, 0≤w≤W-1, It is a plaintext space consisting of K user-side image data of size CH×H×W;

[0077] use For the plaintext vector Encode the plain text;

[0078] Among them, CH is the number of color channels of the user-side image data, H is the height of the user-side image data, and W is the width of the user-side image data. Indicates that in the plaintext encoding process, for a (N / 2)-dimensional complex vector First expand it to After multiplying it by a constant scaling factor Δ, the discretization is calculated The encoding process returns its corresponding integral polynomial

[0079] In this embodiment, the encoded data is subjected to limited-level homomorphic encryption to obtain the user-side image data ciphertext including:

[0080] Set security parameters, generate a key group (pk, sk, evk) according to the security parameters; Perform limited-level homomorphic encryption to obtain the ciphertext of the user-side image data

[0081] in, i=c*H*W+h*W+w, 0≤c≤CH-1, 0≤h≤H-1, 0≤w≤W-1;

[0082] Expressed as an encryption algorithm, using the public key pk to encrypt data Encryption, generate ciphertext C i , specifically, it is expressed as: encrypted plaintext for Output v·pk+(m+e0,e1modqL.

[0083] In this embodiment, a key group (pk, sk, evk) is generated according to the security parameters, and the public key pk is used to Encryption; the following uses the evk key to perform homomorphic calculations on the ciphertext, and uses the private key sk to perform homomorphic calculations on the ciphertext The decryption algorithm can be implemented by relying on existing technologies. The following is an explanation of the security parameter generation key group (pk, sk, evk) and how to use it: According to the security parameter λ, select an integer M=M(λ,q L ), an integer h = h(λ,qL ), an integer P = P(λ,q L ), and a real number σ=σ(λ,q L ); Set the private key to sk←(1,s) and the public key to where b←-as+e(modq L ); for a positive integer h, is {0,±1} N is the set of signed binary vectors whose Hamming weight is exactly h; for real numbers σ>0, By being independent of the variance σ 2 The discrete Gaussian distribution extracts its coefficients to Sampling the vector in ; Set the evaluation key to where b′←-a′s+e′+Ps 2 (mod P·q L ). Represents the plaintext encoding process, for a (N / 2)-dimensional complex vector The first step in encoding is to expand it to After multiplying by a constant scaling factor Δ, the discretization is calculated The encoding process returns its corresponding integral polynomial Dcd(m;Δ) represents the plaintext decoding process. For an input polynomial Output vector For example: Vector The j∈Tth element of is Enc pk (m) represents the encrypted plaintext m, for Output v·pk+(m+e0,e1)(mod q L ); Decrypted ciphertext Output b+a·s(mod q l ); Represents homomorphic addition operation, for Output Represents homomorphic multiplication operation, for Let (d0, d1, d2) = (b1b2, a1b2 + a2b1, a1a2) (modq l ), output represents the scaling calculation, Output

[0084] In this embodiment, the deep neural network includes: a convolutional layer, a pooling layer, an activation layer and / or a fully connected layer.

[0085] The convolution layer, pooling layer, activation layer and / or fully connected layer are used to sequentially encrypt the user-side image data. Perform operations to obtain ciphertext inference data.

[0086] In this embodiment, the convolutional layer encrypts the user-side image data. The operations include:

[0087] Use convolution kernel to cipher the user-side image data From left to right and from top to bottom, perform homomorphic convolution calculations in sequence to obtain the convolution data. Specifically, it can be expressed as:

[0088]

[0089] Among them, the convolution kernel size is K h ×K w , the convolution step size is S h ×S w , the convolution uses a padding size of P h ×P w , the number of output channels of the convolutional layer is O, and the weight parameter size of the convolutional layer is O×K h ×K w Vector The bias vector of the convolutional layer is a vector of length O x1=max(0,h),y1=max(0,w),x2=min(H-1,h+K h +1), y2=min(W-1,w+K w -1), 0≤h <H+P h -K h +1, 0≤w <W+P w -K w +1, p c =c*H*W+h*W+w,p w =o*K h *K w +(xh)*L w +(yw), 0≤o <O,0≤i<O×n H ×n W , Mult evk It means using the evk key to perform homomorphic calculation on the ciphertext;

[0090] After convolution, the data Perform the first scaling operation to convert the ciphertext data at layer j into the ciphertext data at layer j-1, which is specifically expressed as:

[0091] Then the bias parameter of the convolution kernel Add to scaled data In the above formula, it is specifically expressed as:

[0092] Finally, get the first scaled data

[0093] In this embodiment, the pooling layer performs the first scaled data The operation includes: performing the operation on the first scaled data Perform pooling operation on each element in and get the pooled data:

[0094] Use a window size of K h ×K w , step length is S h ×S w , the padding size is P h ×P w The pooling window is in the order from left to right and from top to bottom. Perform homomorphic pooling calculation, assuming First calculate the homomorphic accumulation value of a pooling window:

[0095]

[0096] Among them, pos=c*H*W+h*W+w, 0≤h <H+P h -K h +1, 0≤w <W+P w -K w +1, x1=max(0,h), y1=max(0,w), x2=min(H-1,h+K h +1), y2=min(W-1,w+K w -1);

[0097] Calculate the final homomorphic pooling result:

[0098]

[0099] After pooling, the data The second scaling operation is performed to convert the ciphertext data at layer j into the ciphertext data at layer j-1, which is specifically expressed as:

[0100] Then the bias parameter of the convolution kernel Add to scaled data In the above formula, it is specifically expressed as:

[0101] Finally, the second scaled data is obtained

[0102] Furthermore, the activation layer performs operation on the user-side image data ciphertext, including: converting the second scaled data located at the jth layer The size is CH×H×W. After the activation calculation of the homomorphic activation layer, the ciphertext at the j-1 layer is obtained. Its size is CH×H×W;

[0103] Specifically, it can be expressed as performing homomorphic multiplication on each ciphertext to obtain: Then, the third scaling operation is used to convert the ciphertext data at layer j into the ciphertext data at layer j-1, which is specifically expressed as:

[0104] Then the bias parameter of the convolution kernel Add to scaled data In the above formula, it is specifically expressed as:

[0105] Finally, the third scaled data is obtained

[0106] Where 0≤i <O×n H ×n w , pos=c*H*W+h*W+w, 0≤c <CH,0≤h≤H-1,0≤w≤W-1。

[0107] In this embodiment, the fully connected layer operates on the encrypted image data of the user side including:

[0108] Assume that the input of the fully connected layer is the ciphertext at the jth layer Its length is I; let the weight matrix of the homomorphic fully connected layer be Its length is l×I; weight bias vector The length is l, then the output of the homomorphic fully connected layer is the ciphertext at the j-1th layer Its length is l;

[0109] Calculate the homomorphic multiplication of the ciphertext and the weight matrix,

[0110]

[0111] Where 0≤j <l,pos=o*I+j,0≤o<e;

[0112] Will With the bias vector Perform homomorphic addition operation to obtain the ciphertext: Finally, obtain the ciphertext of the homomorphic fully connected layer

[0113] In this embodiment, the homomorphic decryption layer performs a homomorphic decryption operation on the ciphertext inference data output by the deep neural network to obtain the inference result of the user-side image data, including:

[0114] Perform plaintext decryption on the ciphertext of the homomorphic fully connected layer to obtain decrypted data; perform plaintext decoding on the decrypted data to obtain the inference result;

[0115] Specifically, it can be expressed as: use the decryption algorithm in the limited-level homomorphic encryption scheme for decryption

[0116]

[0117] where 0 ≤ l < e;

[0118] Then use the decoding algorithm in the limited-level homomorphic encryption scheme to perform a decoding operation on the decrypted plaintext M l ’ to obtain the final inference result The length of this vector is K; then use l decoded data to form the inference results of K user data: where pos = k * l + l, 0 ≤ k < K, 0 ≤ l < l; finally, through the decoding of the homomorphic decryption layer, obtain K inference results: Dec sk represents the decryption algorithm, and decrypt the ciphertext using the private key sk.

[0119] To achieve the above object, the present invention also provides an image processing system based on homomorphic encryption, which includes:

[0120] A client, which deploys the user encryption transmission layer in the above-mentioned deep learning homomorphic model; a server, which deploys the deep neural network in the above-mentioned deep learning homomorphic model.

[0121] It should be noted that the above embodiments are only for more clearly explaining the technical solutions of the present invention. Those skilled in the art can understand that the implementation manners of the present invention are not limited to the above content. Obvious changes, substitutions or replacements based on the above content do not exceed the scope covered by the technical solutions of the present invention; without departing from the concept of the present invention, other implementation manners will of course fall within the scope of the present invention.

Claims

1. A deep learning homomorphic model based on homomorphic encryption, characterized in that: The deep learning homomorphic model includes: A user encrypted transport layer and a deep neural network, wherein the user encrypted transport layer includes a homomorphic encryption layer and a homomorphic decryption layer; The homomorphic encryption layer is used to perform batch homomorphic encryption on the input user-side image data to obtain the user-side image data ciphertext; The deep neural network is used to infer the ciphertext of the user-side image data output by the homomorphic encryption layer to obtain ciphertext inference data; The homomorphic decryption layer is used to perform a homomorphic decryption operation on the ciphertext inference data output by the deep neural network to obtain an inference result of the user-side image data.

2. The deep learning homomorphic model based on homomorphic encryption according to claim 1, characterized in that: The homomorphic encryption layer includes a plaintext encoding layer and a plaintext encoding encryption layer: The plaintext encoding layer is used to perform batch plaintext encoding on the input user-side image data to obtain encoded data; The plaintext coding layer encryption layer is used to perform limited-level homomorphic encryption on the encoded data to obtain the user-side image data ciphertext.

3. The deep learning homomorphic model based on homomorphic encryption as claimed in claim 2, characterized in that: The plaintext encoding layer is used to perform batch plaintext encoding on the input user-side image data, and the encoded data obtained includes: Among K user-side image data of size CH×H×W, select the data at the c*H*W+h*W+wth position of each user-side image data to form a plaintext vector in, It is a plaintext space consisting of K user-side image data of size CH×H×W; use For the plaintext vector Encode the plain text; Among them, CH is the number of color channels of the user-side image data, H is the height of the user-side image data, and W is the width of the user-side image data. Indicates that in the plaintext encoding process, for a (N / 2)-dimensional complex vector First expand it to After multiplying it by a scaling factor Δ, calculate its discretization The encoding process returns its corresponding integral polynomial The scaling factor Δ is a constant.

4. The deep learning homomorphic model based on homomorphic encryption as claimed in claim 3, characterized in that: The plaintext coding layer encryption layer performs limited-level homomorphic encryption on the encoded data to obtain the user-side image data ciphertext including: Set security parameters, generate a key group (pk, sk, evk) based on the security parameters, and Perform limited-level homomorphic encryption to obtain the ciphertext of the user-side image data in, Expressed as an encryption algorithm, using the public key pk to encrypt data Encryption, generate ciphertext C i , specifically, it is expressed as: encrypted plaintext for Output v·pk+(m+e0,e1modqL.

5. The deep learning homomorphic model based on homomorphic encryption as claimed in claim 4, characterized in that: The deep neural network includes: a convolutional layer, a pooling layer, an activation layer and / or a fully connected layer, The convolution layer, pooling layer, activation layer and / or fully connected layer are used to sequentially encrypt the user-side image data. Perform operations to obtain ciphertext inference data.

6. The deep learning homomorphic model based on homomorphic encryption as claimed in claim 5, characterized in that: Use the convolutional layer to encrypt the user-side image data Perform the operation: Use convolution kernel to cipher the user-side image data From left to right and from top to bottom, perform homomorphic convolution calculations in sequence to obtain the convolution data. Specifically, it can be expressed as: Among them, the convolution kernel size is K h ×K w , the convolution step size is S h ×S w , the convolution uses a padding size of P h ×P w , the number of output channels of the convolutional layer is O, and the weight parameter size of the convolutional layer is O×K h ×K w Vector The bias vector of the convolutional layer is a vector of length O x1=max(0,h),y1=max(0,w),x2=min(H-1,h+K h +1), y2=min(W-1,w+K w -1), 0≤h <H+P h -K h +1, 0≤w <W+P w -K w +1, p c =c*H*W+h*W+w,p w =o*K h *K w +(xh)*K w +(yw), 0≤o <O,0≤i<O×n H ×n w , Mult eck It means using the evk key to perform homomorphic calculation on the ciphertext; After convolution, the data Perform the first scaling operation to convert the ciphertext data at layer j into the ciphertext data at layer j-1, which is specifically expressed as: Then the bias parameter of the convolution kernel Add to scaled data In the above formula, it is specifically expressed as: Finally, get the first scaled data 7. The deep learning homomorphic model based on homomorphic encryption according to claim 6, characterized in that: Use the pooling layer to rescale the first data Perform pooling operation on each element in and get the pooled data: Use a window size of K h ×K w , the step length is S h ×S w , the padding size is P h ×P w The pooling window is in the order from left to right and from top to bottom. Perform homomorphic pooling calculation, assuming First calculate the homomorphic accumulation value of a pooling window: where pos = c * H * W + h * W + w, 0 ≤ h < H + P h -K h +1, 0 ≤ w < W + P w -K w +1, x1 = max(0, h), y1 = max(0, w), x2 = min(H - 1, h + K h +1), y2 = min(W-1,w+K w -1); Calculate the final homomorphic pooling result: After pooling, the data The second scaling operation is performed to convert the ciphertext data at layer j into the ciphertext data at layer j-1, which is specifically expressed as: Then the bias parameter of the convolution kernel Add to scaled data In the above formula, it is specifically expressed as: Finally, the second scaled data is obtained 8. The deep learning homomorphic model based on homomorphic encryption according to claim 7, characterized in that: Use the activation layer to rescale the second data The second scaled data at layer j in The size is CH×H×W. After the activation calculation of the homomorphic activation layer, the ciphertext at the j-1 layer is obtained. Its size is CH×H×W; Specifically, it can be expressed as performing homomorphic multiplication on each ciphertext to obtain: Then, the third scaling operation is used to convert the ciphertext data at layer j into the ciphertext data at layer j-1, which is specifically expressed as: Then the bias parameter of the convolution kernel Add to scaled data In the above formula, it is specifically expressed as: Finally, the third scaled data is obtained Where 0≤i <O×n H ×n W , pos=c*H*W+h*W+w, 0≤c <CH,0≤h≤H-1,0≤w≤W-1。 9. The deep learning homomorphic model based on homomorphic encryption as claimed in claim 8, characterized in that: Use the fully connected layer to rescale the third data Perform the operation: Assume that the input of the fully connected layer is the ciphertext at the jth layer Its length is I; let the weight matrix of the homomorphic fully connected layer be Its length is Weight bias vector Length is Then the output of the homomorphic fully connected layer is the ciphertext at the j-1th layer Its length is Calculate the homomorphic multiplication of the ciphertext and the weight matrix, in, pos=o*I+j, Will With the bias vector Perform homomorphic addition operation to obtain the ciphertext: Finally, we get the ciphertext of the homomorphic fully connected layer 10. The deep learning homomorphic model based on homomorphic encryption according to claim 9, characterized in that: The homomorphic decryption layer performs a homomorphic decryption operation on the ciphertext inference data output by the deep neural network to obtain an inference result of the user-side image data, including: Encrypted text of the homomorphic fully connected layer Perform plaintext decryption to obtain decrypted data; perform plaintext decoding on the decrypted data to obtain an inference result; Specifically, it can be expressed as: decryption using the decryption algorithm in the finite-level homomorphic encryption scheme in, Then use the decoding algorithm in the finite-level homomorphic encryption scheme to decrypt the plaintext M' l Perform decoding operations to obtain the final inference results The length of this vector is K; then use The inference result of K decoded data forming K user data: in 0≤k <K, Finally, through decoding at the homomorphic decryption layer, K inference results are obtained: Dec sk Represented as a decryption algorithm, the ciphertext Decrypt using the private key sk.

11. An image processing system based on homomorphic encryption, characterized in that: The image processing system comprises: A client, which is deployed with a user encryption transport layer in the deep learning homomorphic model described in any one of claims 1 to 10; A server, on which a deep neural network in the deep learning homomorphic model described in any one of claims 1-10 is deployed.

Images