Convolutional neural network privacy-preserving prediction outsourcing system based on two-party secure computation

Abstract

The present invention discloses a convolutional neural network privacy protection prediction outsourcing system based on two-party secure computing, comprising a convolutional neural network model deployment unit, a query unit and a prediction unit. The convolutional neural network model deployment unit deploys the model in an encrypted manner in the query unit and the prediction unit through secret sharing and homomorphic encryption; during prediction, the query unit secretly shares the queried data with the prediction unit. Then, the query unit and the prediction unit run the two-party secure computing protocol designed by this system to complete the prediction of the model, and each obtains a share of the prediction result. Finally, the prediction unit sends the share to the query unit for the query unit to restore the prediction result. This system supports the convolutional neural network model deployment unit to outsource the convolutional neural network model prediction service to the prediction unit, providing prediction services to the query unit, while ensuring that the model parameters and the query unit data are not leaked to any participant in the entire process.

Description

Technical Field

[0001] The present invention belongs to the field of information security technology and relates to a convolutional neural network privacy protection prediction outsourcing system based on two-party secure computing, and specifically relates to a single-server convolutional neural network privacy protection prediction outsourcing system based on a two-party secure computing protocol. Background Art

[0002] With the rapid advancement of artificial intelligence (AI) technology, its application scenarios are becoming increasingly diverse and diverse, giving rise to "customized predictive services." This emerging service model allows businesses or institutions to leverage carefully trained algorithmic models to provide highly personalized predictive services to users, maximizing commercial benefits while also facilitating user convenience. Users can enjoy a wide range of services, including but not limited to health monitoring, risk warnings, and personalized recommendations. Given the high complexity and resource-intensive nature of machine learning operations, many service providers prefer to deploy these carefully constructed models on high-performance cloud computing platforms, such as Google Cloud Platform, leveraging the cloud's powerful computing power to process massive amounts of data and provide instant predictive responses. Users can quickly access the prediction results they need by uploading their own data to the cloud. Sygic, for example, has deployed the backend model for its Street View mapping service on Amazon Cloud, successfully providing real-time street views to over 200 million users worldwide.

[0003] However, while enjoying convenient services, data model privacy has become an issue that cannot be ignored. On the one hand, user-submitted data and its prediction results often contain sensitive information, which must be protected from being leaked to cloud servers. On the other hand, models are trained using a vast amount of time, computing power, and data, are commercially competitive, and represent intellectual property. Therefore, leakage to cloud servers or users during cloud deployment and prediction processes must be avoided.

[0004] Privacy-preserving convolutional neural network prediction outsourcing solutions are urgently needed. Current solutions have two main limitations: (1) They rely on the strong assumption that multiple servers do not collude. These solutions protect privacy by secretly sharing model parameters and user data with multiple non-colluding servers. However, ensuring that multiple servers do not collude is a very strong security assumption that is difficult to implement in real-world application scenarios. Once there is collusion between servers, user data and model parameters will be leaked. (2) They are inefficient. Current solutions use homomorphic encryption to calculate matrix multiplication, which takes up most of the running time and communication overhead. There is an urgent need to design a low-complexity homomorphic matrix multiplication algorithm to improve prediction efficiency. Summary of the Invention

[0005] To address the limitations of current solutions, the present invention provides a single-server privacy-preserving prediction outsourcing system based on two-party secure computation.

[0006] The technical solution adopted by the present invention is: a convolutional neural network privacy protection prediction outsourcing system based on two-party secure computing, including a convolutional neural network, a convolutional neural network model deployment unit, a query unit and a prediction unit;

[0007] The convolutional neural network is composed of linear layers and nonlinear layers connected alternately. During prediction, the output of the previous layer is the input of the next layer. The calculation of the linear layer is realized by matrix multiplication, and the operation of the nonlinear layer is element-by-element comparison operation.

[0008] The convolutional neural network model deployment unit deploys the convolutional neural network model in the query unit and the prediction unit in an encrypted manner through secret sharing and homomorphic encryption;

[0009] During prediction, the query unit secretly shares the queried data with the prediction unit, and then the query unit and the prediction unit run a two-party secure computing protocol to complete the prediction of the convolutional neural network model, and each obtains a share of the prediction result; finally, the prediction unit sends the share to the query unit for the query unit to restore the prediction result.

[0010] Preferably, the convolutional neural network model deployment unit is used to deploy the convolutional neural network model between the query unit and the prediction unit in a secret manner without leaking the parameters of the model; the convolutional neural network model is divided into a linear layer and a nonlinear layer according to the type of operation involved; the convolutional neural network model deployment unit adopts different deployment methods for the linear layer and the nonlinear layer, and deploys them layer by layer.

[0011] Preferably, for the linear layer, the convolutional neural network model deployment unit generates a random number R of the same dimension as the parameter matrix W. w , set W 1 =R w 、W 2 =WR w , W 1 and W 2 Send them to the query unit and prediction unit respectively; then use the query unit's public key pk1 to encrypt W using the homomorphic matrix encryption algorithm EMat() 1 , get the ciphertext [W 1 ]←EMat(pk1,W 1 ,0); Finally, the ciphertext [W 1 ] is sent to the prediction unit; for the nonlinear layer, the convolutional neural network model deployment unit sends the nonlinear layer type to the query unit and the prediction unit.

[0012] Preferably, the two-party secure computation protocol includes a linear layer two-party secure computation protocol and a nonlinear layer two-party secure computation protocol;

[0013] Before executing the linear layer two-party secure computation protocol or the nonlinear layer two-party secure computation protocol, the query unit and the prediction unit secretly share the input of the linear layer or the nonlinear layer; after executing the protocol, the query unit and the prediction unit secretly share the output of the linear layer or the nonlinear layer; thereby alternately connecting the linear layer two-party secure computation protocol and the nonlinear layer two-party secure computation protocol to complete the prediction process of the convolutional neural network;

[0014] The linear layer two-party secure computation protocol starts with the query unit holding the secret share X of the input matrix X. 1 , the secret share W of the parameter matrix W 1 ; The prediction unit holds the secret share of the input matrix X 2 , the secret share W of the parameter matrix 2 and the encrypted parameter matrix share [W 1 ], where X = X 1 +X 2 , W=W 1 +W 2 After the protocol is executed, the query unit and the prediction unit obtain Y 1 、Y 2 , where Y = Y 1 +Y 2 ;

[0015] The nonlinear layer two-party secure computation protocol starts with the query unit holding the secret share A of the input elements A and B. 1 、B 1 , the forecast unit holds the other half of the share A 2 、B 2 , where A=A 1 +A 2 , B=B 1 +B 2 After executing the protocol, the query unit and the prediction unit secretly share the output of the nonlinear layer Y = max(A, B), that is, the query unit and the prediction unit obtain Y 1 、Y 2 , where Y = Y 1 +Y 2 .

[0016] Preferably, the linear layer two-party secure computation protocol specifically implements the computation steps including:

[0017] Step A1: The query unit and prediction unit calculate W respectively 1 ·X 1 、W 2 ·X 2 ;

[0018] Step A2: Query unit calculates [X 1]←EMat(pk1,X 1 ,1), and then send it to the prediction unit; the prediction unit also calculates [W 1 ·X 2 ]←MulMat([W 1 ],X 2 , 0), where pk1 is the public key of the query unit, EMat() is the homomorphic matrix encryption algorithm, and MulMat() is the homomorphic matrix multiplication algorithm; then, the prediction unit generates a random matrix R with the same dimension as W·X 1 , calculate [W 1 ·X 2 -R 1 ]←AddMat([W 1 ·X 2 ],-R 1 ), where AddMat() is a homomorphic matrix addition algorithm;

[0019] Step A3: The prediction unit receives [X 1 ], generate a random matrix R with the same dimensions as W·X 2 ; Calculate [W 2 ·X 1 ]←MatMul([W 2 ],X 1 , 1) and [W 2 ·X 1 -R 2 ]←AddMat([W 2 ·X 1 ],-R 2 ); then the ciphertext [W 1 ·X 2 -R 1 ] and [W 2 ·X 1 -R 2 ] is sent to the query unit;

[0020] Step A4: The query unit receives [W 1 ·X 2 -R 1 ] and [W 2 ·X 1 -R 2 ], perform decryption: W 1 ·X 2 -R 1 ←DMat(sk1,[W 1 ·X 2 -R 1 ]), W 2 ·X 1 -R 2 ←DMat(sk1,[W2 ·X 1 -R 2 ]), where sk1 is the private key of the query unit and DMat() is the homomorphic matrix decryption algorithm; then calculate: Y 1 =W 1 ·X 1 +(W 1 ·X 2 -R 1 )+(W 2 ·X 1 -R 2 ).

[0021] Step A5: The prediction unit calculates Y 2 =W 2 ·X 2 +R 1 +R 2 , so that Y=Y 1 +Y 2 .

[0022] The nonlinear layer two-party secure computation protocol specifically implements the computation steps including:

[0023] Step B1: Query unit calculates C 1 =A 1 -B 1 , then C 1 After homomorphic encryption, it is sent to the prediction unit;

[0024] Step B2: The prediction unit receives C 1 Then, first calculate C 2 =A 2 -B 2 , then generate a positive random value U; then homomorphically calculate the ciphertext [(C 1 +C 2 )·U], and send it to the query unit;

[0025] Step B3: After receiving the query unit, it performs a decryption operation and obtains H=(C 1 +C 2 )·U; then calculate D=SIGN(H), where SIGN() represents a function that takes positive and negative signs; then the query unit sends D to the prediction unit;

[0026] Step B4: The query unit and prediction unit output Y according to the positive or negative value of D. 1 、Y 2 :If D is a positive number, set Y 1 =A 1 、Y 2 =A 2 ; If D is non-positive, set Y1 = B 1 , Y 2 = B 2 .

[0027] Preferably, for the matrix encryption algorithm EMat(), the input includes the public key pk, the plaintext matrix M to be encrypted, and the Boole type value b indicating whether the encrypted matrix is a multiplicand matrix or a multiplier matrix; the output is the ciphertext matrix [M]; EMat() first calls the matrix encoding algorithm Pack b Encode the elements of M diagonally into z vectors of length N, denoted as M (0) , M (1) , …, M (k-1) , where N represents the maximum number of elements that a single homomorphic encryption ciphertext can encrypt; then, perform packed homomorphic encryption on each encoded vector respectively, obtaining z ciphertexts in total; finally, output these z ciphertexts.

[0028] Preferably, for the vector decryption algorithm DMat(), the input includes the homomorphic encryption private key sk and the ciphertext [V], and the output is the plaintext matrix E; DMat() first calls the decryption algorithm of homomorphic encryption, decrypt the ciphertext [V] using the homomorphic encryption private key sk to obtain the plaintext vector V; then divide V into k sub-vectors of length , denoted as V 0 , V 1 ,..., V k-1 , where N represents the maximum number of elements that a single homomorphic encryption ciphertext can encrypt; finally, encode these k sub-vectors into a plaintext matrix E as the output, where 0 ≤ i < m, 0 ≤ j < k.

[0029] Preferably, for the addition algorithm AddMat(), the input includes the public key pk of homomorphic encryption, a ciphertext [V], and a plaintext matrix R; AddMat() first encodes R into k vectors of length , denoted as R 0 , R 1 , …, R k-1 , where 0 ≤ i < m, 0 ≤ j < k, and N represents the maximum number of elements that a single homomorphic encryption ciphertext can encrypt; then, AddMat() concatenates R 0 , R 1 , …, R k-1 end to end to form a vector R'; then call the encryption algorithm of packed homomorphic encryption to encrypt R' using the public key pk to obtain the ciphertext [R']; finally, add [R'] and [V] homomorphically to obtain the ciphertext [V + R'] as the output.

[0030] Preferably, for the multiplication algorithm MulMat(), the inputs include the ciphertext matrix [M], the plaintext matrix G, and a Boolean type value b indicating whether the ciphertext matrix is the multiplicand matrix or the multiplier matrix, and the output is the ciphertext of matrix multiplication: if b = 0, then [M·G] is output; otherwise, [G·M] is output; MulMat() first divides the ciphertext matrix [M] into [M (0) ,[M (1) ,...,[M (k -1) ; then, it calls the matrix encoding algorithm Pack 1-b to encode the elements of G diagonally into z vectors of length N, denoted as G (0) ,G (1) ,...,G (z-1) ; then, for each G (i) , 0 ≤ i < z, it calls the packed homomorphic encryption Hadamard product algorithm to homomorphically calculate the product of the corresponding elements of G (i) and [M (i) , obtaining the ciphertext C (i) ; finally, it adds all the ciphertexts C (i) by calling the packed homomorphic encryption addition algorithm and outputs the resulting ciphertext.

[0031] Preferably, the matrix encoding algorithm includes the multiplicand matrix encoding algorithm Pack 0 or the multiplier matrix encoding algorithm Pack 1 ;

[0032] The encoding algorithm Pack 0 of the multiplicand matrix, with the input being the m×n multiplicand matrix W, and the output being vectors of length N, W (0) ,W ' (1) ,...,W (z-1) ; where, W (i) is formed by connecting k identical sub-vectors, 0 ≤ i < z; each sub-vector encodes diagonals of the matrix W; the sub-vector is where "mod n" is omitted in the second subscript;

[0033] The encoding algorithm Pack 1 of the multiplier matrix, with the input being the n×k multiplier matrix X, and the output being z vectors of length N, X (0) ,X (1) ,...,X (z-1) ; where, X (i) is formed by connecting k different sub-vectors, 0 ≤ i < z; each sub-vector encodes t diagonals of the matrix X; the j-th sub-vector is The "mod n" and "mod k" are omitted in the first and second subscripts respectively.

[0034] The present invention also provides a convolutional neural network privacy protection prediction outsourcing product based on two-party secure computing, including computer program instructions. When the computer program instructions are run on a computer, they drive the execution of the convolutional neural network privacy protection prediction outsourcing system based on two-party secure computing.

[0035] Compared with the prior art, the advantages and positive effects of the present invention are mainly reflected in the following aspects:

[0036] (1) This invention provides an efficient single-server convolutional neural network privacy-preserving prediction outsourcing system. This system eliminates the assumption of non-collusive servers and only requires a single server to complete the convolutional neural network privacy-preserving prediction outsourcing.

[0037] (2) The present invention designs a homomorphic matrix encryption scheme, which encodes matrix elements into ciphertext vectors in diagonal order, avoids high-cost homomorphic rotation operations when calculating matrix multiplication, and makes full use of the ciphertext space to pack as many diagonals as possible into one ciphertext, thereby reducing the amount of ciphertext transmission and the number of homomorphic operations, and improving efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] The technical solution of this invention is further illustrated below using embodiments and specific implementation methods. In addition, some drawings are used in the process of illustrating the technical solution. Those skilled in the art can also derive other drawings and the intent of the present invention based on these drawings without making any creative efforts.

[0039] Figure 1 This is a system framework diagram of an embodiment of the present invention;

[0040] Figure 2 This is a flow chart of the linear layer protocol according to an embodiment of the present invention;

[0041] Figure 3 This is an example diagram of the EMat, MulMat, and DMat algorithms of the homomorphic matrix encryption scheme in an embodiment of the present invention. DETAILED DESCRIPTION

[0042] In order to facilitate the understanding and implementation of the present invention by those skilled in the art, the present invention is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the implementation examples described herein are only used to illustrate and explain the present invention and are not used to limit the present invention.

[0043] Please see Figure 1, this embodiment provides a convolutional neural network privacy protection prediction outsourcing system based on two-party secure computing, including a convolutional neural network, a convolutional neural network model deployment unit, a query unit and a prediction unit;

[0044] The convolutional neural network is composed of linear layers and nonlinear layers connected alternately. During prediction, the output of the previous layer is the input of the next layer. The calculation of the linear layer can be achieved through matrix multiplication. The basic operation of the nonlinear layer is element-by-element comparison operation.

[0045] The convolutional neural network model deployment unit deploys the convolutional neural network model in the query unit and the prediction unit in an encrypted manner through secret sharing and homomorphic encryption;

[0046] First, during the deployment phase, the convolutional neural network model deployment unit confidentially deploys the convolutional neural network model between the query unit and the prediction unit without revealing the model's parameters. Convolutional neural network models are divided into linear and nonlinear layers based on the type of operations involved. The convolutional neural network model deployment unit uses different deployment methods for linear and nonlinear layers, deploying them layer by layer.

[0047] In the query phase, the query unit uploads the encrypted query data to the prediction unit to obtain the prediction result of the convolutional neural network. When the query unit executes the query, it generates a random number R of the same dimension as the query data X, and sets X 1 =R、X 2 =XR, X 2 Then, the query unit and the prediction unit execute a two-party secure computation protocol to obtain the secret share Y of the convolutional neural network prediction result Y. 1 、Y 2 , satisfying Y=Y 1 +Y 2 Finally, the query unit gets the share of the prediction unit by calculating Y = Y 1 +Y 2 Get the convolutional neural network prediction results.

[0048] During prediction, the query unit secretly shares the queried data with the prediction unit, and then the query unit and the prediction unit run the two-party secure computing protocol designed by this system to complete the prediction of the convolutional neural network model, and each obtains a share of the prediction result; finally, the prediction unit sends the share to the query unit for the query unit to restore the prediction result.

[0049] In one embodiment, for the linear layer, the convolutional neural network model deployment unit generates a random number R of the same dimension as the parameter matrix W w , set W 1 =R w 、W 2 =WRw , W 1 and W 2 Then use the public key pk1 of the query unit to encrypt W 1 ,Right now

[0050] [W 1 ]←EMat(pk1,W 1 ,0), where EMat() is a homomorphic matrix encryption algorithm. Finally, the ciphertext [W 1 ] is sent to the prediction unit. For the nonlinear layer, the convolutional neural network model deployment unit sends the nonlinear layer type to the query unit and the prediction unit.

[0051] In one embodiment, the two-party secure computation protocol includes: a linear layer two-party secure computation protocol and a nonlinear layer two-party secure computation protocol.

[0052] Before executing the two-party secure computation protocol for the linear or nonlinear layer, the query unit and the prediction unit secretly share the input of the linear or nonlinear layer. After executing the protocol, the query unit and the prediction unit secretly share the output of the linear or nonlinear layer. This allows the linear and nonlinear layers to be alternately connected to complete the prediction process of the convolutional neural network.

[0053] Before the linear layer two-party secure computation protocol begins, the query unit holds the secret share X of the input matrix X 1 , the secret share W of the parameter matrix W 1 ; The prediction unit holds the secret share of the input matrix X 2 , the secret share W of the parameter matrix 2 and the encrypted parameter matrix share [W 1 ], where X = X 1 +X 2 , W=W 1 +W 2 After the protocol is executed, the query unit and the prediction unit obtain Y 1 、Y 2 , where Y = Y 1 +Y 2 .

[0054] Before the nonlinear layer two-party secure computation protocol begins, the query unit holds the secret share A of the input elements A and B 1 、B 1 , the forecast unit holds the other half of the share A 2 、B 2 , where A=A 1 +A 2 , B=B 1 +B 2After executing the protocol, the query unit and the prediction unit secretly share the output of the nonlinear layer Y = max(A, B), that is, the query unit and the prediction unit obtain Y 1 、Y 2 , where Y = Y 1 +Y 2 .

[0055] In one embodiment, the linear layer two-party secure computation protocol is as follows Figure 2 As shown, the specific calculation steps include:

[0056] Step A1: The query unit and prediction unit calculate W respectively 1 ·X 1 、W 2 ·X 2 ;

[0057] Step A2: Query unit calculates [X 1 ]←EMat(pk1,X 1 ,1), and then send it to the prediction unit. The prediction unit also calculates [W 1 ·X 2 ]←MulMat([W 1 ],X 2 , 0), where MulMat() is a homomorphic matrix multiplication algorithm. Then, the prediction unit generates a random matrix R with the same dimensions as W·X 1 , calculate [W 1 ·X 2 -R 1 ]←AddMat([W 1 ·X 2 ],-R 1 ), where AddMat() is a homomorphic matrix addition algorithm;

[0058] Step A3: The prediction unit receives [X 1 ], generate a random matrix R with the same dimensions as W·X 2 Calculate [W 2 ·X 1 ]←MatMul([W 2 ],X 1 , 1) and [W 2 ·X 1 -R 2 ]←AddMat([W 2 ·X 1 ],-R 2 ). Then the ciphertext [W 1 ·X 2 -R 1 ] and [W 2 ·X 1 -R2 ] is sent to the query unit.

[0059] Step A4: The query unit receives [W 1 ·X 2 -R 1 ] and [W 2 ·X 1 -R 2 ], perform decryption: W 1 ·X 2 -R 1 ←DMat(sk1,[W 1 ·X 2 -R 1 ]), W 2 ·X 1 -R 2 ←DMat(sk1,[W 2 ·X 1 -R 2 ]), where sk1 is the private key of the query unit and DMat() is the homomorphic matrix decryption algorithm. Then calculate: Y 1 =W 1 ·X 1 +(W 1 ·X 2 -R 1 )+(W 2 ·X 1 -R 2 ).

[0060] Step A5: The prediction unit calculates Y 2 =W 2 ·X 2 +R 1 +R 2 , so that Y=Y 1 +Y 2 .

[0061] In one embodiment, the nonlinear layer two-party secure computation protocol specifically implements the computation steps including:

[0062] Step B1: Query unit calculates C 1 =A 1 -B 1 , then C 1 After homomorphic encryption, it is sent to the prediction unit;

[0063] Step B2: The prediction unit receives C 1 Then, first calculate C 2 =A 2 -B 2 , then generate a positive random value U; then homomorphically calculate the ciphertext [(C 1 +C2 )·U], and send it to the query unit;

[0064] Step B3: After receiving the query unit, it performs a decryption operation and obtains H=(C 1 +C 2 )·U; then calculate D=SIGN(H), where SIGN() represents a function that takes positive and negative signs; then the query unit sends D to the prediction unit;

[0065] Step B4: The query unit and prediction unit output Y according to the positive or negative value of D. 1 、Y 2 :If D is a positive number, set Y 1 =A 1 、Y 2 =A 2 ; If D is non-positive, set Y 1 =B 1 、Y 2 =B 2 .

[0066] In one embodiment, the homomorphic matrix encryption scheme supports homomorphic matrix multiplication calculations and includes five algorithms: the key generation algorithm KeyGen, the matrix encryption algorithm EMat, the matrix decryption algorithm DMat, the addition algorithm AddMat, and the multiplication algorithm MulMat. In the following, (m,n) and (n,k) are used to represent the dimensions of the multiplicand matrix and the multiplier matrix respectively, and N is used to represent the maximum number of elements that can be encrypted in a homomorphic encryption ciphertext. Let The following uses M i,j Represents the element in the i-th row and j-th column of the matrix M.

[0067] The key generation algorithm KeyGen takes as input the security parameter λ and outputs a pair of homomorphically encrypted public and private keys (pk, sk) based on the security parameter λ.

[0068] The matrix encryption algorithm EMat takes as input the public key pk, the plaintext matrix M to be encrypted, and a Boolean value b indicating whether the encrypted matrix is ​​a multiplicand matrix or a multiplier matrix. Its output is the ciphertext matrix [M]. EMat first calls the matrix encoding algorithm Pack b Encode the elements of M diagonally into z vectors of length N, denoted as M (0) ,M (1) ,...,M (k-1) Then, each encoded vector is packaged and homomorphically encrypted, resulting in a total of z ciphertexts. Finally, these z ciphertexts are output.

[0069] Vector decryption algorithm DMat, whose inputs are: the homomorphic encryption private key sk and the ciphertext [V]. Its output is the plaintext matrix E. DMat first calls the decryption algorithm of homomorphic encryption, uses the homomorphic encryption private key sk to decrypt the ciphertext [V], and obtains the plaintext vector V. Then V is divided into k sub-vectors of length , denoted as V 0 , V 1 ,..., V k-1 . Finally, these k sub-vectors are encoded into a plaintext matrix E as the output, where

[0070] Addition algorithm AddMat, whose inputs are: the public key pk of homomorphic encryption, a ciphertext [V] and a plaintext matrix R. AddMat first encodes R into k vectors of length , denoted as R 0 , R 1 ,..., R k-1 , where Then, AddMat connects R 0 , R 1 ,..., R k-1 head to tail to form a vector R'. Next, it calls the encryption algorithm of packed homomorphic encryption, uses the public key pk to encrypt R', and obtains the ciphertext [R']. Finally, it homomorphically adds [R'] and [V] to obtain the ciphertext [V + R'] as the output.

[0071] Multiplication algorithm MulMat, whose inputs are: the ciphertext matrix [M], the plaintext matrix G and the Boole type value b used to indicate whether the ciphertext matrix is the multiplicand matrix or the multiplier matrix. Its output is the ciphertext of matrix multiplication: if b = 0, then output [M·G]; otherwise, output [G·M]. MulMat first divides the ciphertext matrix [M] into [M (0) , [M (1) ,..., [M (k-1) . Then, it calls the matrix encoding algorithm Pack 1-b to encode the elements of G diagonally into z vectors of length N, denoted as G (0) , G (1) ,..., G (z -1) . Next, for each G (i) (0 ≤ i < z), it calls the packed homomorphic encryption Hadamard product algorithm to homomorphically calculate the product of the corresponding elements of G (i) and [M (i) to obtain the ciphertext C (i) . Finally, it adds all the ciphertexts C (i) (0 ≤ i < z) by calling the packed homomorphic encryption addition algorithm and outputs the obtained ciphertext.

[0072] Figure 3 Illustrates the execution of the EncMat(), Mul(), and DecVec() algorithms in sequence to calculate the multiplication of a 3×4 matrix and a 4×2 matrix.

[0073] In one embodiment, the matrix encoding algorithm includes the multiplicand matrix encoding algorithm Pack 0 and the multiplier matrix encoding algorithm Pack 1 .

[0074] The encoding algorithm Pack 0 of the multiplicand matrix, whose input is the m×n multiplicand matrix W and the output is z vectors W (0) , W (1) ,..., W (z-1) of length N. Among them, W (i) (0≤i<z) is formed by connecting k identical sub-vectors. Each sub-vector encodes t diagonals of the matrix W. The sub-vector is as follows (the "mod n" is omitted in the second subscript):

[0075]

[0076] The encoding algorithm Pack 1 of the multiplier matrix, whose input is the n×k multiplier matrix X and the output is z vectors X (0) , X (1) ,..., X (z-1) of length N. Among them, X (i) (0≤i<z) is formed by connecting k different sub-vectors. Each sub-vector encodes t diagonals of the matrix X. The j-th sub-vector is as follows (the "mod n" and "mod k" are omitted in the first and second subscripts respectively):

[0077]

[0078] The present invention can provide:

[0079] 1. An efficient single-server convolutional neural network privacy protection prediction outsourcing system: Allows the use of a third-party cloud server to provide convolutional neural network prediction services to users in an outsourced manner, without revealing model privacy and user privacy throughout the process. This system breaks the dependence on non-colluding servers and only requires a single server to complete the privacy protection prediction outsourcing service.

[0080] 2. Efficient Linear-Layer Two-Party Secure Computation Protocol: This system's carefully designed matrix encoding algorithm encodes matrix elements diagonally into ciphertext vectors, then performs homomorphic matrix multiplication using simple homomorphic addition and multiplication operations. This avoids expensive homomorphic rotation operations and fully utilizes the ciphertext space, reducing the amount of ciphertext transmitted and the number of homomorphic operations in the protocol.

[0081] 3. Approximation-free two-party secure computation protocol for nonlinear layers. Existing privacy-preserving prediction outsourcing solutions for convolutional neural networks use approximate computations for the nonlinear layer, reducing model prediction accuracy. The nonlinear two-party secure computation protocol designed in this system faithfully performs the comparison operations in the convolutional neural network prediction process, ensuring prediction accuracy.

[0082] It should be understood that the embodiments described above are only some of the embodiments of the present invention, rather than all of the embodiments. In addition, the technical features of the various embodiments or individual embodiments provided by the present invention may be arbitrarily combined with each other to form a feasible technical solution. Such combination is not restricted by the order of steps and / or structural composition mode, but must be based on the ability of ordinary technicians in this field to implement it. When the combination of technical solutions is mutually inconsistent or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection claimed by the present invention.

[0083] It should be understood that the above description of the preferred embodiment is relatively detailed and cannot be regarded as limiting the scope of protection of the patent of the present invention. Under the guidance of the present invention, ordinary technicians in this field can also make substitutions or modifications without departing from the scope of protection of the claims of the present invention, which all fall within the scope of protection of the present invention. The scope of protection requested by the present invention shall be based on the attached claims.

Claims

1. A privacy-preserving prediction outsourcing system for convolutional neural networks based on two-party secure computation, characterized by: It includes a convolutional neural network, a convolutional neural network model deployment unit, a query unit, and a prediction unit; The convolutional neural network is composed of linear layers and nonlinear layers connected alternately. During prediction, the output of the previous layer is the input of the next layer. The calculation of the linear layer is realized by matrix multiplication, and the operation of the nonlinear layer is element-by-element comparison operation. The convolutional neural network model deployment unit deploys the convolutional neural network model in the query unit and the prediction unit in an encrypted manner through secret sharing and homomorphic encryption; During prediction, the query unit secretly shares the query data with the prediction unit, and then the query unit and the prediction unit run a two-party secure computing protocol to complete the prediction of the convolutional neural network model, and each obtains a share of the prediction result; finally, the prediction unit sends the share to the query unit for the query unit to restore the prediction result; The two-party secure computation protocol includes a linear layer two-party secure computation protocol and a nonlinear layer two-party secure computation protocol; Before executing the linear layer two-party secure computation protocol or the nonlinear layer two-party secure computation protocol, the query unit and the prediction unit secretly share the input of the linear layer or the nonlinear layer; after executing the protocol, the query unit and the prediction unit secretly share the output of the linear layer or the nonlinear layer; thereby alternately connecting the linear layer two-party secure computation protocol and the nonlinear layer two-party secure computation protocol to complete the prediction process of the convolutional neural network; The linear layer two-party secure computation protocol starts with the query unit holding the secret share X of the input matrix X. 1 , the secret share W of the parameter matrix W 1 ; The prediction unit holds the secret share of the input matrix X 2 , the secret share W of the parameter matrix 2 and the encrypted parameter matrix share [W 1 ], where X = X 1 +X 2 , W=W 1 +W 2 After the protocol is executed, the query unit and the prediction unit obtain Y 1 、Y 2 , where Y = Y 1 +Y 2 ; The nonlinear layer two-party secure computation protocol starts with the query unit holding the secret share A of the input elements A and B. 1 、B 1 , the forecast unit holds the other half of the share A 2 、B 2 , where A=A 1 +A 2 , B=B 1 +B 2 After executing the protocol, the query unit and the prediction unit secretly share the output of the nonlinear layer Y = max(A, B), that is, the query unit and the prediction unit obtain Y 1 、Y 2 , where Y = Y 1 +Y 2 ; The linear layer two-party secure computation protocol specifically implements the computation steps including: Step A1: The query unit and prediction unit calculate W respectively 1 ·X 1 、W 2 ·X 2 ; Step A2: Query unit calculates [X 1 ]←EMat(pk1,X 1 ,1), and then send it to the prediction unit; the prediction unit also calculates [W 1 ·X 2 ]←MulMat([W 1 ],X 2 , 0), where pk1 is the public key of the query unit, EMat() is the homomorphic matrix encryption algorithm, and MulMat() is the homomorphic matrix multiplication algorithm; then, the prediction unit generates a random matrix R with the same dimension as W·X 1 , calculate [W 1 ·X 2 -R 1 ]←AddMat([W 1 ·X 2 ],-R 1 ), where AddMat() is a homomorphic matrix addition algorithm; Step A3: The prediction unit receives [X 1 ], generate a random matrix R with the same dimensions as W·X 2 ; Calculate [W 2 ·X 1 ]←MatMul([W 2 ],X 1 , 1) and [W 2 ·X 1 -R 2 ]←AddMat([W 2 ·X 1 ],-R 2 ); then the ciphertext [W 1 ·X 2 -R 1 ] and [W 2 ·X 1 -R 2 ] is sent to the query unit; Step A4: The query unit receives [W 1 ·X 2 -R 1 ] and [W 2 ·X 1 -R 2 ], perform decryption: W 1 ·X 2 -R 1 ←DMat(sk1,[W 1 ·X 2 -R 1 ]), W 2 ·X 1 -R 2 ←DMat(sk1,[W 2 ·X 1 -R 2 ]), where sk1 is the private key of the query unit, DMat() is the homomorphic matrix decryption algorithm; then calculate: Y 1 =W 1 ·X 1 +(W 1 ·X 2 -R 1 )+(W 2 ·X 1 -R 2 ); Step A5: The prediction unit calculates Y 2 =W 2 ·X 2 +R 1 +R 2 , so that Y=Y 1 +Y 2 .

2. The privacy-preserving prediction outsourcing system for convolutional neural networks based on two-party secure computation according to claim 1, characterized in that: The convolutional neural network model deployment unit is used to deploy the convolutional neural network model between the query unit and the prediction unit in a confidential manner without leaking the parameters of the model; the convolutional neural network model is divided into a linear layer and a nonlinear layer according to the type of operation involved; the convolutional neural network model deployment unit adopts different deployment methods for the linear layer and the nonlinear layer, and deploys them layer by layer.

3. The privacy-preserving prediction outsourcing system for convolutional neural networks based on two-party secure computation according to claim 2, characterized in that: For the linear layer, the convolutional neural network model deployment unit generates a random number R of the same dimension as the parameter matrix W w , set W 1 =R w 、W 2 =WR w , W 1 and W 2 Send them to the query unit and prediction unit respectively; then use the query unit's public key pk1 to encrypt W using the homomorphic matrix encryption algorithm EMat() 1 , get the ciphertext [W 1 ]←EMat(pk1,W 1 ,0); Finally, the ciphertext [W 1 ] is sent to the prediction unit; for the nonlinear layer, the convolutional neural network model deployment unit sends the nonlinear layer type to the query unit and the prediction unit.

4. The privacy-preserving prediction outsourcing system for convolutional neural networks based on two-party secure computation according to claim 1, characterized in that: The nonlinear layer two-party secure computation protocol specifically implements the computation steps including: Step B1: Query unit calculates C 1 =A 1 -B 1 , then C 1 After homomorphic encryption, it is sent to the prediction unit; Step B2: The prediction unit receives C 1 Then, first calculate C 2 =A 2 -B 2 , then generate a positive random value U; then homomorphically calculate the ciphertext [(C 1 +C 2 )·U], and send it to the query unit; Step B3: After receiving the query unit, it performs a decryption operation and obtains H=(C 1 +C 2 )·U; then calculate D=SIGN(H), where SIGN() represents a function that takes positive and negative signs; then the query unit sends D to the prediction unit; Step B4: The query unit and prediction unit output Y according to the positive or negative value of D. 1 、Y 2 :If D is a positive number, set Y 1 =A 1 、Y 2 =A 2 ; If D is non-positive, set Y 1 =B 1 、Y 2 =B 2 .

5. The privacy-preserving prediction outsourcing system for convolutional neural networks based on two-party secure computation according to claim 1 is characterized by: The matrix encryption algorithm EMat(), the input includes a public key pk, a plaintext matrix M to be encrypted and a Boolean type value b for indicating whether the encrypted matrix is ​​a multiplicand matrix or a multiplier matrix; The output is the ciphertext matrix [M]; EMat() first calls the matrix encoding algorithm Pack b Encode the elements of M diagonally into z vectors of length N, denoted as M (0) ,M (1) ,...,M (k-1) , N represents the maximum number of elements that can be encrypted in a homomorphically encrypted ciphertext; then, each encoded vector is packaged and homomorphically encrypted to obtain a total of z ciphertexts; finally, these z ciphertexts are output; The vector decryption algorithm DMat() takes as input the homomorphic encryption private key sk and the ciphertext [V], and outputs the plaintext matrix E. DMat() first calls the homomorphic encryption decryption algorithm and decrypts the ciphertext [V] using the homomorphic encryption private key sk to obtain the plaintext vector V. Then V is divided into k vectors of length The subvector of 0 ,V 1 ,...,V k-1 , N represents the maximum number of elements that can be encrypted in a homomorphically encrypted ciphertext; finally, these k subvectors are encoded into a plaintext matrix E as the output, where The addition algorithm AddMat() takes as input a homomorphically encrypted public key pk, a ciphertext [V] and a plaintext matrix R; AddMat() first encodes R into k numbers of length A vector, denoted as R 0 ,R 1 ,...,R k-1 ,in N represents the maximum number of elements that can be encrypted in a homomorphic encryption ciphertext; then, AddMat() adds R 0 ,R 1 ,...,R k-1 Connect the two vectors head to tail to form a vector R'; then call the packed homomorphic encryption algorithm and encrypt R' with the public key pk to obtain the ciphertext [R']; finally, homomorphically add [R'] and [V] to obtain the ciphertext [V+R'] as the output; The multiplication algorithm MulMat() takes as input a ciphertext matrix [M], a plaintext matrix G, and a boolean type value b indicating whether the ciphertext matrix is the multiplicand matrix or the multiplier matrix, and outputs the ciphertext of matrix multiplication: if b = 0, it outputs [M·G]; otherwise, it outputs [G·M]; MulMat() first divides the ciphertext matrix [M] into [M (0) ,[M (1) ,...,[M (k-1) ; then, it calls the matrix encoding algorithm Pack 1-b to encode the elements of G diagonally into z vectors of length N, denoted as G (0) ,G (1) ,...,G (z-1) ; then, for each G (i) , 0 ≤ i < z, it calls the packed homomorphic encryption Hadamard product algorithm to homomorphically calculate the product of the corresponding elements of G (i) and [M (i) , obtaining the ciphertext C (i) ; finally, it adds all the ciphertexts C (i) by calling the packed homomorphic encryption addition algorithm and outputs the resulting ciphertext.

6. The privacy-preserving prediction outsourcing system for convolutional neural networks based on two-party secure computation according to claim 5, characterized in that: The matrix coding algorithm includes the multiplicand matrix coding algorithm Pack 0 or Multiplier Matrix Coding Algorithm Pack 1 ; The encoding algorithm Pack of the multiplicand matrix 0 , the input is the multiplicand matrix W of m×n, and the output is vectors W of length N (0) , W (1) ,..., W (z-1) ; where W (i) is formed by connecting k identical sub-vectors, 0 ≤ i < z; each sub-vector encodes diagonals of the matrix W; the sub-vector is where "mod n" is omitted in the second subscript; The encoding algorithm Pack of the multiplier matrix 1 , whose input is an n×k multiplier matrix X and whose output is z vectors X of length N (0) , X (1) ,..., X (z-1) ; where X (i) is formed by concatenating k different sub-vectors, 0 ≤ i < z; each sub-vector encodes t diagonals of the matrix X; the j-th sub-vector is where "mod n" and "mod k" are omitted in the first and second indices respectively.

7. A privacy-preserving prediction outsourcing product for a convolutional neural network based on two-party secure computation, comprising computer program instructions, characterized in that: When the computer program instructions are executed on a computer, they drive the execution of the convolutional neural network privacy-preserving prediction outsourcing system based on two-party secure computing as described in any one of claims 1 to 6.

Images