Convolutional neural network inference method and system based on confusion mode component homomorphic encryption

Abstract

The invention relates to the technical field of data security, and discloses a convolutional neural network reasoning method and system based on confusion mode component homomorphic encryption, and the method comprises the steps: generating a public and private key pair of a confusion mode component homomorphic encryption algorithm by a user side, and transmitting a public key to a server; the server encrypts parameters of the convolutional neural network model based on the public key to obtain a ciphertext convolutional neural network model; the user side performs homomorphic encryption on to-be-predicted privacy data based on the public key and then sends the to-be-predicted privacy data to the server; the server performs ciphertext reasoning by using a ciphertext convolutional neural network model, and returns a ciphertext reasoning result to the user side; and the user side decrypts the data through the private key to obtain a final prediction result. The method not only can be used for improving the security of image recognition of the neural network and protecting the privacy of the user, but also improves the recognition efficiency compared with a traditional homomorphic encryption algorithm.

Description

Technical Field

[0001] The present invention relates to the technical field of data security, and more specifically, to a method and system for convolutional neural network inference based on homomorphic encryption of obfuscated modular components. Background Art

[0002] With the rapid development of the Internet, the amount of data has shown exponential growth, and big data and cloud computing have become important driving forces in the field of information technology. In the context of cloud computing, data is centrally stored in the cloud. Combined with big data, users can use machine learning algorithms to process and analyze massive and complex data, greatly improving resource utilization and computing efficiency, and providing more accurate and intelligent decision-making support for individuals, enterprises, and institutions. Especially in the fields of financial business, healthcare, urban management, scientific research, etc., machine learning outsourced computing plays an important role. However, the fact that third-party cloud service providers are responsible for data storage and processing has also led to the problem of privacy leakage.

[0003] On the one hand, as a deep learning model, the convolutional neural network (CNN) performs excellently in visual tasks such as image classification. For example, it is used in medical diagnosis to assist in detecting tumors or retinal lesions; in security monitoring to achieve face detection and recognition, improving system performance; in e-commerce to identify product pictures to optimize search efficiency. On the other hand, fully homomorphic encryption is one of the common cryptographic techniques used in the privacy protection scenario of machine learning. It supports computing on ciphertexts and can process ciphertext data while ensuring the confidentiality of private data.

[0004] However, mainstream homomorphic encryption algorithms often have a large computational overhead and are difficult to apply to complex neural network models. Therefore, how to improve the security and efficiency in the neural network computing process has become a technical problem that urgently needs to be solved. Summary of the Invention

[0005] In view of this, the present invention provides a method and system for convolutional neural network inference based on homomorphic encryption of obfuscated modular components, which can not only be used to improve the security of image recognition of neural networks and protect the privacy of users, but also improve the recognition efficiency compared with traditional homomorphic encryption algorithms.

[0006] To achieve the above object, the present invention adopts the following technical solutions:

[0007] A method for convolutional neural network inference based on homomorphic encryption of obfuscated modular components, comprising:

[0008] The user side generates a public-private key pair of the homomorphic encryption algorithm of obfuscated modular components and sends the public key to the server;

[0009] The server encrypts the parameters of the convolutional neural network model based on the public key to obtain a ciphertext convolutional neural network model;

[0010] The client encrypts the privacy data to be predicted using the public key and then sends it to the server;

[0011] The server performs ciphertext inference using the ciphertext convolutional neural network model and returns the ciphertext inference result to the client;

[0012] The client decrypts it with the private key to obtain the final prediction result.

[0013] Preferably, the client uses the public key pk of homomorphic encryption to encrypt the privacy data to be predicted into ciphertext data ;

[0014] wherein, represents the result after encrypting the privacy data using the public key pk of homomorphic encryption with confusion modulus components, , and respectively represent the height and width of the image to be predicted.

[0015] Preferably, the server uses the ciphertext convolutional neural network model to perform ciphertext inference, including:

[0016] The convolutional layer extracts features by operating the convolution kernel with the ciphertext image, and the calculation formula is:

[0017]

[0018] wherein, represents the value of the output feature map at the position , m and n are the height and width of the convolution kernel, is the bias term, is the weight value of the convolution kernel at the position , and represent the horizontal and vertical step sizes, and respectively represent the ciphertext multiplication operation and the ciphertext addition operation based on the confusion modulus components, represents the value of the corresponding position of the input ciphertext image after step size adjustment;

[0019] The activation layer performs polynomial blind addition, blind multiplication, and blind exponentiation on the output of the convolutional layer;

[0020] Average pooling performs average pooling on the output of the activation layer and completes the dimensionality reduction operation by calculating the average value within the local area;

[0021] The fully connected layer aggregates features by performing ciphertext operations on the ciphertext data output by average pooling and the ciphertext weight matrix.

[0022] A convolutional neural network inference system based on homomorphic encryption of obfuscated modular components, including a client and a server,

[0023] The client includes:

[0024] Public and private key pair generation module: used to generate the public and private keys of the homomorphic encryption algorithm of obfuscated modular components;

[0025] Data encryption module: used to perform homomorphic encryption on the private data to be predicted based on the public key;

[0026] Result decryption module: used to decrypt the ciphertext inference result received from the server through the private key to obtain the final prediction result;

[0027] The server includes:

[0028] Model encryption module: used to encrypt the parameters of the convolutional neural network model based on the received public key to obtain the ciphertext convolutional neural network model;

[0029] Ciphertext inference module: performs ciphertext inference using the ciphertext convolutional neural network model.

[0030] Preferably, the specific implementation process of the data encryption module is:

[0031] Use the public key pk of homomorphic encryption to encrypt the private data to be predicted into ciphertext data ;

[0032] Among them, represents the result after encrypting the private data using the public key pk of homomorphic encryption of obfuscated modular components, , and respectively represent the height and width of the image to be predicted.

[0033] Preferably, the ciphertext inference module processes through the convolutional layer, activation layer, average pooling, and fully connected layer in sequence, and the specific implementation process is:

[0034] The convolutional layer extracts features by performing operations on the ciphertext image with the convolutional kernel, and the calculation formula is:

[0035]

[0036] Among them, represents the value of the output feature map at position , m and n are the height and width of the convolutional kernel, is the bias term, is the weight value of the convolutional kernel at position , and represent the step sizes in the horizontal and vertical directions, and respectively represent the ciphertext multiplication operation and the ciphertext addition operation based on the confusion modulus component, represents the value at the corresponding position of the input ciphertext image after step size adjustment;

[0037] The activation layer performs blind addition, blind multiplication, and blind exponentiation of polynomials on the output of the convolutional layer;

[0038] Average pooling performs average pooling on the output of the activation layer and completes the dimensionality reduction operation by calculating the average value within the local region;

[0039] The fully connected layer aggregates features by performing ciphertext operations on the ciphertext data output by average pooling and the ciphertext weight matrix.

[0040] As can be seen from the above technical solutions, compared with the prior art, the present invention discloses a convolutional neural network inference method and system based on homomorphic encryption of confusion modulus components, which not only ensures data security, but also has high inference accuracy and low computational and communication overheads. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only the embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained according to the provided drawings without creative efforts.

[0042] Figure 1 It is a flowchart of a convolutional neural network inference method based on homomorphic encryption of confusion modulus components provided by the present invention.

[0043] Figure 2 It is a schematic diagram of a convolutional neural network inference system based on homomorphic encryption of confusion modulus components provided by the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0044] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0045] An embodiment of the present invention discloses a convolutional neural network inference method based on homomorphic encryption of confusion modulus components, as Figure 1 shown, including:

[0046] The client generates a public-private key pair of the confusion modulus component homomorphic encryption algorithm and sends the public key to the server;

[0047] The server encrypts the parameters of the convolutional neural network model based on the public key to obtain a ciphertext convolutional neural network model;

[0048] The client homomorphically encrypts the private data to be predicted based on the public key and sends it to the server;

[0049] The server uses the ciphertext convolutional neural network model for ciphertext inference and returns the ciphertext inference result to the client;

[0050] The client decrypts it with the private key to obtain the final prediction result.

[0051] For the convenience of describing the scheme, assume that the private data to be predicted by the user is an image with w*h pixels in size, and the ciphertext convolutional neural network model is for image inference. Specifically, the steps are as follows:

[0052] The client generates a public-private key pair (pk, sk) of the confusion modulus component homomorphic encryption algorithm and sends the public key pk to the server;

[0053] The server uses the public key pk to encrypt the parameters of the convolutional neural network model it holds and transforms it into a ciphertext convolutional neural network model.

[0054] The client uses the public key pk of homomorphic encryption to encrypt the private data to be predicted it holds into ciphertext data , where represents the result of encrypting the private data using the public key pk of the confusion modulus component homomorphic encryption. Here , and the client sends the ciphertext data C to the server;

[0055] The ciphertext convolutional neural network model includes a convolutional layer, an activation layer, an average pooling layer, and a fully connected layer.

[0056] In the convolutional layer, the convolution kernel size is m×n, and the convolution kernel is set as , is the weight value of the convolution kernel at the position , and its calculation process is:

[0057] , where is calculated according to matrix convolution , , and represent the step sizes in the horizontal and vertical directions, denotes the width of the output feature map obtained after the convolution operation, denotes the height of the output feature map obtained after the convolution operation,

[0058]

[0059] where, denotes the value of the output feature map at position m and n are the height and width of the convolution kernel, is the bias term, is the weight value of the convolution kernel at position and denote the strides in the horizontal and vertical directions, and denote the ciphertext multiplication operation and ciphertext addition operation based on the confusion module component respectively, denotes the value of the corresponding position of the input ciphertext image after the stride adjustment;

[0060] The activation layer usually contains non-linear operations, such as PReLU, etc., and uses a specific approximate low-order polynomial function such as to replace the activation function, enabling the neural network in the ciphertext state to perform approximate non-linear operations. Therefore, in the ciphertext convolutional neural network, the activation layer realizes the effect of the activation function by performing blind addition, blind multiplication, and blind power operations of polynomials on the output of the convolutional layer.

[0061] Since homomorphic encryption does not support comparison operations (such as the maximum value selection in max pooling), the pooling layer uses average pooling instead of max pooling. Average pooling completes the dimensionality reduction operation by calculating the average value within the local region. In the homomorphic environment, the division in average pooling is replaced by the pre-computed reciprocal, thus converting the operation into a series of homomorphic addition and multiplication operations, avoiding direct division operations. Pre-computation means calculating the reciprocal of the number to be used for division in advance (usually the number of elements in the pooling window) and saving it. In this way, when performing average pooling, the original required division operation can be achieved by multiplying this pre-computed and saved reciprocal.

[0062] The fully connected layer completes feature aggregation through the ciphertext data output by average pooling and the ciphertext weight matrix through ciphertext operations. The ciphertext data performs blind multiplication with the weights of the fully connected layer element by element, and then accumulates and sums using blind addition.

[0063] The inference process of the present invention is carried out entirely in the ciphertext domain, and the result output by the server is still in ciphertext format, ensuring that the user's privacy is fully protected during the inference process.

[0064] The server returns the ciphertext inference result to the client, and the client uses the private key sk to decrypt it to obtain the final prediction result.

[0065] An embodiment of the present invention provides a convolutional neural network inference system based on homomorphic encryption of confusion modulus components, as Figure 2 shown, including a client and a server.

[0066] The client includes:

[0067] Public and private key pair generation module: used to generate the public and private keys of the homomorphic encryption algorithm of confusion modulus components;

[0068] Data encryption module: used to perform homomorphic encryption on the privacy data to be predicted based on the public key;

[0069] Result decryption module: used to decrypt the ciphertext inference result received from the server through the private key to obtain the final prediction result;

[0070] The server includes:

[0071] Model encryption module: used to encrypt the parameters of the convolutional neural network model based on the received public key to obtain the ciphertext convolutional neural network model;

[0072] Ciphertext inference module: performs ciphertext inference using the ciphertext convolutional neural network model.

[0073] For the specific implementation process of each module in the system of this embodiment, refer to the method embodiment, which will not be elaborated here.

[0074] In this specification, each embodiment is described in a progressive manner. The key point of each embodiment is to illustrate the differences from other embodiments. The same or similar parts among the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the description of the method part.

[0075] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present invention. Various modifications to these embodiments will be obvious to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to these embodiments shown herein, but will be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A convolutional neural network inference method based on homomorphic encryption of confused modular components, characterized in that, Including: The client generates a public-private key pair of the obfuscated modulus component homomorphic encryption algorithm and sends the public key to the server; The server encrypts the parameters of the convolutional neural network model based on the public key to obtain the encrypted convolutional neural network model; The client performs homomorphic encryption on the private data to be predicted based on the public key and sends it to the server; The server uses the encrypted convolutional neural network model for encrypted inference and returns the encrypted inference result to the client; The client decrypts the encrypted inference result through the private key to obtain the final prediction result.

2. The convolutional neural network inference method based on homomorphic encryption of confused modular components according to claim 1, wherein The client uses the public key pk of homomorphic encryption to encrypt the private data to be predicted into ciphertext data ; Among them, represents the result of encrypting the private data using the public key pk of the homomorphic encryption with the confusion modulus component, , and represent the height and width of the image to be predicted, respectively.

3. A convolutional neural network inference method based on homomorphic encryption of confused modular components according to claim 1, characterized in that When the server uses the encrypted convolutional neural network model for encrypted inference, it includes successively: The convolutional layer extracts features by performing operations on the encrypted image with the convolutional kernel, and the calculation formula is: ； Among them, represents the value of the output feature map at position , m and n are the height and width of the convolution kernel, is the bias term, is the weight value of the convolution kernel at position ; and represent the horizontal and vertical strides, and represent the ciphertext multiplication operation and ciphertext addition operation based on the confusion module component respectively, represents the value of the corresponding position of the input ciphertext image after stride adjustment; The activation layer performs blind addition, blind multiplication, and blind power operations of polynomials on the output of the convolutional layer; Average pooling performs average pooling on the output of the activation layer and completes the dimensionality reduction operation by calculating the average value within the local area; The fully connected layer aggregates features by performing encrypted operations on the encrypted data output by average pooling and the encrypted weight matrix.

4. A convolutional neural network inference system based on homomorphic encryption of obfuscated modular components, characterized in that Including the client and the server, The client includes: The public-private key pair generation module: used to generate the public and private keys of the obfuscated modulus component homomorphic encryption algorithm; The data encryption module: used to perform homomorphic encryption on the private data to be predicted based on the public key; The result decryption module: used to decrypt the encrypted inference result received from the server through the private key to obtain the final prediction result; The server includes: The model encryption module: used to encrypt the parameters of the convolutional neural network model based on the received public key to obtain the encrypted convolutional neural network model; The encrypted inference module: uses the encrypted convolutional neural network model for encrypted inference.

5. A convolutional neural network inference system based on homomorphic encryption of confused modular components according to claim 4, characterized in that, The specific implementation process of the data encryption module is: Encrypt the private data to be predicted using the public key pk of homomorphic encryption into ciphertext data ; Among them, denotes the result of encrypting the private data using the public key pk of the homomorphic encryption with the confusion module component, , and respectively denote the height and width of the image to be predicted.

6. A convolutional neural network inference system based on homomorphic encryption of confused modular components according to claim 4, characterized in that, The specific implementation process of the encrypted inference module is: The convolutional layer extracts features by performing operations on the encrypted image with the convolutional kernel, and the calculation formula is: ； Among them, represents the value of the output feature map at position , m and n are the height and width of the convolution kernel, is the bias term, is the weight value of the convolution kernel at position , and represent the strides in the horizontal and vertical directions, and represent the ciphertext multiplication operation and the ciphertext addition operation based on the confusion module component respectively, represents the value of the corresponding position of the input ciphertext image after stride adjustment; The activation layer performs blind addition, blind multiplication, and blind power operations of polynomials on the output of the convolutional layer; Average pooling performs average pooling on the output of the activation layer and completes the dimensionality reduction operation by calculating the average value within the local area; The fully connected layer aggregates features by performing encrypted operations on the encrypted data output by average pooling and the encrypted weight matrix.

Images