A Quantitative Neural Network Privacy Inference Method and System with Dual-branch Structure

Abstract

The present invention relates to the field of privacy-preserving machine learning, and in particular to a method and system for privacy reasoning of a quantitative neural network with a dual-branch structure. The method comprises: a trusted third party generates a public-private key pair according to a multi-key fully homomorphic cryptographic algorithm; the client divides local data into sensitive data and non-sensitive data, and encrypts the sensitive data according to the public key to generate ciphertext data; the server uses a privacy reasoning model of a quantitative neural network with a dual-branch structure to perform privacy reasoning on the data to obtain a prediction result of the encrypted state; the client uses a private key to decrypt to obtain a plaintext reasoning result. The present invention provides a learning method and system for privacy reasoning of a quantitative neural network with a dual-branch structure to better improve computing efficiency and data security. While ensuring the data security of multiple parties, the privacy reasoning task of the neural network is completed with lower communication costs, fewer interactions and higher computing efficiency.

Description

Technical Field

[0001] The present invention relates to the field of privacy-preserving machine learning technology, and in particular to a quantized neural network privacy inference method and system with a dual-branch structure. Background Art

[0002] Neural networks can provide users with reasoning services such as text generation, image generation, and search engines in the cloud, but at the same time they also face security threats from malicious attackers, which may lead to sensitive data leakage. In order to protect data privacy, the concept of privacy reasoning of neural networks is proposed to protect the input data, output reasoning results, specific information of the model (including model structure, parameters, loss function, etc.) and user information during the reasoning process. Privacy reasoning technology allows users to use neural network reasoning services without leaking their sensitive data. In addition, this technology can also effectively protect the data security of service providers. The privacy protection technologies currently used for privacy reasoning of neural networks are:

[0003] 1. Differential privacy method: This method protects data privacy by adding noise to the data or using inductive methods to cover up certain sensitive attributes in the data until a third party cannot distinguish them, making the data unrecoverable. For example, a certain degree of random noise is added to the output of the neural network so that changes in a single input sample will not have a significant impact on the output. In this way, even if a malicious attacker knows all inputs except a certain input, it is impossible to accurately determine whether this specific input is included in the training data. However, this method will cause a decrease in model accuracy and cannot guarantee confidentiality during parameter transmission. The essence of this method still requires that the data be transmitted to other places, and it is usually necessary to trade off between privacy protection capabilities and model prediction performance. In addition, this method is less secure than the homomorphic encryption method and cannot guarantee security under quantum attacks.

[0004] 2. Secure multi-party computing method: Secure multi-party computing method is a collection of numerous protocols and algorithms, including obfuscated circuits, secret sharing, and oblivious transmission, which allows two or more users to jointly evaluate a function without disclosing their input data to other users. Researchers can develop protocols for evaluating neural networks based on these methods, while ensuring the privacy of user input data while implementing neural network reasoning. However, these protocols usually require a lot of communication during the calculation process, which leads to the problem of high network latency.

[0005] 3. Trusted Execution Environment Method: A trusted execution environment is a hardware-protected computing environment, typically composed of hardware and software, used to create a trusted execution area in a computer system. In this protected execution environment, computing operations are isolated and encrypted to ensure the confidentiality and integrity of data and code. The trusted execution environment provides a secure computing environment that protects sensitive data from attacks by malware or the operating system. In privacy reasoning of neural networks, sensitive data can be processed within a trusted execution environment to ensure that the data does not leave the protected environment. In this way, even if an attacker has access to the system outside the execution environment, they cannot obtain sensitive data, thereby improving data privacy security. However, this technology requires special hardware support, which greatly limits its availability on various devices.

[0006] 4. Fully homomorphic encryption method: Fully homomorphic encryption is a special encryption technology in cryptography that allows calculations to be performed in an encrypted state, including addition, multiplication, and even more complex calculations, without decrypting the data. Traditional encryption algorithms need to decrypt data before they can be calculated after encrypting it, while fully homomorphic encryption technology makes it possible to perform calculations in a ciphertext state. Based on the ability of fully homomorphic encryption to perform calculations on ciphertext, users can encrypt the input data using a fully homomorphic encryption scheme, and the server performs calculations on the ciphertext data to complete the privacy reasoning of the neural network. In this process, the user's input data and the reasoning results of the neural network are both in a secret state, which can effectively protect the privacy and security of the data. In addition, most of the current fully homomorphic encryption schemes are built based on difficult problems on lattices, so they have the advantage of being resistant to quantum attacks. However, the computational efficiency of fully homomorphic encryption is usually low, and there are compatibility issues with some components in the neural network (such as nonlinear calculations such as activation functions). In addition, when there are multiple users performing privacy reasoning on neural networks, sharing the same set of keys will also lead to potential security issues.

[0007] There have been some research contents on privacy-preserving machine learning methods based on fully homomorphic encryption. However, there are still aspects that need to be optimized in the current technical solutions. The following is a comparative analysis of the existing patent solutions and the present invention.

[0008] Ciphertext prediction method, related device and storage medium (application publication number: CN115510466A, application publication date: 2022.12.23) A ciphertext prediction method is proposed, which is mainly used in the machine learning model prediction process between the feature party device and the requesting party device. In this method, the requesting party device generates a model prediction request, and then the feature party device uses the public key to homomorphically encrypt the feature value prediction result to generate an encrypted ciphertext, and finally sends the encrypted ciphertext to the requesting party device. The requesting party device uses the private key to process the ciphertext to obtain the prediction result of the machine learning model. The method supports the collaborative work of multiple feature party devices and realizes the generation of encrypted ciphertext through homomorphic operations. The shortcomings include that the use of the CKKS homomorphic algorithm to perform SIMD on the ciphertext can improve the amortization performance, but the computational efficiency is poor when the amount of data is small. Different keys cannot be generated for different requesting parties to ensure the data security of other requesting parties when a certain requesting party is compromised.

[0009] Multi-party privacy-preserving machine learning method based on homomorphic encryption and trusted hardware (application publication number: CN113553610A, application publication date: 2021.10.26) proposes an innovative multi-party privacy-preserving machine learning method. In this method, each data holder uses a secret key and trusted hardware to encrypt the original data and transmit the encrypted data to the server. The server then performs machine learning modeling by applying homomorphic addition and homomorphic multiplication to these encrypted data. The uniqueness of this method is that it utilizes the technical characteristics of fully homomorphic encryption and combines the use of trusted hardware to simulate bootstrapping and execute activation function functions. In this way, a model with the same style accuracy as machine learning training on unencrypted data can be obtained while preserving the integrity and privacy of the original data. Although this method provides a powerful privacy protection mechanism and can protect the security of data while maintaining model accuracy, it also has some limitations that need to be noted. First, this method has relatively high requirements for the system operating environment and requires specific hardware support to work properly. Second, in the current version, there is no optimization for computational efficiency, so the computational complexity of this method is relatively high, which may affect its feasibility in practical scenarios.

[0010] A multi-user privacy-preserving machine learning method and device based on multi-key fully homomorphic encryption (application publication number: CN114844621A, application publication date: 2022.8.2) proposes a multi-user privacy-preserving machine learning method and device based on multi-key fully homomorphic encryption, which uses a multi-key fully homomorphic encryption scheme MKTFHE to encrypt the data of multiple data providers in a multi-user scenario, and then uses a multi-key ciphertext data set to train the machine learning model, and then distributes the obtained model ciphertext to each data provider, and finally decrypts it to obtain the final training model. While ensuring high security, the invention completes the privacy computing task with lower communication cost and fewer interactions. However, the invention does not consider optimizing the adopted fully homomorphic encryption scheme or machine learning algorithm, resulting in poor computational efficiency. Summary of the invention

[0011] In order to solve the problem of poor computational efficiency and data security issues under multiple users in the existing neural network privacy reasoning technology based on fully homomorphic encryption, the embodiment of the present invention provides a quantitative neural network privacy reasoning method and system with a dual-branch structure to better improve computational efficiency and data security. While ensuring the data security of multiple parties, the privacy reasoning task of the neural network is completed with lower communication cost, fewer interactions and higher computational efficiency. The technical solution is as follows:

[0012] On the one hand, a quantized neural network privacy inference method with a dual-branch structure is provided. The method is implemented by a quantized neural network privacy inference system with a dual-branch structure. The system includes a trusted third party, a client, and a server. The method includes:

[0013] S1. The trusted third party generates a public parameter set based on the multi-key fully homomorphic cryptographic algorithm, generates a public-private key pair based on the public parameter set and the key generation algorithm, and distributes the public-private key pair to the client and server.

[0014] S2. The client divides the local data into sensitive data and non-sensitive data, encrypts the sensitive data according to the public key to generate ciphertext data, and sends the ciphertext data and non-sensitive data to the server.

[0015] S3. The server uses a dual-branch structured quantized neural network privacy inference model to perform privacy inference on ciphertext data and non-sensitive data to obtain the predicted results of the encryption status, and returns the predicted results of the encryption status to the corresponding client.

[0016] S4. The client uses the private key to decrypt the encrypted prediction result to obtain the plaintext inference result.

[0017] Optionally, the client in S2 divides local data into sensitive data and non-sensitive data, including:

[0018] The client classifies local data into sensitive data and non-sensitive data based on sensitivity and importance.

[0019] Optionally, a dual-branch structured quantized neural network privacy inference model includes: a ciphertext branch and a plaintext branch.

[0020] The construction process of the dual-branch structured quantized neural network privacy inference model includes:

[0021] A neural network privacy reasoning model is designed according to the data type and scale in different task scenarios. The neurons in the neural network privacy reasoning model are randomly inactivated using pruning operations. The neural network privacy reasoning model after the pruning operation is trained using plaintext data and quantization-aware training methods to obtain a quantitative neural network privacy reasoning model with a dual-branch structure.

[0022] Optionally, the server in S3 uses a dual-branch structured quantized neural network privacy inference model to perform privacy inference on the ciphertext data and non-sensitive data to obtain a prediction result of the encryption status, including:

[0023] The server encrypts the ciphertext data according to the ciphertext branch and the multi-key fully homomorphic cryptographic algorithm to obtain encrypted ciphertext data, and extracts characteristic information of the ciphertext data from the encrypted ciphertext data.

[0024] The server dynamically adjusts non-sensitive data according to the plaintext branches and system security level requirements, and extracts the characteristic information of non-sensitive data.

[0025] The characteristic information of the ciphertext data and the characteristic information of the non-sensitive data are integrated to obtain the prediction result of the encryption status.

[0026] Optionally, extracting characteristic information of the encrypted ciphertext data includes:

[0027] The characteristic information of the encrypted ciphertext data is extracted based on the characteristic information of the non-sensitive data.

[0028] On the other hand, a dual-branch structured quantized neural network privacy inference system is provided, which is applied to a dual-branch structured quantized neural network privacy inference method, and the system includes a trusted third party, a client, and a server; wherein:

[0029] A trusted third party is used to generate a public parameter set based on a multi-key fully homomorphic cryptographic algorithm, generate a public-private key pair based on the public parameter set and the key generation algorithm, and distribute the public-private key pair to the client and server.

[0030] The client is used to divide local data into sensitive data and non-sensitive data, encrypt the sensitive data according to the public key to generate ciphertext data, and send the ciphertext data and non-sensitive data to the server; use the private key to decrypt the prediction results of the encrypted state to obtain the plaintext inference results.

[0031] The server uses a dual-branch structured quantized neural network privacy inference model to perform privacy inference on ciphertext data and non-sensitive data to obtain the prediction results of the encryption status, and returns the prediction results of the encryption status to the corresponding client.

[0032] Optionally, the client is further configured to:

[0033] The client classifies local data into sensitive data and non-sensitive data based on sensitivity and importance.

[0034] Optionally, a dual-branch structured quantized neural network privacy inference model includes: a ciphertext branch and a plaintext branch.

[0035] The server is further used to:

[0036] A neural network privacy reasoning model is designed according to the data type and scale in different task scenarios. The neurons in the neural network privacy reasoning model are randomly inactivated using pruning operations. The neural network privacy reasoning model after the pruning operation is trained using plaintext data and quantization-aware training methods to obtain a quantitative neural network privacy reasoning model with a dual-branch structure.

[0037] Optionally, the server is further configured to:

[0038] The server encrypts the ciphertext data according to the ciphertext branch and the multi-key fully homomorphic cryptographic algorithm to obtain encrypted ciphertext data, and extracts characteristic information of the ciphertext data from the encrypted ciphertext data.

[0039] The server dynamically adjusts non-sensitive data according to the plaintext branches and system security level requirements, and extracts the characteristic information of non-sensitive data.

[0040] The characteristic information of the ciphertext data and the characteristic information of the non-sensitive data are integrated to obtain the prediction result of the encryption status.

[0041] Optionally, the server is further configured to:

[0042] The characteristic information of the encrypted ciphertext data is extracted based on the characteristic information of the non-sensitive data.

[0043] On the other hand, a quantized neural network privacy inference device with a dual-branch structure is provided, and the quantized neural network privacy inference device with a dual-branch structure comprises: a processor; a memory, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, any one of the methods of the quantized neural network privacy inference method with a dual-branch structure is implemented.

[0044] On the other hand, a computer-readable storage medium is provided, wherein at least one instruction is stored in the storage medium, and the at least one instruction is loaded and executed by a processor to implement any one of the above-mentioned dual-branch structured quantized neural network privacy inference methods.

[0045] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

[0046] In an embodiment of the present invention, a privacy reasoning framework with a dual-branch structure can be used to perform hierarchical processing according to the sensitivity and importance of the data, thereby improving computing efficiency. Sensitive information is encrypted based on a multi-key fully homomorphic encryption algorithm, and there is no need to decrypt the data during the reasoning process, thereby ensuring the privacy and security of the data in multi-user scenarios. The pruning operation randomly inactivates some neurons in the neural network, increasing the robustness of the model training while maintaining a low accumulator bit width. In addition, quantizing the neural network into the corresponding integer form is conducive to the calculation of homomorphic addition, homomorphic multiplication and nonlinear activation functions. Finally, a quantized neural network privacy reasoning system with a dual-branch structure is formed, allowing multiple clients and servers to efficiently use the neural network privacy reasoning service while protecting their own data security. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0048] Figure 1 This is a flow chart of a quantized neural network privacy inference method with a dual-branch structure provided by an embodiment of the present invention;

[0049] Figure 2 This is a framework diagram of a quantized neural network privacy inference method with a dual-branch structure provided by an embodiment of the present invention;

[0050] Figure 3 It is a block diagram of a quantized neural network privacy inference system with a dual-branch structure provided by an embodiment of the present invention;

[0051] Figure 4It is a structural schematic diagram of a quantized neural network privacy inference device with a dual-branch structure provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0052] The technical solution of the present invention is described below in conjunction with the accompanying drawings.

[0053] In the embodiments of the present invention, words such as "exemplarily" and "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "example" in the present invention should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of the word "example" is intended to present the concept in a specific way. In addition, in the embodiments of the present invention, the meaning expressed by "and / or" can be both, or it can be either of the two.

[0054] In the embodiments of the present invention, "image" and "picture" can sometimes be used interchangeably. It should be noted that when the difference between them is not emphasized, the meanings they intend to express are the same. "of", "corresponding, relevant" and "corresponding" can sometimes be used interchangeably. It should be noted that when the difference between them is not emphasized, the meanings they intend to express are the same.

[0055] In the embodiments of the present invention, sometimes a subscript such as W1 may be written as a non-subscript such as W1. When the difference is not emphasized, the meanings to be expressed are the same.

[0056] In order to make the technical problems, technical solutions and advantages to be solved by the present invention more clear, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.

[0057] The embodiment of the present invention provides a quantized neural network privacy inference method with a dual-branch structure, which can be implemented by a quantized neural network privacy inference system with a dual-branch structure. Figure 1 , Figure 2 The flowchart of the quantized neural network privacy inference method with a dual-branch structure is shown. The processing flow of the method may include the following steps:

[0058] S1. The trusted third party generates a public parameter set based on the multi-key fully homomorphic cryptographic algorithm, generates a public-private key pair based on the public parameter set and the key generation algorithm, and distributes the public-private key pair to the client and server.

[0059] In one feasible implementation, a trusted third party Initialize the multi-key fully homomorphic cryptographic algorithm to generate security parameters , call To generate a common parameter set , and call it with this parameter set To generate public and private keys , providing higher security protection for sensitive data and distributing the generated keys to the client , sending the evaluation for homomorphic computation to the server .

[0060] Furthermore, the present invention is a solution to realize privacy reasoning of neural networks based on the fully homomorphic encryption algorithm TFHE. The fully homomorphic encryption algorithm supports fully homomorphic operations on encrypted data. The algorithm consists of 5 specific algorithms:

[0061] Public parameter generation algorithm : Security parameters As input, and output a common parameter set ;

[0062] Key generation algorithm : Generate a public-private key pair for each client using a public parameter set;

[0063] Encryption Algorithm : This algorithm uses the public key To encrypt single bit of data , and then returns a ciphertext ;

[0064] Decryption algorithm :The algorithm inputs the encrypted ciphertext and the private keys of the relevant parties , decrypt the ciphertext into plaintext bits ;

[0065] Homomorphic AND-NOT Gate Evaluation Algorithm :The algorithm inputs two ciphertexts and , and the public keys of the parties involved , output the ciphertext after the two input ciphertexts are homomorphically calculated and the NAND gate .

[0066] S2. The client divides the local data into sensitive data and non-sensitive data, encrypts the sensitive data according to the public key to generate ciphertext data, and sends the ciphertext data and non-sensitive data to the server.

[0067] In a feasible implementation, the client Received from a trusted third party After distributing the keys, call To pre-process your local data, divide it into sensitive and non-sensitive data according to sensitivity and importance, aiming to grade information of different sensitivity levels, generate ciphertext and send it to the server together with non-sensitive information Perform calculations.

[0068] For example, in a face recognition task, the image used for training or reasoning includes a face area and a background area that is not related to the face. The present invention uses a face detection algorithm to determine the face area in the image and segment it. Subsequently, the face area is regarded as sensitive information and encrypted using a highly secure fully homomorphic encryption algorithm; the non-face area is regarded as non-sensitive information and processed using a less secure differential privacy method. During the training or reasoning process, the sensitive information branch can unidirectionally use the data of the non-sensitive information branch.

[0069] S3. The server uses a dual-branch structured quantized neural network privacy inference model to perform privacy inference on ciphertext data and non-sensitive data to obtain the predicted results of the encryption status, and returns the predicted results of the encryption status to the corresponding client.

[0070] Optionally, a dual-branch structured quantized neural network privacy inference model includes: a ciphertext branch and a plaintext branch.

[0071] The construction process of the dual-branch structured quantized neural network privacy inference model includes:

[0072] A neural network privacy reasoning model is designed according to the data type and scale in different task scenarios. The neurons in the neural network privacy reasoning model are randomly inactivated using pruning operations. The neural network privacy reasoning model after the pruning operation is trained using plaintext data and quantization-aware training methods to obtain a quantitative neural network privacy reasoning model with a dual-branch structure.

[0073] The following is an explanation of some of the terms used in the embodiments of the present application to facilitate understanding by those skilled in the art:

[0074] Neural network: A neural network is a computational model that mimics the structure of the human brain and consists of multiple interconnected neuron layers. It is used to process complex data pattern recognition and learning tasks. It automatically adjusts internal parameters by learning a large amount of data to achieve various tasks such as classification, regression and prediction of input data.

[0075] Fully homomorphic encryption: Fully homomorphic encryption is an advanced encryption technology that allows computation to be performed in an encrypted state, even if various operations (such as addition and multiplication) are performed on encrypted data, and then the result is decrypted to obtain the same result as the same operation performed on the unencrypted data. This form of encryption has important privacy protection applications, especially in fields such as cloud computing and secure multi-party computing.

[0076] Multi-key fully homomorphic encryption: Multi-key fully homomorphic encryption is an extension of fully homomorphic encryption technology, which allows multiple key holders to operate on the same encrypted data without decrypting the data. In the multi-key fully homomorphic encryption scheme, each key holder has a unique key pair and can operate on the data in an encrypted state, including performing various operations such as addition and multiplication. This scheme allows data to be distributed to multiple clients for calculation, greatly reducing the risk of single point failure.

[0077] Quantized neural network: Quantized neural network refers to converting the weights and activation values ​​of the neural network into fixed-point numbers or binary numbers with lower bits to reduce the storage space of the model and accelerate the reasoning process. Through quantization, the neural network can significantly reduce the computing and memory requirements while maintaining relatively high performance, and is also more suitable for homomorphic evaluation calculations.

[0078] Model pruning: Model pruning is a technique used to reduce the size of deep learning models and improve inference speed. It reduces the number of model parameters by removing unimportant weights or neuron connections in the model and pruning low-magnitude weights. This can reduce the computational and storage overhead of the model while maintaining relatively high accuracy.

[0079] In a feasible implementation, the neural network structure (number of hidden layers, number of neurons, loss function, optimizer, etc.) is designed according to the data type and scale in different task scenarios; the neurons in the neural network are randomly inactivated by pruning operations to reduce computational complexity; the neural network is trained using plaintext data (unencrypted data); during the model training process, the model is converted into the corresponding integer form using quantization-aware training methods.

[0080] Furthermore, a compiler is used to compile the trained model into a corresponding fully homomorphic circuit representation; the input data, model weights, and all intermediate values ​​are quantized into integers; a multi-key fully homomorphic encryption algorithm is used to generate a separate private key for each client. , use their respective private keys to encrypt the input data and then send it to the server; the server performs homomorphic computing on the encrypted data to achieve the reasoning task, and sends the generated reasoning results to the client in a secret form; the client uses its own private key to decrypt and obtain the final reasoning result.

[0081] Specifically, the server further includes the following steps to obtain the pre-trained quantized neural network model in the ciphertext branch:

[0082] Data preparation: Prepare the data set for training, including input data and corresponding labels;

[0083] Model construction: Design and build a quantization-aware model, including an input layer, several quantization-aware layers, and an output layer. Use a binary quantization activation function and select an appropriate loss function and optimizer.

[0084] Model training: Use the prepared data and built model to train the quantization-aware model. In this process, the model parameters are updated according to the loss function and optimizer to achieve better performance on the training data.

[0085] Evaluation and tuning: After training is completed, the model is evaluated and tuned, including adjusting hyperparameters, adding training data, adjusting model structure, etc.

[0086] Optionally, the server in S3 uses a dual-branch structured quantized neural network privacy inference model to perform privacy inference on the ciphertext data and non-sensitive data to obtain a prediction result of the encryption state, which may include S31-S33:

[0087] S31, Server The ciphertext data is encrypted according to the ciphertext branch and the multi-key fully homomorphic cryptographic algorithm to obtain the encrypted ciphertext data, and a neural network calculation is performed on the ciphertext data to extract feature information.

[0088] In a feasible implementation mode, the server After receiving the information sent by the client and the public key sent by the trusted third party, call Perform homomorphic computing and perform neural network inference calculations on the ciphertext data encrypted by the multi-key fully homomorphic cryptographic algorithm in the ciphertext branch. In the process of processing sensitive information in the ciphertext branch, the non-sensitive information in the plaintext branch can be used unidirectionally to better generate features.

[0089] S32, Server Dynamically adjust non-sensitive data according to plaintext branches and system security level requirements to extract feature information of non-sensitive data.

[0090] In a feasible implementation mode, the server After receiving the information sent by the client and the public key sent by the trusted third party, call When performing homomorphic computing in the plaintext branch, you can consider using technologies such as differential privacy to ensure data security based on factors such as data type and system security requirements, and extract features from the input data through a neural network.

[0091] S33, integrating the characteristic information of the ciphertext data and the characteristic information of the non-sensitive data to obtain a prediction result of the encryption state.

[0092] In a feasible implementation, feature information extracted by a quantized neural network under multi-key fully homomorphic encryption is obtained in the ciphertext branch, and then the feature information extracted in the plaintext branch is fused and returned to the client.

[0093] S4. The client uses the private key to decrypt the encrypted prediction result to obtain the plaintext inference result.

[0094] In a feasible implementation, after receiving the secret inference result returned by the server, the client calls Decrypt and obtain the final model inference result.

[0095] The neural network modeling of the present invention supports various types of tasks, including tasks such as computer vision, natural language processing or recommendation systems.

[0096] The present invention uses a dual-branch structure to divide data according to sensitivity and importance, which greatly improves computing efficiency. Based on the multi-key fully homomorphic cryptographic algorithm, it realizes provable security in the post-quantum era and reduces the security risks of single-point failures in multi-user scenarios, while completing the privacy reasoning calculation tasks of quantized neural networks with lower communication costs and fewer interactions. In addition, the present invention further reduces reasoning delays through neural network quantization, programmable bootstrapping and pruning operations.

[0097] In view of the poor computational efficiency of the current privacy inference process of neural networks and the potential security risks in multi-user scenarios, this paper proposes a dual-branch structured quantized neural network privacy inference computation method, which improves the computational efficiency of privacy inference by hierarchical processing of input data and pruning of quantized neural networks, and adopts a multi-key fully homomorphic cryptographic algorithm to ensure data security in multi-user scenarios. Compared with traditional privacy-preserving machine learning solutions, this paper has higher computational efficiency and security, and the key technical points are:

[0098] The dual-branch structure is used to perform hierarchical processing according to the sensitivity and importance of the data, which greatly reduces the amount of data required for encryption processing and greatly improves the computing efficiency;

[0099] In order to prevent the leakage of user data caused by a single user being hacked in a multi-user scenario, the present invention adopts a multi-key fully homomorphic encryption algorithm to generate different keys for different clients, greatly reducing the risk of single point failure.

[0100] In order to ensure the compatibility of neural networks and the homomorphic cryptographic algorithm MKTFHE, the present invention designs and implements a quantized neural network, which supports the use of programmable bootstrapping technology to calculate the nonlinear activation function in the neural network;

[0101] In order to further reduce the accumulator bit width and improve the inference efficiency, the present invention adopts model pruning technology to randomly inactivate redundant neurons in the quantized neural network, thereby improving the inference efficiency while ensuring accuracy.

[0102] Compared with the previous reasoning method based on the CKKS homomorphic scheme, the quantized neural network reasoning method based on the multi-key fully homomorphic encryption scheme MKTFHE proposed in the present invention has lower reasoning latency when the amount of data is small and SIMD cannot be used for amortization.

[0103] In an embodiment of the present invention, a privacy reasoning framework with a dual-branch structure can be used to perform hierarchical processing according to the sensitivity and importance of the data, thereby improving computing efficiency. Sensitive information is encrypted based on a multi-key fully homomorphic encryption algorithm, and there is no need to decrypt the data during the reasoning process, thereby ensuring the privacy and security of the data in multi-user scenarios. The pruning operation randomly inactivates some neurons in the neural network, increasing the robustness of the model training while maintaining a low accumulator bit width. In addition, quantizing the neural network into the corresponding integer form is conducive to the calculation of homomorphic addition, homomorphic multiplication and nonlinear activation functions. Finally, a quantized neural network privacy reasoning system with a dual-branch structure is formed, allowing multiple clients and servers to efficiently use the neural network privacy reasoning service while protecting their own data security.

[0104] Figure 3 The block diagram of a quantized neural network privacy inference system with a dual-branch structure is shown according to an exemplary embodiment. The system is used for a quantized neural network privacy inference method with a dual-branch structure. Figure 3 , the system includes a trusted third party, a client and a server; wherein:

[0105] A trusted third party is used to generate a public parameter set based on a multi-key fully homomorphic cryptographic algorithm, generate a public-private key pair based on the public parameter set and the key generation algorithm, and distribute the public-private key pair to the client and server.

[0106] The client is used to divide local data into sensitive data and non-sensitive data, encrypt the sensitive data according to the public key to generate ciphertext data, and send the ciphertext data and non-sensitive data to the server; use the private key to decrypt the prediction results of the encrypted state to obtain the plaintext inference results.

[0107] The server uses a dual-branch structured quantized neural network privacy inference model to perform privacy inference on ciphertext data and non-sensitive data to obtain the prediction results of the encryption status, and returns the prediction results of the encryption status to the corresponding client.

[0108] Optionally, the client is further configured to:

[0109] The client classifies local data into sensitive data and non-sensitive data based on sensitivity and importance.

[0110] Optionally, a dual-branch structured quantized neural network privacy inference model includes: a ciphertext branch and a plaintext branch.

[0111] The server is further used to:

[0112] A neural network privacy reasoning model is designed according to the data type and scale in different task scenarios. The neurons in the neural network privacy reasoning model are randomly inactivated using pruning operations. The neural network privacy reasoning model after the pruning operation is trained using plaintext data and quantization-aware training methods to obtain a quantitative neural network privacy reasoning model with a dual-branch structure.

[0113] Optionally, the server is further configured to:

[0114] The server encrypts the ciphertext data according to the ciphertext branch and the multi-key fully homomorphic cryptographic algorithm to obtain encrypted ciphertext data, and extracts characteristic information of the ciphertext data from the encrypted ciphertext data.

[0115] The server dynamically adjusts non-sensitive data according to the plaintext branches and system security level requirements, and extracts the characteristic information of non-sensitive data.

[0116] The characteristic information of the ciphertext data and the characteristic information of the non-sensitive data are integrated to obtain the prediction result of the encryption status.

[0117] Optionally, the server is further configured to:

[0118] The characteristic information of the encrypted ciphertext data is extracted based on the characteristic information of the non-sensitive data.

[0119] In an embodiment of the present invention, a privacy reasoning framework with a dual-branch structure can be used to perform hierarchical processing according to the sensitivity and importance of the data, thereby improving computing efficiency. Sensitive information is encrypted based on a multi-key fully homomorphic encryption algorithm, and there is no need to decrypt the data during the reasoning process, thereby ensuring the privacy and security of the data in multi-user scenarios. The pruning operation randomly inactivates some neurons in the neural network, increasing the robustness of the model training while maintaining a low accumulator bit width. In addition, quantizing the neural network into the corresponding integer form is conducive to the calculation of homomorphic addition, homomorphic multiplication and nonlinear activation functions. Finally, a quantized neural network privacy reasoning system with a dual-branch structure is formed, allowing multiple clients and servers to efficiently use the neural network privacy reasoning service while protecting their own data security.

[0120] Figure 4 is a schematic diagram of the structure of a quantized neural network privacy inference device with a dual-branch structure provided by an embodiment of the present invention. Figure 4 As shown, the dual-branch structured quantized neural network privacy inference device may include the above Figure 3 The quantized neural network privacy inference system with a dual-branch structure is shown. Optionally, the quantized neural network privacy inference device 410 with a dual-branch structure may include a first processor 2001.

[0121] Optionally, the dual-branch structured quantized neural network privacy inference device 410 may further include a memory 2002 and a transceiver 2003 .

[0122] The first processor 2001, the memory 2002 and the transceiver 2003 may be connected via a communication bus.

[0123] Combine the following Figure 4 The components of the dual-branch structured quantized neural network privacy inference device 410 are specifically introduced:

[0124] The first processor 2001 is the control center of the dual-branch structured quantized neural network privacy inference device 410, which can be a processor or a general term for multiple processing elements. For example, the first processor 2001 is one or more central processing units (CPUs), or an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement an embodiment of the present invention, such as one or more microprocessors (digital signal processors, DSPs), or one or more field programmable gate arrays (field programmable gate arrays, FPGAs).

[0125] Optionally, the first processor 2001 can perform various functions of the dual-branch structured quantized neural network privacy inference device 410 by running or executing a software program stored in the memory 2002 and calling data stored in the memory 2002.

[0126] In a specific implementation, as an embodiment, the first processor 2001 may include one or more CPUs, such as Figure 4 CPU0 and CPU1 are shown in FIG.

[0127] In a specific implementation, as an embodiment, the dual-branch structure quantized neural network privacy inference device 410 may also include multiple processors, such as Figure 4The first processor 2001 and the second processor 2004 are shown in FIG. Each of these processors may be a single-core processor (single-CPU) or a multi-core processor (multi-CPU). The processor here may refer to one or more devices, circuits, and / or processing cores for processing data (e.g., computer program instructions).

[0128] The memory 2002 is used to store the software program for executing the solution of the present invention, and is controlled to be executed by the first processor 2001. The specific implementation method can refer to the above method embodiment, which will not be repeated here.

[0129] Optionally, the memory 2002 may be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 2002 may be integrated with the first processor 2001, or may exist independently, and may be connected to the first processor 2001 through the interface circuit ( Figure 4 (not shown) is coupled to the first processor 2001, which is not specifically limited in this embodiment of the present invention.

[0130] The transceiver 2003 is used to communicate with a network device or a terminal device.

[0131] Optionally, the transceiver 2003 may include a receiver and a transmitter ( Figure 4 The receiver is used to implement a receiving function, and the transmitter is used to implement a sending function.

[0132] Optionally, the transceiver 2003 may be integrated with the first processor 2001, or may exist independently, and may be connected to the first processor 2001 through the interface circuit ( Figure 4 (not shown) is coupled to the first processor 2001, which is not specifically limited in this embodiment of the present invention.

[0133] It should be noted that Figure 4 The structure of the dual-branch structured quantized neural network privacy inference device 410 shown in the figure does not constitute a limitation on the router. The actual knowledge structure recognition device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0134] In addition, the technical effects of the dual-branch structured quantized neural network privacy inference device 410 can refer to the technical effects of the dual-branch structured quantized neural network privacy inference method described in the above method embodiment, and will not be repeated here.

[0135] It should be understood that the first processor 2001 in the embodiment of the present invention may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0136] It should also be understood that the memory in the embodiments of the present invention may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0137] The above embodiments can be implemented in whole or in part by software, hardware (such as circuits), firmware or any other combination. When implemented by software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center by wired (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that contains one or more available media sets. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a tape), an optical medium (for example, a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state hard disk.

[0138] It should be understood that the term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. A and B can be singular or plural. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship, but it may also indicate an "and / or" relationship. Please refer to the context for specific understanding.

[0139] In the present invention, "at least one" means one or more, and "plurality" means two or more. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b, or c can mean: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.

[0140] It should be understood that in various embodiments of the present invention, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0141] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0142] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described equipment, devices and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0143] In the several embodiments provided by the present invention, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0144] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0145] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0146] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, and other media that can store program codes.

[0147] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A quantized neural network privacy inference method with a dual-branch structure, characterized in that: The method is implemented by a quantized neural network privacy inference system with a dual-branch structure, and the system includes a trusted third party, a client, and a server; The method comprises: S1. The trusted third party generates a public parameter set according to a multi-key fully homomorphic cryptographic algorithm, generates a public-private key pair according to the public parameter set and the key generation algorithm, and distributes the public-private key pair to the client; S2. The client divides local data into sensitive data and non-sensitive data, encrypts the sensitive data according to a fully homomorphic encryption algorithm to generate ciphertext data, processes the non-sensitive data according to a differential privacy method, and sends the ciphertext data and the processed non-sensitive data to the server; S3, the server uses a quantized neural network privacy inference model with a dual-branch structure to perform privacy inference on the ciphertext data and the processed non-sensitive data to obtain a prediction result of the encryption state, and returns the prediction result of the encryption state to the corresponding client; S4, the client uses a private key to decrypt the prediction result of the encrypted state to obtain a plaintext reasoning result; The dual-branch structured quantized neural network privacy inference model includes: a ciphertext branch and a plaintext branch; The construction process of the dual-branch structured quantized neural network privacy inference model includes: Design a neural network privacy inference model based on the data type and scale in different task scenarios; use pruning operations to randomly inactivate neurons in the neural network privacy inference model; use plaintext data and quantization-aware training methods to train the neural network privacy inference model after pruning operations to obtain a quantized neural network privacy inference model with a dual-branch structure; The server in S3 uses a quantized neural network privacy inference model with a dual-branch structure to perform privacy inference on the ciphertext data and the processed non-sensitive data to obtain a prediction result of the encryption state, including: The server encrypts the ciphertext data according to the ciphertext branch and the multi-key fully homomorphic cryptographic algorithm to obtain encrypted ciphertext data, and extracts characteristic information of the ciphertext data from the encrypted ciphertext data; the non-sensitive information in the plaintext branch can be used unidirectionally in the process of processing sensitive information in the ciphertext branch; The server dynamically adjusts the processed non-sensitive data according to the plaintext branch and the system security level requirements, and extracts the characteristic information of the non-sensitive data; The characteristic information of the ciphertext data and the characteristic information of the non-sensitive data are integrated to obtain a prediction result of the encryption state.

2. The quantized neural network privacy inference method with a dual-branch structure according to claim 1 is characterized in that: The client in S2 divides the local data into sensitive data and non-sensitive data, including: The client classifies local data into sensitive data and non-sensitive data according to sensitivity and importance.

3. The quantized neural network privacy inference method with a dual-branch structure according to claim 1 is characterized in that: The step of extracting characteristic information of the encrypted ciphertext data from the encrypted ciphertext data comprises: The characteristic information of the encrypted ciphertext data is extracted based on the characteristic information of the non-sensitive data.

4. A quantized neural network privacy inference system with a dual-branch structure, characterized in that: The system includes a trusted third party, a client, and a server; wherein: The trusted third party is used to generate a public parameter set according to a multi-key fully homomorphic cryptographic algorithm, generate a public-private key pair according to the public parameter set and the key generation algorithm, and distribute the public-private key pair to the client; The client is used to divide local data into sensitive data and non-sensitive data, encrypt the sensitive data according to the fully homomorphic encryption algorithm to generate ciphertext data, process the non-sensitive data according to the differential privacy method, and send the ciphertext data and the processed non-sensitive data to the server; use the private key to decrypt the prediction result of the encrypted state to obtain the plaintext reasoning result; The server is used to perform privacy reasoning on the ciphertext data and the processed non-sensitive data using a quantized neural network privacy reasoning model with a dual-branch structure to obtain a prediction result of the encryption state, and return the prediction result of the encryption state to the corresponding client; The dual-branch structured quantized neural network privacy inference model includes: a ciphertext branch and a plaintext branch; The construction process of the dual-branch structured quantized neural network privacy inference model includes: Design a neural network privacy inference model based on the data type and scale in different task scenarios; use pruning operations to randomly inactivate neurons in the neural network privacy inference model; use plaintext data and quantization-aware training methods to train the neural network privacy inference model after pruning operations to obtain a quantized neural network privacy inference model with a dual-branch structure; The server uses a quantized neural network privacy inference model with a dual-branch structure to perform privacy inference on the ciphertext data and the processed non-sensitive data to obtain a prediction result of the encryption state, including: The server encrypts the ciphertext data according to the ciphertext branch and the multi-key fully homomorphic cryptographic algorithm to obtain encrypted ciphertext data, and extracts characteristic information of the ciphertext data from the encrypted ciphertext data; the non-sensitive information in the plaintext branch can be used unidirectionally in the process of processing sensitive information in the ciphertext branch; The server dynamically adjusts the processed non-sensitive data according to the plaintext branch and the system security level requirements, and extracts the characteristic information of the non-sensitive data; The characteristic information of the ciphertext data and the characteristic information of the non-sensitive data are integrated to obtain a prediction result of the encryption state.

5. The quantized neural network privacy inference system with a dual-branch structure according to claim 4 is characterized in that: The client divides local data into sensitive data and non-sensitive data, including: The client classifies local data into sensitive data and non-sensitive data according to sensitivity and importance.

6. A quantized neural network privacy inference device with a dual-branch structure, characterized in that: The dual-branch structured quantized neural network privacy inference device comprises: processor; A memory having computer-readable instructions stored thereon, wherein when the computer-readable instructions are executed by the processor, the method according to any one of claims 1 to 3 is implemented.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program codes, which can be called by a processor to execute the method according to any one of claims 1 to 3.

Images