Efficient dense state joint neural network reasoning method and system

Abstract

The invention discloses an efficient secret state joint neural network inference method. The method comprises the following steps: S100, a data owner C and a model owner # imgabs0 # respectively send privacy data and a privacy model to computational nodes # imgabs1 # and # imgabs2 # in a secret sharing manner; s200, computing nodes # imgabs3 # and # imgabs4 # to run a secret state joint neural network reasoning algorithm, and respectively calculating secret shares Z0 and Z1 of reasoning results by sequentially executing secret state computing layer protocols, the secret state computing layer protocols comprising: S210: a secret state convolution layer algorithm # imgabs5 #; s220, a function layer algorithm # imgabs6 # is activated in a dense state; the invention further discloses an efficient dense state joint neural network inference system. According to the technical scheme, the ReLU algorithm with few communication rounds is designed based on the function secret sharing technology, so that the communication cost required for calculating the activation function is reduced, the convolutional neural network inference sub-protocol is designed by utilizing bilateral comparison and correct symbol discrimination, the inference accuracy is improved, and the inference result is consistent with a plaintext inference result.

Description

Technical Field

[0001] The present invention relates to the field of machine learning as a service technology, and in particular to an efficient dense-state joint neural network reasoning method and system. Background Art

[0002] Machine Learning as a Service is a method of providing machine learning models and algorithms as services, which allows users to access and use powerful machine learning tools and functions in a low-cost and low-complexity way through simple application programming interfaces, and obtain the reasoning capabilities of machine learning models. However, in the process of providing services, machine learning model parameters and users' data to be inferred will be leaked, so there is an urgent need for secure neural network reasoning methods to ensure the correctness of reasoning results and protect the privacy of machine learning models and users' data to be inferred.

[0003] In several scenarios of machine learning as a service, secure neural network reasoning supports users to obtain reasoning results based on the data to be reasoned using a neural network model trained on the server side. In this process, the user's data to be reasoned and the input layer, several operation layers, and the operation layer are set according to the actual neural network structure. Several dense convolution layers, dense activation function layers, dense maximum pooling layers, and dense full connection layers are configured as: server-side model. At present, in the implementation scheme of secure neural network reasoning, cryptography-based methods are a very important type of methods, which are mainly divided into the following categories according to the methods used: (1) Homomorphic encryption method: The data owner first encrypts the data to obtain ciphertext data, and then directly performs neural network inference based on the ciphertext data, and ensures that the obtained structure is consistent with the calculation result in plaintext after decryption.

[0004] (2) Differential privacy method: Before the data holder inputs the data into the neural network reasoning system, he first adds noise to the data to achieve differential privacy protection. During the entire reasoning process, it is ensured that the statistical characteristics of the reasoning results are not significantly affected by the noise, while protecting the user's privacy data from being leaked.

[0005] (3) Trusted Execution Environment Method: The data holder transfers the data to a hardware device with a trusted execution environment, and performs neural network inference calculations in the device. The trusted execution environment can ensure the security and privacy of the data during the inference process and prevent it from being obtained or tampered with by external attackers.

[0006] The above prior art has the following disadvantages: (1) Homomorphic encryption method: The computational complexity of the homomorphic encryption method is large because the amount of data when it is represented in ciphertext is larger than that in plaintext. When calculations are performed directly on the ciphertext data, the computational complexity is very large.

[0007] (2) Differential privacy methods: Differential privacy methods need to introduce noise into the data to protect privacy, and this noise may affect the accuracy of the inference results. Although differential privacy can provide strong privacy protection, in practical applications, in order to achieve a sufficient level of privacy protection, it is usually necessary to add a large amount of noise, which may lead to a decrease in the accuracy of the inference results, especially in scenarios with small data volumes or requiring high-precision results.

[0008] (3) Trusted Execution Environment Method: The trusted execution environment method is highly dependent on hardware devices. Only on hardware devices with a trusted execution environment can secure neural network reasoning calculations be performed. Therefore, the scope of application of the TEE method is limited by hardware devices, and the cost of hardware devices is high, which may not be suitable for large-scale distributed deployment. Summary of the invention

[0009] The main purpose of the present invention is to overcome the shortcomings and disadvantages of the prior art and propose an efficient secret-state joint neural network reasoning method and system. In view of the privacy issues and deployment challenges in neural network reasoning, a secret-state joint computing framework is designed based on additive secret sharing and distributed comparison functions. While ensuring high accuracy and not relying on special hardware, a practical and efficient secret-state joint neural network reasoning method is implemented with low communication cost and fewer interactions.

[0010] To achieve the above object, the present invention proposes an efficient dense-state joint neural network reasoning method, which includes the following steps: S100, Data Holder and the model owner The private data and privacy model are sent to the computing nodes through secret sharing. and ,in and express The secret share of data and models held, and express The secret share of data and models held; S200, computing nodes and Run the secret state federated neural network inference algorithm and calculate the secret shares of the inference results by executing the secret state computing layer protocol in sequence. and , the secret computing layer protocol includes: S210: Dense Convolutional Layer Algorithm ; S220: Dense Activation Function Layer Algorithm , using a distributed comparison function to implement , thereby realizing the linear rectification function commonly used in the activation function , including: S221: Use distributed comparison function for comparison of unsigned numbers; S222: using a symbolic discriminant function; S223: Use function; S230: Dense Max Pooling Layer Algorithm ; S240: Dense fully connected layer algorithm ; S300, computing nodes and Get the secret share of the inference result respectively and , and returns it to the data holder , data holder Reconstructing the secret share , and get the inference result .

[0011] Optionally, in step S210, the convolution operation is implemented by a matrix multiplication operation, and the dense convolution layer algorithm It can be obtained by dense matrix multiplication accomplish.

[0012] Optionally, dense matrix multiplication Matrix multiplication in the form of secret shares can be realized, which is divided into offline phase and online phase. The main process is as follows: S01, Offline phase: Generate multiplication triples by a trusted third party ,satisfy , then the trusted third party will Distribute to ,in Indicates the computing node index; S02, Online Stage: and Computing with multiplication triplets , the steps are as follows: S021, Local Computing ; Local Computing ; S022, Will Send to ; Will Send to , and Local Computing Reconstruction ; S023, Local Computing , Local Computing ; S024, can be verified .

[0013] Optionally, in step S230, the dense state maximum pooling layer algorithm Calculate the maximum value of each window. The algorithm can be implemented by calling the distributed comparison function multiple times. The specific implementation process is as follows: S231: Expand the area to be pooled in the original data matrix into a row vector to obtain a new matrix; S232: For each row vector in the matrix ,calculate ,in is a column vector, The meaning is that middle The maximum value of corresponds to the position , and the rest are ,but for The largest element in the pool, which is the output of the maximum pooling, is calculated as and Using the dense comparison algorithm, we can get The secret share , using dense matrix multiplication to obtain The secret share of , finally get a column vector; S233: Reconstruct the obtained column vector so that it has a pooled output structure.

[0014] Optionally, in step S240, the dense fully connected layer algorithm It can be obtained by dense matrix multiplication accomplish.

[0015] Optionally, dense matrix multiplication It is divided into offline stage and online stage. The main process steps are as follows: S1. Offline phase: Multiplication triples are generated by a trusted third party ,satisfy , then the trusted third party will Distribute to ,in Indicates the computing node index; S2, online stage: and Computing with multiplication triplets , the steps are as follows: S21, Local Computing ; Local Computing ; S22, Will Send to ; Will Send to , and Local Computing Reconstruction ; S23, Local Computing , Local Computing ; S24, can be verified .

[0016] On the other hand, the present invention also proposes an efficient dense joint neural network reasoning system to implement the above method. The system includes an input layer, a calculation layer and an output layer. Owners of private data and models The privacy model held is processed by the input layer, the operation layer and the output layer to obtain the inference result Z, and the operation layer includes the dense joint neural network inference core module.

[0017] Optionally, the dense joint neural network reasoning core module includes a convolution layer SecConv, an activation function layer SecAct, a maximum pooling layer SecMP and a fully connected layer SecFC.

[0018] Optionally, the computing layer includes two computing nodes: and , data holder C will transfer the private data Divided into , respectively sent to and , model owner Privacy Model Divided into and , respectively sent to and , and The private data is processed sequentially through the convolution layer SecConv, the activation function layer SecAct, the maximum pooling layer SecMP, and the fully connected layer SecFC to obtain the secret shares of the inference results. .

[0019] Optionally, a secret share Transmitted to data holder C through the output layer, C reconstructs the secret share , and get the inference result Z.

[0020] Under the offline-online computing paradigm, the present invention designs a secure sub-protocol based on function secret sharing and additive secret sharing for operations such as convolutional layer, activation layer, maximum pooling layer, and fully connected layer in the convolutional neural network reasoning process, and constructs a secure convolutional neural network reasoning system. Compared with the prior art, the present invention has the following advantages and beneficial effects: (1) Correct convolutional neural network reasoning sub-protocol: The present invention utilizes bilateral comparison and correct symbol discrimination, and designs a correct convolutional neural network reasoning sub-protocol based on function secret sharing technology, thereby improving the reasoning accuracy and being consistent with the plaintext reasoning results.

[0021] (2) Communication-efficient activation function algorithm: The present invention utilizes the idea of ​​correct symbol discrimination and function coefficients and designs a ReLU algorithm with fewer communication rounds based on function secret sharing technology, thereby reducing the communication cost required to calculate the activation function. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying creative work.

[0023] Figure 1 A schematic diagram of a distributed comparison function binary tree structure according to an embodiment of the present invention; Figure 2 A system framework diagram of an efficient dense-state joint neural network reasoning method according to an embodiment of the present invention; Figure 3 A schematic diagram of the steps of an efficient dense-state joint neural network reasoning method according to an embodiment of the present invention; Figure 4 A schematic diagram of a dense-state computing layer protocol of an efficient dense-state joint neural network reasoning method according to an embodiment of the present invention; Figure 5 A schematic diagram of an efficient dense-state joint neural network reasoning system according to an embodiment of the present invention.

[0024] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0025] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0026] It should be noted that all directional indications (such as up, down, left, right, front, back, etc.) in the embodiments of the present invention are only used to explain the relative position relationship, movement status, etc. between the components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.

[0027] In addition, the technical solutions between the various embodiments can be combined with each other, but it must be based on the fact that ordinary technicians in the field can implement it. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0028] The present invention proposes an efficient dense-state joint neural network reasoning method and system.

[0029] In view of the privacy issues and deployment challenges in neural network reasoning, the present invention designs a secret joint computing framework based on additive secret sharing and distributed comparison functions. While ensuring high accuracy and not relying on special hardware, it implements a practical and efficient secret joint neural network reasoning method with low communication cost and fewer interactions. The following is an introduction to the background technology: (1) Neural Network Inference as a Service Neural Network Inference as a Service (NNIaaS) is a method of providing neural network model inference capabilities through cloud services. Users do not need to understand the complex model training and deployment process, they only need to upload the data to be inferred to the service platform, and the platform will return the inference results. This service model greatly reduces the threshold for users to use neural network inference, making a wide range of application scenarios possible.

[0030] However, under this service model, the privacy protection of user data and neural network models becomes an important issue. On the one hand, the data uploaded by users may contain sensitive information; on the other hand, the neural network model of the service provider is also part of its core competitiveness, and the leakage of model parameters will lead to economic losses. Therefore, a solution is needed that can effectively protect user data and model privacy while ensuring the accuracy of inference results.

[0031] Based on secure multi-party computing technology, the present invention proposes an efficient privacy-preserving neural network reasoning method. The neural network reasoning framework is divided into multiple sub-modules, such as addition and multiplication sub-modules and comparison sub-modules, and then the secret addition and multiplication protocols are designed using additive secret sharing, and the secret comparison protocol is designed using distributed comparison functions, thereby ensuring the correctness and privacy of the neural network reasoning service in the trusted third-party scenario.

[0032] (2) Secret addition and multiplication protocols based on additive secret sharing Secret sharing is a basic technical module used in secure multi-party computing, which mainly includes two stages: secret distribution and secret reconstruction. There are secret managers responsible for secret distribution and reconstruction and secret participants who save secret shares. The present invention considers a secret manager and two secret participants (set as ). In the secret sharing scheme, it is defined on a ring, that is, the secret value is an element on the ring, and modular arithmetic addition and modular arithmetic multiplication operations are performed on the ring. Suppose the ring , secret value , whose length is 1 bit. Representation model Specifically, is a list containing all integers in the The set of equivalence classes under .

[0033] The secret distribution and reconstruction operations of the secret sharing scheme are: Secret distribution: Secret managers are randomly selected , then Send to , Send to ,in and Usually called the secret share, denoted by ; Secret Reconstruction: Share the secret Send to secret keeper, Share the secret Sent to the secret keeper, who calculates , reconstruct the secret .

[0034] Among them, the present invention uses The secret value is expressed in the form of The secret shares of the participants are respectively and hold.

[0035] In particular, the present invention requires arithmetic operations, such as addition and multiplication, to be performed on the secret shares. Assuming there is a secret value and The secret has been distributed to and ,at this time have , have For addition, and Need to calculate The secret share ; For multiplication, and Need to calculate The secret share .

[0036] For addition operations, Compute locally , Compute locally It can be verified .

[0037] For multiplication, a common method is to use multiplication triples for calculation. Assume and The secret value has been obtained The secret share and ,in is A random value uniformly distributed on .Then calculate , and send it to ; calculate , and send it to , and then the two parties calculated .at last, calculate , calculate . It can be verified .

[0038] (3) Confidential comparison protocol based on distributed comparison function Distributed comparison function is a cryptographic primitive, often used for comparison between unsigned numbers, expressed as a comparison function ,in Indicates input, Represents the number to be compared. For this function, when the input If , the output is 1, otherwise the output is 0. go through Covering, that is ,but express .for Similarly, there is a binary tree The corresponding special path. Unlike the distributed point function, in the construction of the distributed comparison function, Figure 2-3 As shown, for each non-leaf node, there is an associated The triplet of and Consistent with distributed point functions, new variables added . Give pairs of nodes according to special paths Agreement: If the current node is not a special node, and the node is the left child node of the parent node, then ,otherwise .

[0039] Figure 1 The distributed comparison function binary tree structure is shown.

[0040] The following relationship exists between the triplets of upper and lower adjacent nodes, i.e., parent and child nodes: ,in Will one The long bit string seed is expanded to a Similar to the distributed point function, the distributed comparison function also needs to correct the word pair The output of is corrected. The following discusses the situation of correction words in two cases.

[0041] When the parent node is on a special path and the bit value associated with the parent node is 1, the left child node is a non-special node and the right child node is a special node. According to the convention, the triple of the left child node should be , the triple of the right child node should be In order to meet the requirements for node triples, we now construct the correction word , you can verify the correction results Satisfies the requirement. Next, consider the secret share form of this situation, that is, for the participants , now there is a triple about the parent node , since the parent node is on a special path, , Using a pseudo-random number generator Expand to get , based on the above correction word structure , agreed to be a member of the parent node triple Use only when Correction, it may be helpful to set , which can be verified Meet the node requirements.

[0042] When the parent node is on a special path and the bit value associated with the parent node is 0, according to convention, the triple of the left child node should be , the triple of the right child node should be , now construct the corrective word , you can verify the correction results Satisfies the requirement. Next, consider the secret share form of this situation. , now there is a triple about the parent node , Using a pseudo-random number generator Expand to get , based on the above corrected word structure , let's assume , which can be verified Meet the node requirements.

[0043] When the parent node is not on a special path, its two child nodes are non-special nodes. According to the convention, the triple of the two child nodes should be , the triple of the parent node is , so the triples of the two parties are equal, that is ,further, ,like , both parties make corrections, , meeting the requirements for nodes; if , neither party needs to make corrections, and it can also be verified that this situation meets the requirements for the node.

[0044] like Figures 2 to 4 As shown, in one embodiment of the present invention, the efficient dense-state joint neural network reasoning method includes the following steps: S100, Data Holder and the model owner The private data and privacy model are sent to the computing nodes through secret sharing. and ,in and express The secret share of data and models held, and express The secret share of data and models held; S200, computing nodes and Run the secret state federated neural network inference algorithm and calculate the secret shares of the inference results by executing the secret state computing layer protocol in sequence. and , the dense computing layer protocols include: S210: Dense Convolutional Layer Algorithm , S220: Dense Activation Function Layer Algorithm , using a distributed comparison function to implement , thereby realizing the linear rectification function commonly used in the activation function , including: S221: Use distributed comparison function for comparison of unsigned numbers; S222: using a symbolic discriminant function; S223: Use function; S230: Dense Max Pooling Layer Algorithm ; S240: Dense fully connected layer algorithm ; S300, computing nodes and Get the secret share of the inference result respectively and , and returns it to the data holder , data holder Reconstructing the secret share , and get the inference result .

[0045] Specifically, in step S210, the convolutional layer (Conv) performs a dot product operation on the convolution kernel and the corresponding position element of the input matrix, and obtains each element in the output matrix by moving the convolution kernel, which is mainly used for feature extraction. In the actual calculation process, the convolution operation is implemented by matrix multiplication operation, so the dense convolution layer algorithm It can be obtained by dense matrix multiplication Implementation, dense matrix multiplication Matrix multiplication can be implemented in the form of secret shares, assuming that the computing nodes and Holds the multiplication input to be calculated and , calculated and satisfy . Dense matrix multiplication The specific implementation scheme is as follows, which is divided into offline stage and online stage: S01, Offline phase: Generate multiplication triples by a trusted third party ,satisfy , then the trusted third party will Distribute to ,in Indicates the computing node index; S02, Online Stage: and Computing with multiplication triplets , the steps are as follows: S021, Local Computing ; Local Computing ; S022, Will Send to ; Will Send to , and Local Computing Reconstruction ; S023, Local Computing , Local Computing ; S024, can be verified .

[0046] Specifically, in step S220, the dense activation function layer algorithm The specific implementation is as follows: In neural network reasoning, the activation function often uses the linear rectified function (Rectified Linear Unit, ReLU), which is denoted as the linear rectified function Given input ,if Then output , otherwise output . You can see The main thing is to calculate the comparison function. In the actual calculation process, it is implemented using the secret comparison function, denoted as , If If yes, the output is 1, otherwise the output is 0. , the present invention uses a distributed comparison function to achieve , thereby achieving , the specific implementation plan is as follows: (a) The distributed comparison function is used to compare unsigned numbers. Its scheme is as follows: The specific structure is as follows: S1, :in Represents the safety parameters, Indicates input, Indicates the size of the ring in which the output is located.

[0047] S11. Calculation ,Will Send to , namely the client and the server.

[0048] S2, :in Indicates the index of the participant. express The output value of Represents the input value.

[0049] S21. Participants calculate ,if ,but ;otherwise .

[0050] (b) The functional secret sharing scheme of the symbol discriminant function is given by The specific structure is as follows: S1. Key generation algorithm , the specific steps are as follows: S11. Random sampling ,make ,in , Represents the input mask.

[0051] S12. Calculation ,in It is a key generation algorithm for distributed comparison functions.

[0052] S13. Calculation , the addition of the above formula is in the ring On, where .

[0053] S14. Random sampling , which satisfies ,structure .

[0054] S15, Output .

[0055] S2. Calculation Algorithm ,in Indicates the index of the participant. express The output value of Represents the input value. The specific execution steps are as follows: S21. Parses as , respectively calculated ,in It is the calculation algorithm of the distributed comparison function.

[0056] S22. Calculation .

[0057] S23. Calculation , output .if , ,otherwise , for Represents a signed number.

[0058] (c) The function secret sharing scheme for functions is given by The specific structure is as follows: S1. Key generation algorithm , the specific steps are as follows: S11. Random sampling ,make ,in , Represents the input mask.

[0059] S12, take ,calculate ,in is a key generation algorithm for distributed comparison functions, Indicates that the output is 2 in the loop The number on.

[0060] S13. Calculation , the addition of the above formula is in the ring On, where .

[0061] S14. Random sampling ,satisfy ,structure .

[0062] S15, Output .

[0063] S2. Calculation Algorithm ,in Indicates the index of the participant. express The output value of Represents the input value. The specific execution steps are as follows: S21. Parses as , respectively calculated ,in It is the calculation algorithm of the distributed comparison function.

[0064] S22. Calculation ,calculate .

[0065] S23. Calculation , output .if , ,otherwise , for Represents a signed number.

[0066] Specifically, in step S230, the dense maximum pooling layer algorithm Calculate the maximum value of each window. The algorithm can be implemented by calling the distributed comparison function multiple times. The specific implementation process is as follows: S231: Expand the area to be pooled in the original data matrix into a row vector to obtain a new matrix; S232: For each row vector in the matrix ,calculate ,in is a column vector, The meaning is that middle The maximum value of corresponds to the position , and the rest are ,but for The largest element in the pool, which is the output of the maximum pooling, is calculated as and Using the dense comparison algorithm, we can get The secret share , using dense matrix multiplication to obtain The secret share of , finally get a column vector; S233: Reconstruct the obtained column vector so that it has a pooled output structure.

[0067] Specifically, in step S240, the dense fully connected layer algorithm It can be obtained by dense matrix multiplication Implementation, dense matrix multiplication It is divided into offline stage and online stage. The main process steps are as follows: S1. Offline phase: Multiplication triples are generated by a trusted third party ,satisfy , then the trusted third party will Distribute to ,in Indicates the computing node index; S2, online stage: and Computing with multiplication triplets , the steps are as follows: S21, Local Computing ; Local Computing ; S22, Will Send to ; Will Send to , and Local Computing Reconstruction ; S23, Local Computing , Local Computing ; S24, can be verified .

[0068] In another embodiment, by providing a systematic operational description of the specific steps of an efficient dense-state joint neural network reasoning method described in Example 1, personnel in this technical field can better understand the solution of the present application.

[0069] like Figure 5 As shown, on the other hand, the present invention also proposes an efficient dense joint neural network reasoning system to implement the above method. The system includes an input layer, a calculation layer and an output layer. The data holder Owners of private data and models The privacy model held is processed by the input layer, the operation layer and the output layer to obtain the inference result Z. The operation layer includes the dense joint neural network inference core module, and the dense joint neural network inference core module includes the convolution layer SecConv, the activation function layer SecAct, the maximum pooling layer SecMP and the fully connected layer SecFC.

[0070] Specifically, the computing layer includes two computing nodes: and , data holder C divides private data X into X 0 and X 1 , respectively sent to and , model owner The privacy model W is divided into W 0 and W 1 , respectively sent to and , and The private data is processed by the convolution layer SecConv, the activation function layer SecAct, the maximum pooling layer SecMP, and the fully connected layer SecFC in sequence to obtain the secret share Z of the inference result. 0 and Z 1， Secret Share Z 0 and Z 1 Transmitted to the data holder C through the output layer, C reconstructs the secret share Z=Z 0 +Z 1 , and get the inference result Z.

[0071] It should be noted here that the system provided in the above embodiment is only illustrated by the division of the above functional modules. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure is divided into different functional modules to complete all or part of the functions described above. The system is an efficient dense joint neural network reasoning method applied to the above embodiment.

[0072] The above description is only a preferred embodiment of the present invention, and does not limit the patent scope of the present invention. All equivalent structural changes made by using the contents of the present invention specification and drawings under the inventive concept of the present invention, or directly / indirectly applied in other related technical fields are included in the patent protection scope of the present invention.

Claims

1. An efficient dense joint neural network reasoning method, characterized in that: The steps include: S100, data holder C and model owner The private data and privacy model are sent to the computing nodes through secret sharing. and ,in and express The secret share of data and models held, and express The secret share of data and models held; S200, computing nodes and Run the secret state federated neural network inference algorithm and calculate the secret shares of the inference results by executing the secret state computing layer protocol in sequence. and , the secret computing layer protocol includes: S210: Dense Convolutional Layer Algorithm ; S220: Dense Activation Function Layer Algorithm , using a distributed comparison function to implement , thereby realizing the linear rectification function commonly used in the activation function , specifically including: S221: Use distributed comparison function for comparison of unsigned numbers; S222: using a symbolic discriminant function; S223: Use function; S230: Dense Max Pooling Layer Algorithm ; S240: Dense fully connected layer algorithm ; S300, computing nodes and Get the secret share of the inference result respectively and , and returns it to the data holder , data holder Reconstructing the secret share , and get the inference result .

2. The efficient dense-state joint neural network reasoning method according to claim 1 is characterized in that: In step S210, the convolution operation is implemented by matrix multiplication operation, and the dense convolution layer algorithm It can be obtained by dense matrix multiplication accomplish.

3. The efficient dense-state joint neural network reasoning method according to claim 2 is characterized in that: Dense matrix multiplication Matrix multiplication in the form of secret shares can be realized, which is divided into offline phase and online phase. The main process is as follows: S01, Offline phase: Generate multiplication triples by a trusted third party ,satisfy , then the trusted third party will Distribute to ,in Indicates the computing node index; S02, Online Stage: and Computing with multiplication triplets , the steps are as follows: S021, Local Computing ; Local Computing ; S022, Will Send to ; Will Send to , and Local Computing Reconstruction ; S023, Local Computing , Local Computing ; S024, can be verified .

4. The efficient dense-state joint neural network reasoning method according to claim 1 is characterized in that: In step S230, the dense maximum pooling layer algorithm Calculate the maximum value of each window. The algorithm can be implemented by calling the distributed comparison function multiple times. The specific implementation process is as follows: S231: Expand the area to be pooled in the original data matrix into a row vector to obtain a new matrix; S232: For each row vector in the matrix ,calculate ,in is a column vector, The meaning is that middle The maximum value of corresponds to the position , and the rest are ,but for The largest element in the pool, which is the output of the maximum pooling, is calculated as and Using the dense comparison algorithm, we can get The secret share , using dense matrix multiplication to obtain The secret share of , finally get a column vector; S233: Reconstruct the obtained column vector so that it has a pooled output structure.

5. The efficient dense-state joint neural network reasoning method according to claim 1 is characterized in that: In step S240, the dense fully connected layer algorithm It can be obtained by dense matrix multiplication accomplish.

6. The efficient dense-state joint neural network reasoning method according to claim 5 is characterized in that: Dense matrix multiplication It is divided into offline stage and online stage. The main process steps are as follows: S1. Offline phase: Multiplication triples are generated by a trusted third party ,satisfy , then the trusted third party will Distribute to ,in Indicates the computing node index; S2, online stage: and Computing with multiplication triplets , the steps are as follows: S21, Local Computing ; Local Computing ; S22, Will Send to ; Will Send to , and Local Computing Reconstruction ; S23, Local Computing , Local Computing ; S24, can be verified .

7. An efficient dense-state joint neural network reasoning system, implementing the efficient dense-state joint neural network reasoning method according to any one of claims 1 to 6, characterized in that: Including input layer, operation layer and output layer, data holder Owners of private data and models The privacy model held is processed by the input layer, the operation layer and the output layer to obtain the inference result Z, and the operation layer includes the dense joint neural network inference core module.

8. The efficient dense-state joint neural network reasoning system according to claim 7 is characterized in that: The dense joint neural network reasoning core module includes a convolution layer SecConv, an activation function layer SecAct, a maximum pooling layer SecMP and a fully connected layer SecFC.

9. The efficient dense-state joint neural network reasoning system according to claim 7 is characterized in that: The computing layer includes two computing nodes and , the data holder C divides the private data X into X0 and X1 and sends them to and , model owner The privacy model W is divided into W0 and W1 and sent to and , and The private data is sequentially processed through the convolution layer SecConv, the activation function layer SecAct, the maximum pooling layer SecMP, and the fully connected layer SecFC to obtain the secret shares Z0 and Z1 of the inference results, respectively.

10. The efficient dense-state joint neural network reasoning system according to claim 9 is characterized in that: The secret shares Z0 and Z1 are transmitted to the data holder C through the output layer. C reconstructs the secret share Z=Z0+Z1 and obtains the inference result Z.

Images