A neural network model training method, device and secure computing platform

Abstract

Embodiments of the present invention provide a neural network model training method, device, and apparatus for neural network model training. The method includes: obtaining training data from participating parties, the training data being ciphertext; initializing a neural network model by executing a training script, wherein the network layers of the neural network model include a plaintext layer and a ciphertext layer; iteratively training the neural network model based on the training data held. In each round of iterative training, the training data of the current round is securely calculated layer by layer through each network layer of the neural network model. When the iterative termination condition is not met, the parameters of each network layer are adjusted layer by layer through backpropagation based on the model output results. When the iterative termination condition is met, a trained neural network model is obtained. Calculations on the plaintext layer are performed based on plaintext, and calculations on the ciphertext layer are performed based on ciphertext. While ensuring data privacy and security, the embodiments of the present invention reduce the communication overhead and resource consumption caused by ciphertext calculations.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a neural network model training method, device and secure computing platform. Background Art

[0002] With the widespread application of machine learning, methods based on machine learning or deep learning have been applied to multiple technical fields and achieved great success. When training various machine learning models, the data required for machine learning often involves multiple fields.

[0003] However, due to concerns about information security and user privacy, different institutions or individuals hold different types of data. These data are often isolated from each other, preventing efficient and convenient data circulation and forming information silos. Therefore, how to combine multiple data sources to jointly train machine learning models while balancing data security and computational efficiency is a pressing issue in existing technologies. Summary of the Invention

[0004] Embodiments of the present invention provide a neural network model training method, device, and secure computing platform, which can greatly improve the model training speed and reduce the communication overhead and resource consumption caused by ciphertext calculations while ensuring data privacy and security.

[0005] To address the above issues, an embodiment of the present invention discloses a neural network model training method, which is applied to a computing node in a secure computing platform. The secure computing platform includes at least one computing unit, each of which includes at least one computing node. The method includes:

[0006] Obtaining training data from a participant, wherein the training data is ciphertext;

[0007] Initialize the neural network model by executing the training script, wherein the network layer of the neural network model includes a plaintext layer and a ciphertext layer;

[0008] The neural network model is iteratively trained based on the held training data. In each round of iterative training, the training data of this round is securely calculated layer by layer through each network layer of the neural network model. When the iteration termination condition is not met, the parameters of each network layer are adjusted layer by layer through back propagation according to the model output result. When the iteration termination condition is reached, a trained neural network model is obtained; wherein, the calculation of the plaintext layer is based on the plaintext, and the calculation of the ciphertext layer is based on the ciphertext.

[0009] On the other hand, an embodiment of the present invention discloses a prediction method based on a neural network model, which is applied to a computing node in a secure computing platform. The secure computing platform includes at least one computing unit, each computing unit includes at least one computing node, and the method includes:

[0010] Obtaining prediction data from a participant, wherein the prediction data is ciphertext;

[0011] Based on the held prediction data, the prediction data is securely calculated layer by layer through each network layer of the neural network model to obtain a model prediction result, wherein, in the process of securely calculating the prediction data layer by layer through each network layer of the neural network model, the calculation of the plaintext layer is performed based on the plaintext, and the calculation of the ciphertext layer is performed based on the ciphertext.

[0012] In another aspect, an embodiment of the present invention discloses a neural network model training device, which is applied to a computing node in a secure computing platform, wherein the secure computing platform includes at least one computing unit, each computing unit includes at least one computing node, and the device includes:

[0013] A training data acquisition module, used to acquire training data from participants, wherein the training data is ciphertext;

[0014] A model initialization module is used to initialize the neural network model by executing a training script, wherein the network layer of the neural network model includes a plaintext layer and a ciphertext layer;

[0015] An iterative training module is used to iteratively train the neural network model based on the training data held. In each round of iterative training, the training data of this round is securely calculated layer by layer through each network layer of the neural network model. When the iteration termination condition is not met, the parameters of each network layer are adjusted layer by layer through back propagation according to the model output result. When the iteration termination condition is reached, a trained neural network model is obtained; wherein, the calculation of the plaintext layer is based on the plaintext, and the calculation of the ciphertext layer is based on the ciphertext.

[0016] In another aspect, an embodiment of the present invention discloses a prediction device based on a neural network model, which is applied to a computing node in a secure computing platform. The secure computing platform includes at least one computing unit, each computing unit includes at least one computing node, and the device includes:

[0017] A prediction data acquisition module, used to obtain prediction data from participants, wherein the prediction data is ciphertext;

[0018] The model prediction module is used to securely calculate the prediction data layer by layer through each network layer of the neural network model based on the prediction data held, so as to obtain a model prediction result. In the process of securely calculating the prediction data layer by layer through each network layer of the neural network model, the calculation of the plaintext layer is performed based on the plaintext, and the calculation of the ciphertext layer is performed based on the ciphertext.

[0019] In another aspect, an embodiment of the present invention discloses a secure computing platform, comprising at least one computing unit, each computing unit comprising at least one computing node; wherein,

[0020] The computing node is used to obtain training data from the participants, initialize the neural network model by executing the training script, and iteratively train the neural network model based on the held training data. In each round of iterative training, the training data of this round is securely calculated layer by layer through each network layer of the neural network model. When the iteration termination condition is not met, the parameters of each network layer are adjusted layer by layer through back propagation according to the model output result. When the iteration termination condition is reached, a trained neural network model is obtained; wherein, the training data is ciphertext, and the network layer of the neural network model includes a plaintext layer and a ciphertext layer. The calculation of the plaintext layer is based on the plaintext, and the calculation of the ciphertext layer is based on the ciphertext.

[0021] On the other hand, an embodiment of the present invention discloses a device for training a neural network model, comprising a memory and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by one or more processors, and the one or more programs include instructions for performing one or more of the neural network model training methods described above.

[0022] On the other hand, an embodiment of the present invention discloses a machine-readable medium having instructions stored thereon, which, when executed by one or more processors of a device, causes the device to execute one or more of the neural network model training methods described above.

[0023] The embodiments of the present invention include the following advantages:

[0024] An embodiment of the present invention proposes a neural network model training method based on a secure computing platform, optimizes the structure of the neural network model, and proposes a mixed plaintext and ciphertext neural network model, that is, the neural network model is configured to include a plaintext layer and a ciphertext layer. When training the neural network model, the computing nodes participating in the training task obtain training data from the participating parties, use the training data for forward feature propagation and reverse gradient propagation, and adjust the parameters of each network layer layer by layer through reverse gradient propagation. During the forward feature propagation and reverse gradient propagation processes, the training data is processed by the plaintext layer and the ciphertext layer. The plaintext layer is used to ensure training speed, and the ciphertext layer is used to ensure data privacy. When the iterative termination condition is reached, the trained neural network model can be obtained. Through the combined action of the plaintext layer and the ciphertext layer, the embodiment of the present invention greatly improves the model training speed while ensuring data privacy and security, and reduces the communication overhead and resource consumption caused by ciphertext calculations. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments of the present invention. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0026] Figure 1 is a flowchart of the steps of an embodiment of a neural network model training method of the present invention;

[0027] Figure 2 It is a structural schematic diagram of a neural network model of the present invention;

[0028] Figure 3 It is a flowchart of the steps of a prediction method based on a neural network model of the present invention;

[0029] Figure 4 It is a structural block diagram of an embodiment of a neural network model training device of the present invention;

[0030] Figure 5 It is a structural block diagram of an embodiment of a prediction device based on a neural network model of the present invention;

[0031] Figure 6 is a block diagram of an apparatus 800 for training a neural network model according to the present invention;

[0032] Figure 7 It is a schematic diagram of the structure of the server in some embodiments of the present invention. DETAILED DESCRIPTION

[0033] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0034] The terms "first", "second", etc. in the specification and claims of the present invention are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present invention can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, the term "and / or" in the specification and claims is used to describe the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the objects before and after the association are in an "or" relationship. In the embodiments of the present invention, the term "multiple" refers to two or more, and other quantifiers are similar.

[0035] Reference Figure 1 , shows a flowchart of the steps of an embodiment of a neural network model training method of the present invention. The method can be applied to a computing node in a secure computing platform. The secure computing platform includes at least one computing unit, each computing unit includes at least one computing node, and the method may include the following steps:

[0036] Step 101: Obtain training data from a participant, where the training data is ciphertext;

[0037] Step 102: Initialize a neural network model by executing a training script, wherein the network layer of the neural network model includes a plaintext layer and a ciphertext layer;

[0038] Step 103: Iteratively train the neural network model based on the training data held. In each round of iterative training, the training data of this round is securely calculated layer by layer through each network layer of the neural network model. When the iteration termination condition is not met, the parameters of each network layer are adjusted layer by layer through back propagation according to the model output result. When the iteration termination condition is reached, a trained neural network model is obtained; wherein, the calculation of the plaintext layer is based on the plaintext, and the calculation of the ciphertext layer is based on the ciphertext.

[0039] The secure computing platform can be a computing platform that protects data privacy and security. The embodiment of the present invention does not limit the secure computing protocol adopted by the secure computing platform. For example, the secure computing platform can be based on the MPC (Secure Muti-Party Computation) protocol. In a secure computing platform based on the MPC protocol, multiple participants can use multi-party secure computing technology to perform collaborative computing to obtain computing results without leaking their own data. The data, intermediate results, and final results of the participating calculations can be ciphertext. Of course, the secure computing platform can also be based on secure computing protocols implemented by technologies such as secret sharing, semi-homomorphism, and oblivious transfer.

[0040] In each round of iterative training, the specific manner in which the training data is securely calculated layer by layer through each network layer of the neural network model is determined according to the secure computing protocol adopted by the secure computing platform.

[0041] The secure computing platform may include at least one computing unit. A computing unit refers to a basic unit that can collaboratively perform ciphertext computations. Each computing unit may include at least one computing node. The computing node may be any device with computing capabilities, such as a server. The embodiments of the present invention do not limit the number of computing nodes that a computing unit may contain. Preferably, a computing unit may contain four computing nodes.

[0042] Traditional neural network models typically require all training data to be transmitted in plaintext, which poses a risk of data privacy leakage. While training neural network models based on pure ciphertext can ensure data privacy, the large amount of ciphertext calculations introduces additional computational overhead and transmission delays, making the time and resource consumption of neural network training difficult to meet the needs of practical applications.

[0043] An embodiment of the present invention proposes a neural network model training method based on a secure computing platform, optimizes the structure of the neural network model, and proposes a mixed plaintext and ciphertext neural network model, that is, the neural network model is set to include a plaintext layer and a ciphertext layer. Through the joint action of the plaintext layer and the ciphertext layer, the training process of the neural network model can take into account both data security and computing efficiency.

[0044] It should be noted that the embodiment of the present invention does not limit the number of participants involved in training the neural network model. The number of participants can be greater than or equal to 1. Furthermore, when the number of participants is greater than 1, through the embodiment of the present invention, each participant can jointly train the neural network model under the premise of protecting the privacy and security of their respective data. Among them, the participant can be the data owner and can provide training data for training the neural network model. The training data held by each participant can be a private data, such as environmental monitoring data, user credit data, user account data, personal health data, etc. The embodiment of the present invention is mainly described by taking the scenario in which multiple participants jointly train a neural network model based on the data they hold as an example.

[0045] First, the computing nodes participating in the training task obtain training data from each participant, where the training data is ciphertext.

[0046] A training task refers to a computing task used to train a neural network model. A computing node participating in a training task refers to at least one computing node in at least one computing unit participating in the training task specified in a training sample.

[0047] Before executing a training task, you can deploy the computing nodes participating in the training task on the secure computing platform. Each computing node participating in the training task should have sufficient storage space (including memory space and hard disk space) and CPU (central processing unit) resources. Different computing nodes should be able to communicate over the network and have sufficient bandwidth to ensure normal data transmission.

[0048] The training data may include feature data and label data, typically represented in the form of a two-dimensional matrix. The training data may come from at least one participant, and each participant may encrypt the original data held by the participant and upload it to the computing node. The participant may be the data owner. In specific implementations, different participants may hold different feature data. Furthermore, the feature data and label data may be held by different participants.

[0049] In an embodiment of the present invention, the secure computing platform may further include a scheduling node for receiving training scripts sent by participating parties. The method may further include: receiving the training script sent by the scheduling node, wherein the training script includes setting information and training code of the neural network model.

[0050] The training script can be determined through negotiation among all participating parties. It includes neural network model configuration information, such as the neural network model's structure, preprocessing procedures, and loss function. It also includes training code, specifically the specific training process. Each computing node participating in the training task completes the neural network model training process by executing the training script.

[0051] In the embodiment of the present invention, the scheduling node receives the training script sent by the participant, and sends the training script to each computing node participating in the training task.

[0052] Furthermore, the training script may also include operation codes such as filtering duplicate features and filtering duplicate samples to eliminate defects in the training data.

[0053] In this example, assume a training task involves n participants, each of whom holds their own training data. Each participant encrypts their training data and uploads it to the compute node designated for the task. The training script is agreed upon by all participants and stored locally on Participant 1. Participant 1 can be any of the n participants.

[0054] Participant 1 uploads the training script to the scheduling node, which distributes the training script to the computing nodes participating in the training task. Each computing node executes the training script, generates a neural network model, and performs executable parts such as model initialization, training data preprocessing, and process construction according to the training script, waiting for the transmission of training data.

[0055] It should be noted that each participant may upload training data to the computing node before participant 1 uploads the training script to the scheduling node, or may upload training data to the computing node after participant 1 uploads the training script to the scheduling node. This embodiment of the present invention does not impose any restrictions on this.

[0056] After the scheduling node distributes the training script to the computing nodes participating in the training task, each computing node participating in the training task initializes the neural network model by executing the training script and waits for the transmission of training data. After each computing node obtains the training data, each computing node iteratively trains the neural network model based on the training data it holds. In each round of iterative training, the training data of this round is securely calculated layer by layer through each network layer of the neural network model. When the iteration termination condition is not met, the parameters of each network layer are adjusted layer by layer through back propagation according to the model output result. When the iteration termination condition is reached, the trained neural network model is obtained; when the iteration termination condition is reached, the trained neural network model is obtained. The network layer of the neural network model includes a plaintext layer and a ciphertext layer. The calculation of the plaintext layer is based on the plaintext, and the calculation of the ciphertext layer is based on the ciphertext.

[0057] In an embodiment of the present invention, each computing node participating in the training task uses the training data for forward feature propagation and reverse gradient propagation according to the training process in the training script, and adjusts the parameters of each network layer layer by layer through reverse gradient propagation. During the process of forward feature propagation and reverse gradient propagation, the training data is processed by the plaintext layer and the ciphertext layer. The plaintext layer can be used to ensure the training speed, and the ciphertext layer can be used to ensure data privacy. When the iterative termination condition is reached, a trained neural network model can be obtained. Through the combined effect of the plaintext layer and the ciphertext layer, the embodiment of the present invention greatly improves the model training speed and reduces the communication overhead and resource consumption caused by ciphertext calculations while ensuring data privacy and security.

[0058] In an optional embodiment of the present invention, the network layers of the neural network model may include an input layer, at least one hidden layer, and an output layer in sequence, and the plaintext layer may include at least one hidden layer and / or output layer.

[0059] The hidden layer is the network layer between the input and output layers. The hidden layer abstracts the input feature data into another dimension, presenting more abstract features that can be better partitioned linearly. Multiple hidden layers allow for multi-level abstraction of the input feature data.

[0060] Since the data in the hidden layer is a more abstract feature of the input feature data and does not expose the original data, the embodiment of the present invention can set at least one hidden layer as a plaintext layer to improve the training speed of the neural network model.

[0061] Reference Figure 2 , shows a schematic diagram of the structure of a neural network model of the present invention. Figure 2 As shown in the figure, the structure of the neural network model is: Linear-ReLU-Linear-ReLU-…-ReLU-Linear. The ellipsis indicates that the part can be added arbitrarily. Figure 2 The neural network model shown is a supervised fully connected neural network. Linear represents the linear layer, and ReLU represents the activation layer (non-linear layer). The hidden layer includes the Linear layer and the ReLU layer. The neural network model also includes a loss function (Loss) layer. Figure 2 The Loss layer is used to calculate the error between the forward prediction result and the label data. Based on this error, the parameters of each network layer can be adjusted layer by layer through reverse gradient propagation.

[0062] It is understandable that Figure 2The structure of the neural network model shown is only an example of an application of the present invention. The embodiment of the present invention does not limit the specific structure of the neural network model. Figure 2 The setting of the plaintext layer and the ciphertext layer is only used as an application example of the present invention.

[0063] In an optional embodiment, the structure of the neural network model can be a front ciphertext layer, an intermediate plaintext layer, and a rear ciphertext layer. This structure indicates that the input layer and the output layer are ciphertext layers, and the intermediate hidden layer (one or more hidden layers) is a plaintext layer. The front ciphertext layer (the input layer is a ciphertext layer) can be used to protect the privacy and security of the input feature data, and the rear ciphertext layer (the output layer is a ciphertext layer) can be used to protect the privacy and security of the label data. In the intermediate hidden layer, the forward feature data and the reverse gradient data are no longer the original input data. Therefore, the intermediate hidden layer, as a plaintext layer, will not expose the privacy information of the data.

[0064] Furthermore, the training method of the neural network model in the embodiment of the present invention can include supervised training or unsupervised training. The embodiment of the present invention mainly uses supervised training as an example for explanation. The difference between unsupervised training and unsupervised training is only that there is no input of labeled data. The training process can refer to the process of supervised training.

[0065] For supervised training neural network models, to ensure data privacy and security, the input layer and output layer can be set as ciphertext layers, and at least one hidden layer can be set as a plaintext layer. In one example, for supervised training neural network models, the input layer and a first number of hidden layers connected to the input layer can be set as ciphertext layers, the output layer and a second number of hidden layers connected to the input label data can be set as ciphertext layers, and the remaining hidden layers in the middle can be set as plaintext layers. The first number of ciphertext layers is used to protect the privacy and security of the input feature data, and the second number of ciphertext layers is used to protect the privacy and security of the input label data. The first number and the second number can be set according to actual needs.

[0066] Unlike supervised training, unsupervised neural network models do not require labeled data input. Therefore, setting the output layer to a plaintext layer does not expose the labeled data. For unsupervised neural network models, simply setting the input layer to an encrypted layer ensures data privacy. In one example, for an unsupervised neural network model, the input layer and the first number of hidden layers connected to the input layer can be set as encrypted layers, while the hidden layers and output layer can be set as plaintext layers.

[0067] In practice, the plaintext and ciphertext layers in the neural network model can be adjusted based on the actual application scenario. For scenarios with high security requirements, more ciphertext layers can be stacked to ensure data privacy. For scenarios with relatively low security requirements but high real-time performance, more plaintext layers can be added to increase model training speed.

[0068] In specific implementations, the plaintext layer and the ciphertext layer can be set according to the requirements of the actual application scenario to meet the balance between data security and computing efficiency in the actual application scenario.

[0069] In an optional embodiment of the present invention, if the network layer preceding the plaintext layer is a ciphertext layer, a first conversion layer is further included between the plaintext layer and the ciphertext layer, and the method further includes:

[0070] During the forward feature data propagation process of each round of iterative training, the forward feature data output by the previous network layer of the plaintext layer is converted into plaintext through the first conversion layer and then passed into the plaintext layer. In the reverse gradient propagation process, the reverse gradient data of the plaintext layer is converted into ciphertext through the first conversion layer and then passed into the previous network layer of the plaintext layer.

[0071] In an embodiment of the present invention, the first conversion layer is referred to as a ciphertext-to-plaintext layer (C2P layer). During the forward feature data propagation process, the previous network layer of the first conversion layer is a ciphertext layer, and the next network layer of the first conversion layer is a plaintext layer. During the forward feature data propagation process, the forward feature data passing through the first conversion layer is converted from ciphertext to plaintext and input into the next network layer (plaintext layer). During the reverse gradient propagation process, the previous network layer of the first conversion layer is a plaintext layer, and the next network layer of the first conversion layer is a ciphertext layer. During the reverse gradient propagation process, the reverse gradient data passing through the first conversion layer is converted from plaintext to ciphertext and input into the next network layer (ciphertext layer). That is, the first conversion layer is used to decrypt the forward feature data and encrypt the reverse gradient data.

[0072] In an optional embodiment of the present invention, if the network layer preceding the ciphertext layer is a plaintext layer, a second conversion layer is further included between the ciphertext layer and the plaintext layer, and the method further includes:

[0073] During the forward feature data propagation process of each round of iterative training, the forward feature data output by the previous network layer of the ciphertext layer is converted into ciphertext through the second conversion layer and then passed into the ciphertext layer. In the reverse gradient propagation process, the reverse gradient data of the ciphertext layer is converted into plaintext through the second conversion layer and then passed into the previous network layer of the ciphertext layer.

[0074] In an embodiment of the present invention, the second conversion layer is referred to as a plaintext-to-ciphertext layer (P2C layer). During the forward feature data propagation process, the previous network layer of the second conversion layer is a plaintext layer, and the next network layer of the second conversion layer is a ciphertext layer. During the forward feature data propagation process, the forward feature data passing through the second conversion layer is converted from plaintext to ciphertext and input into the next network layer (ciphertext layer). During the reverse gradient propagation process, the previous network layer of the second conversion layer is a ciphertext layer, and the next network layer of the first conversion layer is a plaintext layer. During the reverse gradient propagation process, the reverse gradient data passing through the second conversion layer is converted from ciphertext to plaintext and input into the next network layer (plaintext layer). That is, the second conversion layer is used to encrypt the forward feature data and decrypt the reverse gradient data.

[0075] It should be noted that the specific manner in which the first conversion layer and the second conversion layer encrypt and decrypt the forward feature data and / or reverse gradient data can be determined according to the secure computing protocol adopted by the secure computing platform.

[0076] Through the first and second conversion layers of the embodiments of the present invention, any existing neural network model can be conveniently converted into a mixed plaintext and ciphertext neural network model. This allows for flexible configuration of the plaintext and ciphertext layers of the neural network model, increasing the adaptability of the mixed plaintext and ciphertext neural network model. Furthermore, the embodiments of the present invention provide flexible configuration solutions for the plaintext and ciphertext layers, meeting the needs of various application scenarios while ensuring data privacy and security.

[0077] In an optional embodiment of the present invention, the calculation of the plaintext layer is performed based on the plaintext, and the calculation of the ciphertext layer is performed based on the ciphertext, which may include:

[0078] Execute the calculation of the plaintext layer based on the plaintext respectively by the computing nodes in the computing unit specified in the training script;

[0079] The calculation of the ciphertext layer is performed based on the ciphertext by the computing nodes in the computing unit specified in the training script.

[0080] For calculations in the plaintext layer, both the forward feature data and the reverse gradient data are displayed in plaintext. For calculations in the ciphertext layer, both the forward feature data and the reverse gradient data are displayed in ciphertext.

[0081] In specific applications, the computing unit involved in each step of the training task and the computing nodes in the computing unit can be specified in the training script.

[0082] For example, for a certain plaintext layer, assuming that the computing unit specified in the training script to participate in the calculation of the plaintext layer is computing unit A, and computing unit A includes 4 computing nodes, then one of the computing nodes in computing unit A can be set to perform the calculation of the plaintext layer, and the calculation result of the computing node can be obtained; or, 4 computing nodes can be set to perform the calculation of the plaintext layer at the same time, and the calculation result of one of the specified computing nodes can be obtained (because the calculation results of the 4 computing nodes are theoretically the same).

[0083] For example, for a certain ciphertext layer, assuming that the computing unit specified in the training script to participate in the ciphertext layer calculation is computing unit B, and computing unit B includes 4 computing nodes, the calculation result of the ciphertext layer calculation can be obtained through the collaborative calculation of the 4 computing nodes in computing unit B.

[0084] It is understandable that, in specific applications, the calculation method of the plaintext layer and the calculation method of the ciphertext layer can be set according to the actual application scenario and the secure computing protocol adopted by the secure computing platform.

[0085] Furthermore, the parameters of the ciphertext layer can be stored in plaintext or ciphertext. For example, in scenarios with high security requirements, the parameters of the ciphertext layer can be stored in ciphertext. In scenarios with low security requirements but high real-time requirements, the parameters of the ciphertext layer can be stored in plaintext, improving calculation speed at the expense of certain security. Preferably, embodiments of the present invention store the parameters of the ciphertext layer in ciphertext.

[0086] When the iteration termination condition is met, the parameters of the ciphertext layer are decrypted and the parameters of each network layer are saved to obtain the trained neural network model. The iteration termination condition may include: the error between the forward prediction result output by the neural network model and the label data is less than a preset threshold, or the number of iterative training reaches a preset number.

[0087] When the number of participants is greater than 1, through the embodiment of the present invention, each participant can jointly train the neural network model while protecting the privacy and security of their own data.

[0088] Exemplarily, when the number of participants is greater than 1, the application scenarios of an embodiment of the present invention may include: multiple banks each hold a portion of user-related data, and the multiple banks hope to jointly utilize multi-party data while exchanging encrypted original data to train a deep neural network model, which is used to predict and evaluate the risk of users.

[0089] When the number of participants is 1, embodiments of the present invention can support model prediction for ciphertext input samples. For example, a trained neural network model can be used to predict computing power, which can be applied to computing power rental scenarios. For example, the computing power renter can provide the structure and parameters of the neural network model, information about the sample to be predicted, and a prediction script. The computing power provider cannot obtain the private information of the predicted sample while running the prediction script. The prediction process is equivalent to the first half of model training: only forward feature propagation is involved. The ciphertext prediction result ultimately output by the neural network model can be transmitted back to the computing power renter.

[0090] In summary, the embodiment of the present invention proposes a neural network model training method based on a secure computing platform, optimizes the structure of the neural network model, and proposes a mixed plaintext and ciphertext neural network model, that is, the neural network model is set to include a plaintext layer and a ciphertext layer. When training the neural network model, the computing nodes participating in the training task obtain training data from the participating parties, use the training data for forward feature propagation and reverse gradient propagation, and adjust the parameters of each network layer layer by layer through reverse gradient propagation. During the forward feature propagation and reverse gradient propagation process, the training data is processed by the plaintext layer and the ciphertext layer. The plaintext layer is used to ensure the training speed, and the ciphertext layer is used to ensure data privacy. When the iterative termination condition is reached, the trained neural network model can be obtained. Through the combined effect of the plaintext layer and the ciphertext layer, the embodiment of the present invention greatly improves the model training speed while ensuring data privacy and security, and reduces the communication overhead and resource consumption caused by ciphertext calculation.

[0091] Reference Figure 3 , shows a flowchart of the steps of a prediction method based on a neural network model of the present invention. The method can be applied to a computing node in a secure computing platform. The secure computing platform includes at least one computing unit, each computing unit includes at least one computing node. The method may include:

[0092] Step 301: Obtain prediction data from a participant, where the prediction data is ciphertext;

[0093] Step 302: Based on the held prediction data, the prediction data is securely calculated layer by layer through each network layer of the neural network model to obtain a model prediction result, wherein, in the process of securely calculating the prediction data layer by layer through each network layer of the neural network model, the calculation of the plaintext layer is performed based on the plaintext, and the calculation of the ciphertext layer is performed based on the ciphertext.

[0094] After the neural network model is trained using the neural network model training method described in the above embodiment, the trained neural network model can be used for prediction. The neural network model is a plaintext and ciphertext mixed neural network model, including a plaintext layer and a ciphertext layer.

[0095] The process of using the trained neural network model to make predictions is the process of forward feature propagation when training the neural network model, which will not be described in detail here.

[0096] Participants in the prediction task can be data owners. Each participant holds their own prediction data. Each participant encrypts the original prediction data and uploads it to the computing node. The prediction data held by each participant can be private data, such as environmental monitoring data, user credit data, user account data, personal health data, etc.

[0097] When the number of participants is greater than 1, through the embodiment of the present invention, each participant can use the neural network model to perform joint prediction while protecting the privacy and security of their own data.

[0098] Exemplarily, when the number of participants is greater than 1, the application scenarios of an embodiment of the present invention may include: multiple banks each hold a portion of user-related data, and the multiple banks hope to jointly utilize multi-party data while exchanging encrypted original data to train a deep neural network model, which is used to predict and evaluate the risk of users.

[0099] When the number of participants is 1, embodiments of the present invention can support model prediction for ciphertext input samples. For example, a trained neural network model can be used to predict computing power, which can be applied to computing power rental scenarios. For example, the computing power renter can provide the structure and parameters of the neural network model, information about the sample to be predicted, and a prediction script. The computing power provider cannot obtain the private information of the predicted sample while running the prediction script. The prediction process is equivalent to the first half of model training: only forward feature propagation is involved. The ciphertext prediction result ultimately output by the neural network model can be transmitted back to the computing power renter.

[0100] The embodiment of the present invention proposes a mixed plaintext and ciphertext neural network model, that is, the neural network model is set to include a plaintext layer and a ciphertext layer. After the training of the neural network model is completed, the neural network model can be used to jointly predict the prediction data held by multiple participants to obtain a prediction result. In the process of prediction by the neural network model, the computing nodes participating in the prediction task obtain the prediction data from the participants and use the prediction data for forward feature propagation. In the forward feature propagation process, the prediction data is processed by the plaintext layer and the ciphertext layer. The plaintext layer is used to ensure the training speed, and the ciphertext layer is used to ensure data privacy. The embodiment of the present invention, through the joint action of the plaintext layer and the ciphertext layer, greatly improves the calculation speed of the model while ensuring data privacy and security, and reduces the communication overhead and resource consumption caused by ciphertext calculation.

[0101] It should be noted that for the sake of simplicity, the method embodiments are described as a series of actions. However, those skilled in the art should be aware that the embodiments of the present invention are not limited by the order of the actions described, because according to the embodiments of the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in this specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.

[0102] Reference Figure 4 , shows a structural block diagram of an embodiment of a neural network model training device of the present invention. The device can be applied to a computing node in a secure computing platform. The secure computing platform includes at least one computing unit, each computing unit includes at least one computing node, and the device may include:

[0103] The training data acquisition module 401 is used to obtain training data from the participants, wherein the training data is ciphertext;

[0104] A model initialization module 402 is configured to initialize a neural network model by executing a training script, wherein the network layer of the neural network model includes a plaintext layer and a ciphertext layer;

[0105] The iterative training module 403 is used to iteratively train the neural network model based on the held training data. In each round of iterative training, the training data of this round is securely calculated layer by layer through each network layer of the neural network model. When the iteration termination condition is not met, the parameters of each network layer are adjusted layer by layer through back propagation according to the model output result. When the iteration termination condition is reached, a trained neural network model is obtained; wherein, the calculation of the plaintext layer is based on the plaintext, and the calculation of the ciphertext layer is based on the ciphertext.

[0106] Optionally, the network layer includes an input layer, at least one hidden layer, and an output layer in sequence, and the plaintext layer includes at least one hidden layer and / or an output layer.

[0107] Optionally, if the network layer before the plaintext layer is a ciphertext layer, a first conversion layer is further included between the plaintext layer and the ciphertext layer, and the apparatus further includes:

[0108] The first conversion module is used to convert the forward feature data output by the previous network layer of the plaintext layer into plaintext through the first conversion layer during the forward feature data propagation process of each round of iterative training, and then pass it into the plaintext layer; and during the reverse gradient propagation process, convert the reverse gradient data of the plaintext layer into ciphertext through the first conversion layer, and then pass it into the previous network layer of the plaintext layer.

[0109] Optionally, if the network layer preceding the ciphertext layer is a plaintext layer, a second conversion layer is further included between the ciphertext layer and the plaintext layer, and the apparatus further includes:

[0110] The second conversion module is used to convert the forward feature data output by the previous network layer of the ciphertext layer into ciphertext through the second conversion layer during the forward feature data propagation process of each round of iterative training, and then pass it into the ciphertext layer; and in the reverse gradient propagation process, convert the reverse gradient data of the ciphertext layer into plaintext through the second conversion layer, and then pass it into the previous network layer of the ciphertext layer.

[0111] Optionally, the secure computing platform further includes a scheduling node for receiving a training script sent by a participant, and the apparatus further includes:

[0112] The script receiving module is used to receive the training script sent by the scheduling node, where the training script includes setting information and training code of the neural network model.

[0113] Optionally, the iterative training module includes:

[0114] A plaintext calculation submodule, configured to perform calculations of the plaintext layer based on plaintexts respectively through calculation nodes in the specified calculation units in the training script;

[0115] The ciphertext calculation submodule is used to perform the calculation of the ciphertext layer based on the ciphertext by using the calculation nodes in the specified calculation unit in the training script.

[0116] Optionally, the training data comes from at least one participant, and the training data includes sample data and label data, and the sample data and label data are held by different participants.

[0117] Optionally, the secure computing includes any one of the following: secret sharing, obfuscated circuits, and homomorphic encryption.

[0118] An embodiment of the present invention proposes a neural network model training device based on a secure computing platform, optimizes the structure of the neural network model, and proposes a mixed plaintext and ciphertext neural network model, that is, the neural network model is set to include a plaintext layer and a ciphertext layer. When training the neural network model, the computing nodes participating in the training task obtain training data from the participating parties, use the training data to perform forward feature propagation and reverse gradient propagation, and adjust the parameters of each network layer layer by layer through reverse gradient propagation. During the forward feature propagation and reverse gradient propagation process, the training data is processed by the plaintext layer and the ciphertext layer. The plaintext layer is used to ensure training speed, and the ciphertext layer is used to ensure data privacy. When the iterative termination condition is reached, the trained neural network model can be obtained. Through the combined action of the plaintext layer and the ciphertext layer, the embodiment of the present invention greatly improves the model training speed while ensuring data privacy and security, and reduces the communication overhead and resource consumption caused by ciphertext calculation.

[0119] Reference Figure 5 , shows a structural block diagram of an embodiment of a prediction device based on a neural network model of the present invention. The device can be applied to a computing node in a secure computing platform. The secure computing platform includes at least one computing unit, each computing unit includes at least one computing node, and the device may include:

[0120] The prediction data acquisition module 501 is used to obtain prediction data from the participants, wherein the prediction data is encrypted;

[0121] The model prediction module 502 is used to securely calculate the prediction data layer by layer through each network layer of the neural network model based on the held prediction data to obtain a model prediction result. In the process of securely calculating the prediction data layer by layer through each network layer of the neural network model, the calculation of the plaintext layer is based on the plaintext, and the calculation of the ciphertext layer is based on the ciphertext.

[0122] Optionally, the network layer includes an input layer, at least one hidden layer, and an output layer in sequence, and the plaintext layer includes at least one hidden layer and / or an output layer.

[0123] The embodiment of the present invention proposes a mixed plaintext and ciphertext neural network model, that is, the neural network model is set to include a plaintext layer and a ciphertext layer. After the training of the neural network model is completed, the neural network model can be used to jointly predict the prediction data held by multiple participants to obtain a prediction result. In the process of prediction by the neural network model, the computing nodes participating in the prediction task obtain the prediction data from the participants and use the prediction data for forward feature propagation. In the forward feature propagation process, the prediction data is processed by the plaintext layer and the ciphertext layer. The plaintext layer is used to ensure the training speed, and the ciphertext layer is used to ensure data privacy. The embodiment of the present invention, through the joint action of the plaintext layer and the ciphertext layer, greatly improves the calculation speed of the model while ensuring data privacy and security, and reduces the communication overhead and resource consumption caused by ciphertext calculation.

[0124] An embodiment of the present invention provides a secure computing platform, which includes at least one computing unit, each of which includes at least one computing node; wherein,

[0125] The computing node is used to obtain training data from the participants, initialize the neural network model by executing the training script, and iteratively train the neural network model based on the held training data. In each round of iterative training, the training data of this round is securely calculated layer by layer through each network layer of the neural network model. When the iteration termination condition is not met, the parameters of each network layer are adjusted layer by layer through back propagation according to the model output result. When the iteration termination condition is reached, a trained neural network model is obtained; wherein, the training data is ciphertext, and the network layer of the neural network model includes a plaintext layer and a ciphertext layer. The calculation of the plaintext layer is based on the plaintext, and the calculation of the ciphertext layer is based on the ciphertext.

[0126] Optionally, the secure computing platform further includes a scheduling node for receiving training scripts sent by participating parties; the computing node is further used to receive the training scripts sent by the scheduling node, wherein the training scripts include setting information and training code of the neural network model.

[0127] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0128] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0129] Regarding the apparatus in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0130] An embodiment of the present invention provides a device for training a neural network model, comprising a memory and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by one or more processors, wherein the one or more programs include instructions for performing the following operations: obtaining training data from a participant, wherein the training data is ciphertext; initializing a neural network model by executing a training script, wherein the network layer of the neural network model includes a plaintext layer and a ciphertext layer; iteratively training the neural network model based on the held training data, wherein in each round of iterative training, the training data of this round is securely calculated layer by layer through each network layer of the neural network model; when the iteration termination condition is not met, the parameters of each network layer are adjusted layer by layer through back propagation according to the model output result, and when the iteration termination condition is reached, a trained neural network model is obtained; wherein the calculation of the plaintext layer is performed based on the plaintext, and the calculation of the ciphertext layer is performed based on the ciphertext.

[0131] Figure 6 FIG8 is a block diagram of an apparatus 800 for training a neural network model according to an exemplary embodiment. For example, the apparatus 800 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0132] Reference Figure 6 , the device 800 may include one or more of the following components: a processing component 802 , a memory 804 , a power component 806 , a multimedia component 808 , an audio component 810 , an input / output (I / O) interface 812 , a sensor component 814 , and a communication component 816 .

[0133] The processing component 802 generally controls the overall operation of the device 800, such as operations associated with display, phone calls, data communications, camera operation, and recording operations. The processing component 802 may include one or more processors 820 to execute instructions to perform all or part of the steps of the above-described method. In addition, the processing component 802 may include one or more modules to facilitate interaction between the processing component 802 and other components. For example, the processing component 802 may include a multimedia module to facilitate interaction between the multimedia component 808 and the processing component 802.

[0134] The memory 804 is configured to store various types of data to support operations on the device 800. Examples of such data include instructions for any application or method operating on the device 800, contact data, phone book data, messages, pictures, videos, etc. The memory 804 can be implemented by any type of volatile or non-volatile storage device, or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk, or optical disk.

[0135] The power supply component 806 provides power to the various components of the device 800. The power supply component 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the device 800.

[0136] The multimedia component 808 includes a screen that provides an output interface between the device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen can be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor can not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 808 includes a front camera and / or a rear camera. When the device 800 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera can receive external multimedia data. Each front camera and rear camera can be a fixed optical lens system or have a focal length and optical zoom capability.

[0137] The audio component 810 is configured to output and / or input audio signals. For example, the audio component 810 includes a microphone (MIC), which is configured to receive external audio signals when the device 800 is in an operating mode, such as a call mode, a recording mode, and a voice information processing mode. The received audio signal can be further stored in the memory 804 or transmitted via the communication component 816. In some embodiments, the audio component 810 also includes a speaker for outputting audio signals.

[0138] I / O interface 812 provides an interface between processing component 802 and peripheral interface modules, such as a keyboard, click wheel, buttons, etc. These buttons may include but are not limited to: a home button, volume buttons, a start button, and a lock button.

[0139] The sensor assembly 814 includes one or more sensors for providing various aspects of the status assessment of the device 800. For example, the sensor assembly 814 can detect the open / closed state of the device 800, the relative positioning of components, such as the display and keypad of the device 800, and the sensor assembly 814 can also search for changes in the position of the device 800 or a component of the device 800, the presence or absence of user contact with the device 800, the orientation or acceleration / deceleration of the device 800, and the temperature change of the device 800. The sensor assembly 814 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 814 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 814 may also include an accelerometer, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0140] The communication component 816 is configured to facilitate wired or wireless communication between the device 800 and other devices. The device 800 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 816 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 816 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency information processing (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0141] In an exemplary embodiment, the apparatus 800 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above-described method.

[0142] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 804 including instructions, and the instructions can be executed by the processor 820 of the apparatus 800 to perform the above method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0143] Figure 71 is a schematic diagram of the structure of a server in some embodiments of the present invention. The server 1900 may vary greatly due to different configurations or performance, and may include one or more central processing units (CPUs) 1922 (for example, one or more processors) and memory 1932, and one or more storage media 1930 (for example, one or more mass storage devices) storing application programs 1942 or data 1944. Among them, the memory 1932 and the storage medium 1930 can be temporary storage or permanent storage. The program stored in the storage medium 1930 may include one or more modules (not shown in the figure), each module may include a series of instruction operations on the server. Furthermore, the central processing unit 1922 can be configured to communicate with the storage medium 1930 to execute a series of instruction operations in the storage medium 1930 on the server 1900.

[0144] The server 1900 may also include one or more power supplies 1926, one or more wired or wireless network interfaces 1950, one or more input and output interfaces 1958, one or more keyboards 1956, and / or one or more operating systems 1941, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, etc.

[0145] A non-transitory computer-readable storage medium, when the instructions in the storage medium are executed by the processor of a device (server or terminal), enables the device to perform Figure 1 The neural network model training method shown.

[0146] A non-transitory computer-readable storage medium, when the instructions in the storage medium are executed by the processor of the device (server or terminal), enables the device to perform the above Figure 1 The description of the neural network model training method in the corresponding embodiment will not be repeated here. In addition, the description of the beneficial effects of adopting the same method will not be repeated. For technical details not disclosed in the computer program product or computer program embodiment involved in this application, please refer to the description of the method embodiment of this application.

[0147] In addition, it should be noted that: the embodiment of the present application also provides a computer program product or computer program, which may include computer instructions, which may be stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor may execute the computer instructions, so that the computer device performs the above Figure 1The description of the neural network model training method in the corresponding embodiment will not be repeated here. In addition, the description of the beneficial effects of adopting the same method will not be repeated. For technical details not disclosed in the computer program product or computer program embodiment involved in this application, please refer to the description of the method embodiment of this application.

[0148] Other embodiments of the present invention will readily occur to those skilled in the art after considering the specification and practicing the invention disclosed herein. The present invention is intended to cover any variations, uses, or adaptations of the present invention that follow from the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered as exemplary only, with the true scope and spirit of the invention being indicated by the following claims.

[0149] It should be understood that the present invention is not limited to the exact construction described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present invention is limited only by the appended claims.

[0150] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

[0151] The above is a detailed introduction to a neural network model training method, a neural network model training device, a device for neural network model training and a secure computing platform provided by the present invention. Specific examples are used herein to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the ideas of the present invention, there will be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A neural network model training method, characterized in that: A computing node applied to a secure computing platform, the secure computing platform including at least one computing unit, each computing unit including at least one computing node and a scheduling node, the scheduling node being used to receive a training script sent by a participant, the method comprising: Obtaining training data from a participant, wherein the training data is ciphertext; Receiving the training script sent by the scheduling node; Initialize the neural network model by executing the training script, wherein the network layer of the neural network model includes a plaintext layer and a ciphertext layer, and further includes a conversion layer between the plaintext layer and the ciphertext layer, wherein the conversion layer is used to convert between plaintext and ciphertext; The neural network model is iteratively trained based on the held training data. In each round of iterative training, the training data of this round is securely calculated layer by layer through each network layer of the neural network model. When the iteration termination condition is not met, the parameters of each network layer are adjusted layer by layer through back propagation according to the model output result. When the iteration termination condition is reached, a trained neural network model is obtained; wherein, the calculation of the plaintext layer is based on the plaintext, and the calculation of the ciphertext layer is based on the ciphertext.

2. The method according to claim 1, characterized in that The network layer includes an input layer, at least one hidden layer, and an output layer in sequence, and the plaintext layer includes at least one hidden layer and / or an output layer.

3. The method according to claim 1, characterized in that If the network layer preceding the plaintext layer is a ciphertext layer, a first conversion layer is further included between the plaintext layer and the ciphertext layer, and the method further includes: During the forward feature data propagation process of each round of iterative training, the forward feature data output by the previous network layer of the plaintext layer is converted into plaintext through the first conversion layer and then passed into the plaintext layer. In the reverse gradient propagation process, the reverse gradient data of the plaintext layer is converted into ciphertext through the first conversion layer and then passed into the previous network layer of the plaintext layer.

4. The method according to claim 1, wherein If the network layer preceding the ciphertext layer is a plaintext layer, a second conversion layer is further included between the ciphertext layer and the plaintext layer, and the method further includes: During the forward feature data propagation process of each round of iterative training, the forward feature data output by the previous network layer of the ciphertext layer is converted into ciphertext through the second conversion layer and then passed into the ciphertext layer. In the reverse gradient propagation process, the reverse gradient data of the ciphertext layer is converted into plaintext through the second conversion layer and then passed into the previous network layer of the ciphertext layer.

5. The method according to claim 1, wherein The training script includes setting information and training code of the neural network model.

6. The method according to claim 1, characterized in that The calculation of the plaintext layer is performed based on the plaintext, and the calculation of the ciphertext layer is performed based on the ciphertext, including: Execute the calculation of the plaintext layer based on the plaintext respectively by the computing nodes in the computing unit specified in the training script; The calculation of the ciphertext layer is performed based on the ciphertext by the computing nodes in the computing unit specified in the training script.

7. The method according to claim 1, characterized in that The training data comes from at least one participant, and the training data includes sample data and label data, and the sample data and label data are held by different participants.

8. The method according to claim 1, characterized in that The secure computing includes any one of the following: secret sharing, obfuscated circuits, and homomorphic encryption.

9. A prediction method based on a neural network model, characterized in that: Applied to a computing node in a secure computing platform, the secure computing platform includes at least one computing unit, each computing unit includes at least one computing node, the method includes: Obtaining prediction data from a participant, wherein the prediction data is ciphertext; Based on the held prediction data, the prediction data is securely calculated layer by layer through each network layer of the neural network model to obtain a model prediction result, wherein, in the process of securely calculating the prediction data layer by layer through each network layer of the neural network model, the calculation of the plaintext layer is performed based on the plaintext, and the calculation of the ciphertext layer is performed based on the ciphertext, and a conversion layer is also included between the plaintext layer and the ciphertext layer, and the conversion layer is used to convert between plaintext and ciphertext.

10. The method according to claim 9, characterized in that The network layer includes an input layer, at least one hidden layer, and an output layer in sequence, and the plaintext layer includes at least one hidden layer and / or an output layer.

11. A neural network model training device, characterized in that: A computing node applied to a secure computing platform, the secure computing platform including at least one computing unit, each computing unit including at least one computing node and a scheduling node, the scheduling node being used to receive a training script sent by a participant, the apparatus comprising: A training data acquisition module, used to acquire training data from participants, wherein the training data is ciphertext; A script receiving module, configured to receive the training script sent by the scheduling node; A model initialization module is used to initialize the neural network model by executing the training script, wherein the network layer of the neural network model includes a plaintext layer and a ciphertext layer, and a conversion layer is further included between the plaintext layer and the ciphertext layer, wherein the conversion layer is used to convert between plaintext and ciphertext; An iterative training module is used to iteratively train the neural network model based on the training data held. In each round of iterative training, the training data of this round is securely calculated layer by layer through each network layer of the neural network model. When the iteration termination condition is not met, the parameters of each network layer are adjusted layer by layer through back propagation according to the model output result. When the iteration termination condition is reached, a trained neural network model is obtained; wherein, the calculation of the plaintext layer is based on the plaintext, and the calculation of the ciphertext layer is based on the ciphertext.

12. The device according to claim 11, characterized in that The network layer includes an input layer, at least one hidden layer, and an output layer in sequence, and the plaintext layer includes at least one hidden layer and / or an output layer.

13. The device according to claim 11, characterized in that If the network layer preceding the plaintext layer is a ciphertext layer, a first conversion layer is further included between the plaintext layer and the ciphertext layer, and the apparatus further includes: The first conversion module is used to convert the forward feature data output by the previous network layer of the plaintext layer into plaintext through the first conversion layer during the forward feature data propagation process of each round of iterative training, and then pass it into the plaintext layer; and during the reverse gradient propagation process, convert the reverse gradient data of the plaintext layer into ciphertext through the first conversion layer, and then pass it into the previous network layer of the plaintext layer.

14. The device according to claim 11, characterized in that If the network layer preceding the ciphertext layer is a plaintext layer, a second conversion layer is further included between the ciphertext layer and the plaintext layer, and the apparatus further includes: The second conversion module is used to convert the forward feature data output by the previous network layer of the ciphertext layer into ciphertext through the second conversion layer during the forward feature data propagation process of each round of iterative training, and then pass it into the ciphertext layer; and in the reverse gradient propagation process, convert the reverse gradient data of the ciphertext layer into plaintext through the second conversion layer, and then pass it into the previous network layer of the ciphertext layer.

15. The device according to claim 11, characterized in that The training script includes setting information and training code of the neural network model.

16. The device according to claim 11, characterized in that The iterative training module includes: A plaintext calculation submodule, configured to perform calculations of the plaintext layer based on plaintexts respectively through calculation nodes in the specified calculation units in the training script; The ciphertext calculation submodule is used to perform the calculation of the ciphertext layer based on the ciphertext by using the calculation nodes in the specified calculation unit in the training script.

17. The device according to claim 11, characterized in that The training data comes from at least one participant, and the training data includes sample data and label data, and the sample data and label data are held by different participants.

18. The device according to claim 11, characterized in that The secure computing includes any one of the following: secret sharing, obfuscated circuits, and homomorphic encryption.

19. A prediction device based on a neural network model, characterized in that: The device is applied to a computing node in a secure computing platform, wherein the secure computing platform includes at least one computing unit, each computing unit includes at least one computing node, and the device includes: A prediction data acquisition module, used to obtain prediction data from participants, wherein the prediction data is ciphertext; The model prediction module is used to securely calculate the prediction data layer by layer through each network layer of the neural network model based on the held prediction data to obtain a model prediction result. In the process of securely calculating the prediction data layer by layer through each network layer of the neural network model, the calculation of the plaintext layer is based on the plaintext, and the calculation of the ciphertext layer is based on the ciphertext. A conversion layer is also included between the plaintext layer and the ciphertext layer, and the conversion layer is used to convert between plaintext and ciphertext.

20. The device according to claim 19, characterized in that The network layer includes an input layer, at least one hidden layer, and an output layer in sequence, and the plaintext layer includes at least one hidden layer and / or an output layer.

21. A secure computing platform, characterized in that: The secure computing platform includes at least one computing unit, each computing unit includes at least one computing node and a scheduling node, and the scheduling node is used to receive the training script sent by the participant; wherein, The computing node is used to obtain training data from the participants, receive the training script sent by the scheduling node, initialize the neural network model by executing the training script, and iteratively train the neural network model based on the held training data. In each round of iterative training, the training data of this round is securely calculated layer by layer through each network layer of the neural network model. When the iteration termination condition is not met, the parameters of each network layer are adjusted layer by layer through back propagation according to the model output result. When the iteration termination condition is reached, a trained neural network model is obtained; wherein, the training data is ciphertext, and the network layer of the neural network model includes a plaintext layer and a ciphertext layer. The calculation of the plaintext layer is based on the plaintext, and the calculation of the ciphertext layer is based on the ciphertext. A conversion layer is also included between the plaintext layer and the ciphertext layer, and the conversion layer is used to convert between plaintext and ciphertext.

22. The secure computing platform according to claim 21, wherein: The secure computing platform also includes a scheduling node for receiving training scripts sent by participants; The computing node is further configured to receive the training script sent by the scheduling node, wherein the training script includes setting information and training code of the neural network model.

23. A device for training a neural network model, characterized in that: The invention comprises a memory and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by one or more processors, and the one or more programs include instructions for performing the neural network model training method as described in any one of claims 1 to 8.

24. A machine-readable medium having instructions stored thereon, which, when executed by one or more processors of a device, causes the device to execute the neural network model training method as described in any one of claims 1 to 8.

Images