Multi-party security training method and system of neural network

Abstract

The invention provides a multi-party security training method and system for a neural network, and belongs to the field of distributed neural network training, and the method comprises the steps: training a local neural network by a user side through private training data to obtain a private gradient value, and randomly dividing the private gradient value held by the user side into a first preset integer part; each user side respectively substitutes the split private gradient value into at least two preset functions to obtain at least two substituted function values, the preset functions are first preset integer element functions, the sum of all the preset functions is the sum of independent variables multiplied by a second preset constant, and the second preset constant is not zero; the user side sends all the substituted function values to the operator, so that the operator calculates the sum of all the function values; and the operator calculates that the sum of all the function values is divided by an agreed second preset constant to be equal to the sum of private gradient values held by all the user parties, so that neural network training can be realized on the premise that private training data and the private gradient values are not leaked.

Description

Technical Field

[0001] The present application relates to the technical field of neural network training, and in particular to a multi-party secure training method and system for a neural network. Background Art

[0002] Neural networks are generally multi-user. Multiple users may have private training data for neural networks that they do not want to be obtained by third parties. However, if this part of the training data is not used to train the neural network, the value of the training data will be difficult to reflect and the quality of the neural network will be greatly reduced. Summary of the invention

[0003] The present application provides a multi-party secure training method and system for a neural network, which can implement distributed training of a distributed neural network without leaking the user's original training data.

[0004] In a first aspect, the present application provides a multi-party secure training method for a neural network. The method is applied to a system consisting of an operator of a neural network and multiple distributors holding distributed neural networks, each distributor's distributed neural network having a system consisting of multiple users holding private training data, the method comprising: as a user, Use private training data to train the held neural network and obtain private gradient values; Randomly split the private gradient value into an agreed first preset integer number of parts; Substituting the split private gradient values ​​into at least two agreed preset functions respectively to obtain at least two substituted function values, wherein the preset functions are all first preset integer-element functions, and the sum of all the preset functions is the sum of each independent variable times a second preset constant, and the second preset constant is non-zero; All function values ​​are sent to the operator so that the operator can obtain the sum of the private gradient values ​​of all users based on the sum of all function values ​​of all users divided by the agreed second preset constant. The operator trains the neural network based on the sum of the private gradient values ​​and synchronizes it to the distributed neural network held by the distributor.

[0005] In a second aspect, the present application provides a multi-party secure training method for a neural network. The method is applied to a system consisting of a neural network operator and multiple distributors holding distributed neural networks, each distributor's distributed neural network having a system consisting of multiple users holding private training data, the method comprising: as an operator, Receive all function values ​​sent by all users, where the function values ​​are obtained by each user randomly splitting a private gradient value obtained by training using private training data into agreed first preset integer parts, and substituting the split private gradient values ​​into at least two agreed preset functions, respectively. The preset functions are all first preset integer element functions, and the sum of all preset functions is the sum of each independent variable times a second preset constant, where the second preset constant is non-zero; The sum of all function values ​​divided by the agreed second preset constant is equal to the sum of the private gradient values ​​held by all users. The operator trains the neural network based on the sum of the private gradient values ​​and synchronizes it to the distributed neural network held by the distributor.

[0006] In a third aspect, the present application provides a multi-party secure training method for a neural network. The method is applied to a system consisting of an operator of a neural network and multiple distributors holding distributed neural networks, each distributor's distributed neural network having a system consisting of multiple users holding private training data, and the method includes: Each user uses private training data to train the neural network and obtains private gradient values; Randomly split the private gradient value into an agreed first preset integer number of parts; Substituting the split private gradient values ​​into at least two agreed preset functions respectively to obtain at least two substituted function values, wherein the preset functions are all first preset integer-element functions, and the sum of all the preset functions is the sum of each independent variable times a second preset constant, and the second preset constant is non-zero; The user sends all the function values ​​to the operator so that the operator can calculate the sum of all the function values; The operator calculates the sum of all function values ​​divided by the agreed second preset constant equal to the sum of private gradient values ​​held by all users. The operator trains the neural network based on the sum of private gradient values ​​and synchronizes it to the distributed neural network held by the distributor.

[0007] Furthermore, there are m preset functions, and the jth preset function , where , is the second preset constant.

[0008] Furthermore, there are two preset functions, the first preset function is , the second preset function is , where p+1 is the second preset constant.

[0009] Furthermore, the first preset integer is not less than 5.

[0010] In a fourth aspect, the present application provides a multi-party secure training system for a neural network. The system includes an operator of a neural network and multiple users holding private data, wherein the user applies any one of the methods described in the first aspect above, and / or the operator applies any one of the methods described in the second aspect above, and / or the system applies any one of the methods described in the third aspect above.

[0011] By adopting the above technical solution, multi-party secure training calculation of the neural network can be realized, ensuring that the user's private training data and the original gradient components obtained by training with the private training data are not leaked.

[0012] It should be understood that the contents described in the Summary of the Invention are not intended to limit the key or important features of the embodiments of the present application, nor are they intended to limit the scope of the present application. Other features of the present application will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] The above and other features, advantages and aspects of the embodiments of the present application will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. In the accompanying drawings, the same or similar reference numerals represent the same or similar elements, wherein: Figure 1 A flowchart of a multi-party secure training method for a neural network in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0014] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0015] In addition, the term "and / or" in this article is only a description of the association relationship between the associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0016] The present application provides a multi-party secure training method and system for a neural network, which can calculate the sum of private gradient values ​​held by multiple users while ensuring that the private gradient values ​​held by the users are not leaked.

[0017] In a first aspect, the present application provides a multi-party security training system for a neural network. The system includes an operator and multiple distributors, each distributor holds a distributed neural network, each distributor's distributed neural network has multiple users, each user holds private training data, and the operator operates the neural network.

[0018] The system realizes secure multi-party training of distributed neural networks and is specifically introduced in accordance with the disclosure of the second aspect of this application.

[0019] In a second aspect, the present application provides a multi-party secure training method for a neural network. The method can be performed by Figure 1 The user and operator parties cooperate in the implementation.

[0020] Figure 1 A flowchart of a multi-party secure training method for a neural network in an embodiment of the present application is shown.

[0021] Reference Figure 1 , the method specifically comprises the following steps: S110: Each user uses private training data to train the neural network to obtain private gradient values; the process of obtaining the gradient components through holding and training is performed locally on the user side; S120: randomly splitting the private gradient value into agreed first preset integer parts; S130: Substituting the split private gradient values ​​into at least two agreed preset functions respectively to obtain at least two substituted function values, wherein the preset functions are all first preset integer-element functions, and the sum of all the preset functions is the sum of each independent variable times a second preset constant, and the second preset constant is non-zero; S140: The user sends all the substituted function values ​​to the operator, so that the operator calculates the sum of all the function values; S150: The operator calculates the sum of all function values ​​divided by the agreed second preset constant equal to the sum of private gradient values ​​held by all users, and the operator trains the neural network based on the sum of private gradient values ​​and synchronizes it to the distributed neural network held by the distributor.

[0022] Specifically, there are m preset functions, and the jth preset function , where , is the second preset constant.

[0023] In one example, there are two preset functions, the first preset function is , the second preset function is , where p+1 is a second preset constant. The first preset integer is not less than 5.

[0024] The technology of substituting training data into a neural network to obtain private gradient values ​​is common knowledge and is not disclosed here. Only the secure multi-party summation method of gradient components is disclosed in detail. After the operator has trained the neural network, it will synchronize the training results to all distributed neural networks, so that each dealer's distributed neural network can also be trained. The interaction between the user and the operator can be achieved directly through the Internet or through the dealer.

[0025] The following integrates the above content to introduce a specific example.

[0026] Suppose there are n users, and the users agree on a first preset integer p, where n is not less than 2 and p is not less than 5; each user randomly splits the private gradient value it holds into p parts, and obtains p split data, and the sum of the p split data is equal to the private gradient value. Here, the split function is defined as , where d represents the private gradient value, Indicates split data.

[0027] Suppose the private gradient value of the nth user is , , ..., , The corresponding p split data are , The corresponding p split data are , ..., The corresponding p split data are .

[0028] Define two p-ary functions, , , each user substitutes its own private gradient value into the two functions and obtains the function values ​​respectively.

[0029] Then n users substitute The obtained function values ​​are ; ; ... ; n users enter The obtained function values ​​are ; ; ... ; These function values ​​are clear numerical results and can be sent to the operator. The operator can calculate the sum of the private gradient values ​​held by the user based on the following steps: calculate ; calculate ; calculate That is, it is equal to p+1 times the private gradient values ​​held by n users. Dividing it by p+1 can get the sum of the private gradient values ​​held by n users.

[0030] Similarly, the average of the private gradient values ​​held by n users can be calculated.

[0031] In a third aspect, the present application provides a multi-party secure training method for a neural network. The method is applied to a system consisting of a neural network operator and multiple distributors holding distributed neural networks, each distributor's distributed neural network having a system consisting of multiple users holding private training data, the method comprising: as a user, Use private training data to train the held neural network and obtain private gradient values; Randomly split the private gradient value into an agreed first preset integer number of parts; Substituting the split private gradient values ​​into at least two agreed preset functions respectively to obtain at least two substituted function values, wherein the preset functions are all first preset integer-element functions, and the sum of all the preset functions is the sum of each independent variable times a second preset constant, and the second preset constant is non-zero; All function values ​​are sent to the operator so that the operator can obtain the sum of the private gradient values ​​of all users based on the sum of all function values ​​of all users divided by the agreed second preset constant. The operator trains the neural network based on the sum of the private gradient values ​​and synchronizes it to the distributed neural network held by the distributor.

[0032] In a fourth aspect, the present application provides a multi-party secure training method for a neural network. The method is applied to a system consisting of a neural network operator and multiple distributors holding distributed neural networks, each distributor's distributed neural network having a system consisting of multiple users holding private training data, the method comprising: as an operator, Receive all function values ​​sent by all users, where the function values ​​are obtained by each user randomly splitting a private gradient value obtained by training using private training data into agreed first preset integer parts, and substituting the split private gradient values ​​into at least two agreed preset functions, respectively. The preset functions are all first preset integer element functions, and the sum of all preset functions is the sum of each independent variable times a second preset constant, where the second preset constant is non-zero; The sum of all function values ​​divided by the agreed second preset constant is equal to the sum of the private gradient values ​​held by all users. The operator trains the neural network based on the sum of the private gradient values ​​and synchronizes it to the distributed neural network held by the distributor.

[0033] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the embodiments of the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily required by the present application.

[0034] The above description is only a preferred embodiment of the present application and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present application is not limited to the technical solution formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the aforementioned disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in this application (but not limited to) by each other to form a technical solution.

Claims

1. A multi-party secure training method for a neural network, characterized in that: The method is applied to a system consisting of an operator of a neural network and multiple dealers holding distributed neural networks, wherein each dealer's distributed neural network has multiple users holding private training data, and the method comprises: as a user, Use private training data to train the held neural network and obtain private gradient values; Randomly split the private gradient value into an agreed first preset integer number of parts; Substituting the split private gradient values ​​into at least two agreed preset functions respectively to obtain at least two substituted function values, wherein the preset functions are all first preset integer-element functions, and the sum of all the preset functions is the sum of each independent variable times a second preset constant, and the second preset constant is non-zero; All function values ​​are sent to the operator so that the operator can obtain the sum of the private gradient values ​​of all users based on the sum of all function values ​​of all users divided by the agreed second preset constant. The operator trains the neural network based on the sum of the private gradient values ​​and synchronizes it to the distributed neural network held by the distributor.

2. A multi-party secure training method for a neural network, characterized in that: The method is applied to a system consisting of an operator of a neural network and multiple dealers holding distributed neural networks, wherein each dealer's distributed neural network has a system consisting of multiple users holding private training data, the method comprising: as the operator, Receive all function values ​​sent by all users, where the function values ​​are obtained by each user randomly splitting a private gradient value obtained by training using private training data into agreed first preset integer parts, and substituting the split private gradient values ​​into at least two agreed preset functions, respectively. The preset functions are all first preset integer element functions, and the sum of all preset functions is the sum of each independent variable times a second preset constant, where the second preset constant is non-zero; The sum of all function values ​​divided by the agreed second preset constant is equal to the sum of the private gradient values ​​held by all users. The operator trains the neural network based on the sum of the private gradient values ​​and synchronizes it to the distributed neural network held by the distributor.

3. A multi-party secure training method for a neural network, characterized in that: Applied to a system consisting of an operator of a neural network and multiple distributors holding distributed neural networks, each distributor's distributed neural network having a system consisting of multiple users holding private training data, the method comprising: Each user uses private training data to train the neural network and obtains private gradient values; Randomly split the private gradient value into an agreed first preset integer number of parts; Substituting the split private gradient values ​​into at least two agreed preset functions respectively to obtain at least two substituted function values, wherein the preset functions are all first preset integer-element functions, and the sum of all the preset functions is the sum of each independent variable times a second preset constant, and the second preset constant is non-zero; The user sends all the function values ​​to the operator so that the operator can calculate the sum of all the function values; The operator calculates the sum of all function values ​​divided by the agreed second preset constant equal to the sum of private gradient values ​​held by all users. The operator trains the neural network based on the sum of private gradient values ​​and synchronizes it to the distributed neural network held by the distributor.

4. The multi-party secure training method for a neural network according to any one of claims 1 to 3, characterized in that: There are m preset functions, and the jth preset function , where , is the second preset constant.

5. The multi-party secure training method for a neural network according to any one of claims 1 to 3, characterized in that: There are two preset functions. The first preset function is , the second preset function is , where p+1 is the second preset constant.

6. The multi-party secure training method for a neural network according to any one of claims 1 to 3, characterized in that: The first preset integer is not less than 5.

7. A multi-party secure training system for a neural network, characterized in that: The system includes a neural network operator and multiple dealers holding distributed neural networks. The distributed neural network of each dealer has multiple users holding private data. The users apply the method described in any one of claims 1 and 4-6, and / or the operator applies the method described in any one of claims 2 and 4-6, and / or the system applies the method described in any one of claims 3 and 4-6.

Images