Neural network training method and system based on data isolation
By splitting the gradient component value and substituting it into a preset function, the problem of users worrying about private training data being leaked is solved, and the secure summary of multi-user gradient values and distributed training of neural networks are realized, which improves data Security and neural network quality.
Patent Information
- Application Number
- CN202411535058.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-31
- Publication Date
- 2025-02-11
AI Technical Summary
When multiple users train neural networks, they are worried that private training data will be acquired by third parties, which makes it difficult to reflect the value of the training data and the quality of the neural network declined.
Using a neural network training method based on data isolation, the gradient component value is randomly split into multiple copies, and the split value is substituted into a preset function, and the function value is sent to the operator, and the operator calculates and summarizes it to train the nerves. network.
Without leaking user's private training data, the sum of gradient component values of multiple users is calculated and distributed training of neural networks is realized to ensure data security and improve neural network quality.
Smart Images
Figure FT_1
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of neural network training, and in particular to a neural network training method and system based on data isolation. Background Art
[0002] Neural networks are generally multi-user. Multiple users may have private training data for neural networks that they do not want to be obtained by third parties. However, if this part of the training data is not used to train the neural network, the value of the training data will be difficult to reflect and the quality of the neural network will be greatly reduced. Summary of the invention
[0003] The present application provides a neural network training method and system based on data isolation, which can realize distributed training of distributed neural networks without leaking the user's original training data.
[0004] In a first aspect, the present application provides a neural network training method based on data isolation. The method is applied to a system consisting of a neural network operator and multiple users holding private training data. The method includes: as a user, Use private training data to train the held neural network and obtain the gradient component value; Randomly split the gradient component value into agreed first preset integer parts; Substituting the split gradient component values into at least two agreed preset functions respectively to obtain at least two substituted function values, wherein the preset functions are all first preset integer-element functions, and the sum of all the preset functions is the sum of each independent variable times a second preset constant, and the second preset constant is non-zero; All function values are sent to the operator so that the operator can obtain the sum of the gradient component values of all users by dividing the sum of all function values of all users by the agreed second preset constant, and the operator trains the neural network based on the sum of the gradient component values.
[0005] In a second aspect, the present application provides a neural network training method based on data isolation. The method is applied to a system consisting of a neural network operator and multiple users holding private training data. The method includes: as the operator, Receive all function values sent by all users, wherein the function values are obtained by each user randomly splitting the gradient component values obtained by training using private training data into agreed first preset integer parts, and substituting the split gradient component values into agreed at least two preset functions, wherein the preset functions are all first preset integer element functions, and the sum of all preset functions is the sum of each independent variable times a second preset constant, and the second preset constant is non-zero; The sum of all function values divided by the agreed second preset constant is equal to the sum of the gradient component values held by all users, and the operator trains the neural network based on the sum of the gradient component values.
[0006] In a third aspect, the present application provides a neural network training method based on data isolation. The method is applied to a system consisting of a neural network operator and multiple users holding private training data, and the method includes: Each user uses private training data to train the neural network and obtains the gradient component value; Randomly split the gradient component value into agreed first preset integer parts; Substituting the split gradient component values into at least two agreed preset functions respectively to obtain at least two substituted function values, wherein the preset functions are all first preset integer-element functions, and the sum of all the preset functions is the sum of each independent variable times a second preset constant, and the second preset constant is non-zero; The user sends all the function values to the operator so that the operator can calculate the sum of all the function values; The operator calculates that the sum of all function values divided by the agreed second preset constant is equal to the sum of the gradient component values held by all users, and the operator trains the neural network based on the sum of the gradient component values.
[0007] Furthermore, there are m preset functions, and the jth preset function , where , is the second preset constant.
[0008] Furthermore, there are two preset functions, the first preset function is , the second preset function is , where p+1 is the second preset constant.
[0009] Furthermore, the first preset integer is not less than 5.
[0010] In a fourth aspect, the present application provides a neural network training system based on data isolation. The system includes a neural network operator and multiple users holding private data, wherein the user applies any one of the methods described in the first aspect above, and / or the operator applies any one of the methods described in the second aspect above, and / or the system applies any one of the methods described in the third aspect above.
[0011] By adopting the above technical solution, neural network training calculation based on data isolation can be realized, ensuring that the user's private training data and the original gradient components obtained by training with the private training data are not leaked.
[0012] It should be understood that the contents described in the Summary of the Invention are not intended to limit the key or important features of the embodiments of the present application, nor are they intended to limit the scope of the present application. Other features of the present application will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS
[0013] The above and other features, advantages and aspects of the embodiments of the present application will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. In the accompanying drawings, the same or similar reference numerals represent the same or similar elements, wherein: Figure 1 A flow chart of a neural network training method based on data isolation in an embodiment of the present application is shown. DETAILED DESCRIPTION
[0014] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0015] In addition, the term "and / or" in this article is only a description of the association relationship between the associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.
[0016] The present application provides a neural network training method and system based on data isolation, which can realize the calculation of the sum of gradient component values held by multiple users under the premise of ensuring that the gradient component values held by the users are not leaked.
[0017] In a first aspect, the present application provides a neural network training system based on data isolation, wherein the system includes an operator and multiple users, wherein the users hold private training data and the operator operates the neural network.
[0018] The system realizes secure multi-party training of distributed neural networks and is specifically introduced in accordance with the disclosure of the second aspect of this application.
[0019] In a second aspect, the present application provides a neural network training method based on data isolation. The method can be Figure 1 The user and operator parties cooperate in the implementation.
[0020] Figure 1A flow chart of a neural network training method based on data isolation in an embodiment of the present application is shown.
[0021] Reference Figure 1 , the method specifically comprises the following steps: S110: Each user uses private training data to train the neural network to obtain gradient component values; the process of obtaining gradient components through holding and training is performed locally on the user. S120: randomly splitting the gradient component value into agreed first preset integer parts; S130: Substituting the split gradient component values into at least two predetermined preset functions respectively to obtain at least two substituted function values, wherein the preset functions are all first preset integer-element functions, and the sum of all the preset functions is the sum of each independent variable times a second preset constant, and the second preset constant is non-zero; S140: The user sends all the substituted function values to the operator, so that the operator calculates the sum of all the function values; S150: The operator calculates that the sum of all function values divided by the agreed second preset constant is equal to the sum of the gradient component values held by all users, and the operator trains the neural network based on the sum of the gradient component values.
[0022] Specifically, there are m preset functions, and the jth preset function , where , is the second preset constant.
[0023] In one example, there are two preset functions, the first preset function is , the second preset function is , where p+1 is a second preset constant. The first preset integer is not less than 5.
[0024] The technology of substituting training data into a neural network to obtain gradient component values is common knowledge and is not disclosed here. Only the secure multi-party summation method of gradient components is disclosed in detail. The following integrates the above content to introduce a specific example.
[0025] Suppose there are n users, and the users agree on a first preset integer p, where n is not less than 2 and p is not less than 5; each user randomly splits the gradient component value it holds into p parts, and obtains p split data, and the sum of the p split data is equal to the gradient component value. The split function is defined here , where d represents the gradient component value, Indicates split data.
[0026] Assume that the gradient component value of the nth user is , , ..., , The corresponding p split data are , , ..., , The corresponding p split data are , , ..., , ..., The corresponding p split data are , , ..., .
[0027] Define two p-ary functions, , , each user substitutes the gradient component value held by itself into the two functions respectively to obtain the function values.
[0028] Then n users substitute The obtained function values are ; ; ... ; n users enter The obtained function values are ; ; ... ; These function values are clear numerical results and can be sent to the operator. The operator can calculate the sum of the gradient component values held by the user based on the following steps: calculate ; calculate ; calculate That is, it is equal to p+1 times the gradient component values held by n users. Dividing it by p+1 can get the sum of the gradient component values held by n users.
[0029] Similarly, the average of the gradient component values held by n users can be calculated.
[0030] In a third aspect, the present application provides a neural network training method based on data isolation. The method is applied to a system consisting of a neural network operator and multiple users holding private training data. The method includes: as a user, Use private training data to train the held neural network and obtain the gradient component value; Randomly split the gradient component value into agreed first preset integer parts; Substituting the split gradient component values into at least two agreed preset functions respectively to obtain at least two substituted function values, wherein the preset functions are all first preset integer-element functions, and the sum of all the preset functions is the sum of each independent variable times a second preset constant, and the second preset constant is non-zero; All function values are sent to the operator so that the operator can obtain the sum of the gradient component values of all users by dividing the sum of all function values of all users by the agreed second preset constant, and the operator trains the neural network based on the sum of the gradient component values.
[0031] In a fourth aspect, the present application provides a neural network training method based on data isolation. The method is applied to a system consisting of a neural network operator and multiple users holding private training data. The method includes: as the operator, Receive all function values sent by all users, wherein the function values are obtained by each user randomly splitting the gradient component values obtained by training using private training data into agreed first preset integer parts, and substituting the split gradient component values into agreed at least two preset functions, wherein the preset functions are all first preset integer element functions, and the sum of all preset functions is the sum of each independent variable times a second preset constant, and the second preset constant is non-zero; The sum of all function values divided by the agreed second preset constant is equal to the sum of the gradient component values held by all users, and the operator trains the neural network based on the sum of the gradient component values.
[0032] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the embodiments of the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily required by the present application.
[0033] The above description is only a preferred embodiment of the present application and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present application is not limited to the technical solution formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the aforementioned disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in this application (but not limited to) by each other to form a technical solution.
Claims
1. A neural network training method based on data isolation, characterized in that: Applied to a system consisting of an operator of a neural network and multiple users holding private training data, the method comprises: as a user, Use private training data to train the held neural network and obtain the gradient component value; Randomly split the gradient component value into agreed first preset integer parts; Substituting the split gradient component values into at least two agreed preset functions respectively to obtain at least two substituted function values, wherein the preset functions are all first preset integer-element functions, and the sum of all the preset functions is the sum of each independent variable times a second preset constant, and the second preset constant is non-zero; All function values are sent to the operator so that the operator can obtain the sum of the gradient component values of all users by dividing the sum of all function values of all users by the agreed second preset constant, and the operator trains the neural network based on the sum of the gradient component values.
2. A neural network training method based on data isolation, characterized in that: Applied to a system consisting of an operator of a neural network and multiple users holding private training data, the method comprises: as the operator, Receive all function values sent by all users, wherein the function values are obtained by each user randomly splitting the gradient component values obtained by training using private training data into agreed first preset integer parts, and substituting the split gradient component values into agreed at least two preset functions, wherein the preset functions are all first preset integer element functions, and the sum of all preset functions is the sum of each independent variable times a second preset constant, and the second preset constant is non-zero; The sum of all function values divided by the agreed second preset constant is equal to the sum of the gradient component values held by all users, and the operator trains the neural network based on the sum of the gradient component values.
3. A neural network training method based on data isolation, characterized in that: Applied to a system consisting of an operator of a neural network and multiple users holding private training data, the method comprises: Each user uses private training data to train the neural network and obtains the gradient component value; Randomly split the gradient component value into agreed first preset integer parts; Substituting the split gradient component values into at least two agreed preset functions respectively to obtain at least two substituted function values, wherein the preset functions are all first preset integer-element functions, and the sum of all the preset functions is the sum of each independent variable times a second preset constant, and the second preset constant is non-zero; The user sends all the function values to the operator so that the operator can calculate the sum of all the function values; The operator calculates that the sum of all function values divided by the agreed second preset constant is equal to the sum of the gradient component values held by all users, and the operator trains the neural network based on the sum of the gradient component values.
4. The neural network training method based on data isolation according to any one of claims 1 to 3, characterized in that: There are m preset functions, and the jth preset function , where , is the second preset constant.
5. The neural network training method based on data isolation according to any one of claims 1 to 3, characterized in that: There are two preset functions. The first preset function is , the second preset function is , where p+1 is the second preset constant.
6. The neural network training method based on data isolation according to any one of claims 1 to 3, characterized in that: The first preset integer is not less than 5.
7. A neural network training system based on data isolation, characterized in that: The system includes an operator of a neural network and multiple users holding private data, the users applying the method described in any one of claims 1 and 4-6, and / or the operator applying the method described in any one of claims 2 and 4-6, and / or the system applying the method described in any one of claims 3 and 4-6.