A training method and device of a longitudinal federated neural network based on MPC, and an inference method

Abstract

The application discloses a training method and device of a longitudinal federal neural network based on MPC. In order to solve the problem that the longitudinal federal split learning is vulnerable to model completion attack, the first n layers of the neural network adopt model fragmentation technology, and there is no complete bottom model, so the model completion attack cannot be initiated. In order to solve the efficiency problem caused by model fragmentation, after the n layers, the model fragmentation and the fragmentation of the calculation result are added by using the existing model sharing patent idea, that is, the training and reasoning mode of a single server is used to speed up the running speed. Since n is usually 1 or 2, the training and reasoning efficiency of the whole system is close to the effect of plaintext, and the security and privacy characteristics are maintained. For the scene of multiple label providers, the model and the model result can be split and combined, and the model and the result are split again to protect the privacy of the label. And the overall structure is still applicable to the application scene of split learning.

Description

Technical Field

[0001] The present invention relates to the field of data security and privacy utilization technology, and is a privacy-preserving fast privacy calculation method for a vertical federated neural network, and in particular to a training method and device, and an inference method for a vertical federated neural network based on MPC. Background Art

[0002] Privacy computing provides a method for utilizing data while preserving security and privacy. Currently, common privacy computing solutions include privacy computing based on secure multiparty computation (SMPC), privacy computing based on federated learning (FL), and privacy computing based on trusted execution environments (TEEs).

[0003] Federated learning can be categorized into horizontal, vertical, and transfer federation. For vertical federated learning, a wide range of application scenarios are suitable for the split neural network approach. This approach splits the neural network model into two parts: the bottom and top models. The client providing the data trains the bottom model locally and sends the intermediate results to the server. The server aggregates the intermediate results from multiple parties and adds its own to complete the remaining neural network operations. During the backpropagation phase, the server transmits the gradient to each client, and each client completes the subsequent backpropagation. The server is typically operated by the active party that owns the label.

[0004] Many studies have shown that the client can launch a label inference attack on the label through the received gradient or bottom model or model output, and the server can obtain the client's data features through the intermediate results of the bottom output. Although there has been a lot of research to prevent the above attacks, according to the research updated in August 2015, "A Framework for Evaluating Privacy-Utility Trade-off in Vertical Federated Learning", model inversion attacks and most label inference attacks can be protected using existing methods, but model completion attacks are difficult to prevent. This is mainly because the trained bottom model is trained with labels and data and already has the ability to partition data. Therefore, only a small amount of additional labeled data is needed to generate a complete neural network based on the bottom model for predicting labels.

[0005] From this, we can see that the existing vertical federated neural network has the technical problem of model completion attack. SUMMARY

[0006] The present application aims at the deficiencies of the prior art, and provides a training method and device of a longitudinal federated neural network based on MPC, and an inference method.

[0007] The technical solutions adopted by the present application are as follows:

[0008] The first aspect provides a training method of a longitudinal federated neural network based on MPC, applied to a system including a data provider and two servers, the first n layers of a neural network model are respectively deployed on the first server and the second server in an arithmetic slicing manner, and each layer above the n layers of the neural network model is all deployed on the second server, and the training method comprises the following steps:

[0009] The data provider performs attribute expansion on respective sample data, performs slicing on the sample data after attribute expansion, and then sends the sample data to the first server and the second server;

[0010] When the first server and the second server receive a certain number of sample slices, if a sample has a label, the first vector is obtained by performing arithmetic addition on each attribute of all slices of the sample and dividing the number of slices, as training data;

[0011] The first server and the second server train the neural network model of the first n layers based on the training data according to a known two-party secure protocol, in the forward propagation stage, when the neural network model runs to the n+1 layer, the first server sends the calculation result of the output of the n layer of itself to the second server, the second server performs arithmetic addition on the received calculation result and the calculation result of itself as the input of the n+1 layer, and runs the neural network after that; in the backward propagation stage, when the neural network model runs to the n layer, the second server performs arithmetic slicing on the calculated n layer gradient, and sends one of the gradient slices to the first server, and then the first server and the second server complete the backward propagation process of the gradient respectively, and n is a positive integer.

[0012] In an embodiment, when a sample has a label, the method further comprises that the data provider slices the label.

[0013] In an embodiment, the method further comprises that when a sample slice has no label, the corresponding sample slice is left to the next batch processing, and if a sample slice has no label within a preset time, the corresponding sample slice is deleted.

[0014] In an embodiment, when only one of the data providers provides a label, the method further comprises that the second server is operated by the data provider providing the label.

[0015] In one embodiment, when there are multiple data providers providing labels, the method further includes: splitting the model weight parameters of layers mk to m in the initialization phase, specifically:

[0016] In the forward propagation phase, when running to the mk layer, where k = 1 or 2, the second server splits the model's calculation results in the last layer or two layers of the model. Specifically, the input calculation results of the mk layer are sliced, and then the first server and the second server respectively use their respective slices as input to run their respective neural networks; in the backward propagation phase, the first server and the second server respectively calculate the difference between the label slice and the output of their respective neural networks, use the chain rule to calculate the gradient of the difference with respect to each model parameter, and use the gradient to update the model parameters, where m represents the total number of layers of the neural network model;

[0017] In the back propagation stage, when the back propagation reaches the end of the mk layer, the first server sends the gradient slice propagated in this layer to the second server. The second server calculates the arithmetic sum of its own gradient slice and the received gradient slice, and then continues back propagation using the chain rule. When the back propagation reaches the n layer, the gradient of the n layer is arithmetically sliced, and one of the gradient slices is sent to the first server; then the first server and the second server each use the chain rule to calculate the gradient of the model parameters based on their own gradient slices, and use the gradient to update the model parameters.

[0018] Based on the same inventive concept, a second aspect of the present invention provides an MPC-based vertical federated neural network training device, which is applied to a system including a data provider and two servers. The first n layers of the neural network model are respectively deployed on the first server and the second server in an arithmetic slicing manner, and all layers above the nth layer of the neural network model are deployed on the second server. The training device includes:

[0019] The sample attribute expansion module is used for data providers to expand the attributes of their respective sample data, and to split the attribute-expanded sample data into slices before sending them to the first server and the second server;

[0020] A training data acquisition module is configured to, when the first server and the second server receive a certain number of sample fragments, perform arithmetic summation of the attributes of all fragments of the sample, if a sample has a label, and divide the sum by the number of fragments to obtain a first vector as training data;

[0021] A training module is used for the first server and the second server to run the first n layers of the neural network model for training based on the training data according to a known two-party security protocol. In the forward propagation stage, when the neural network model runs to the n+1 layer, the first server sends the calculation result of its own n-th layer output to the second server, and the second server performs arithmetic addition on the received calculation result and its own calculation result as the input of the n+1 layer, and runs the subsequent neural network; in the reverse propagation stage, when the neural network model runs to the n-th layer, the second server performs arithmetic slicing on the calculated n-layer gradient and sends one of the gradient slices to the first server, and then the first server and the second server each complete the reverse propagation process of the gradient, and n is a positive integer.

[0022] Based on the same inventive concept, the third aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the training method of the MPC-based longitudinal federated neural network described in the first aspect.

[0023] Based on the same inventive concept, the fourth aspect of the present invention provides a computer device, including a memory, a processor, and a computer program stored in the memory and runnable on the processor. When the processor executes the program, it implements the training method of the MPC-based longitudinal federated neural network described in the first aspect.

[0024] Based on the same inventive concept, the fourth aspect of the present invention provides an MPC-based longitudinal federated neural network reasoning method, which uses the neural network model obtained in the first aspect to perform reasoning calculations.

[0025] In one embodiment, performing inference calculation using the neural network model obtained in the first aspect includes:

[0026] Each data provider provides data, and the neural network model is run according to the forward propagation process. When only one data provider provides a label, the output of the neural network model is used as the inference result; when multiple data providers provide labels, the output results of the first server and the second server are arithmetic summed as the inference result.

[0027] Compared with the prior art, the advantages and beneficial technical effects of the present invention are as follows:

[0028] Because the first n layers utilize model sharding, a complete bottom model does not exist, making model completion attacks impossible. To address the efficiency issues associated with model sharding, after the nth layer, the model shards and the shards of the computational results are mathematically summed, drawing on existing model sharing patents. This results in a single-server training and inference model to accelerate execution. Since n typically takes a value of 1 or 2, the training and inference efficiency of the entire system approaches that of plaintext, while maintaining security and privacy. For scenarios with multiple label providers, the splittable and mergeable nature of the model and model results allows the model and results to be further split to protect the privacy of the labels. Furthermore, the overall structure remains applicable to split learning scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0030] Figure 1 A flowchart of a method for training a longitudinal federated neural network based on MPC provided in an embodiment of the present invention;

[0031] Figure 2 A diagram showing the overall system architecture of the MPC-based vertical federated neural network training method provided in an embodiment of the present invention;

[0032] Figure 3 Schematic diagram of data sample expansion in an embodiment of the present invention. DETAILED DESCRIPTION

[0033] The inventors of this application have found through extensive research and practice that although the existing machine learning algorithms based on secure multi-party computing are secure and private, they cannot be used in practice due to performance issues. Although vertical federation based on split learning has many protection measures for security and privacy, it still has the problem of model completion attacks. Therefore, the present invention intends to use a method based on secure multi-party computing to solve the model completion attack problem faced by vertical federated deep neural networks, while improving the speed of the solution. In order to prevent the passive party from launching a model completion attack, this solution adopts an MPC-based machine learning method to perform arithmetic sharding on the neural network model parameters, so that the passive party does not have a complete sub-model and cannot launch a model completion attack. In order to reduce the efficiency reduction problem caused by model sharding, the neural network is layered, the bottom layer adopts a sharding method, and the upper layer merges the model shards and result shards, adopting a single server method to increase the speed. The overall solution is safe and practical. When there are multiple label providers, in order to protect the privacy of the labels, the model and the results are split again by taking advantage of the splittable and mergeable characteristics of the model and model results.

[0034] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0035] Example 1

[0036] The present invention discloses a training method for a vertical federated neural network based on MPC, which is applied to a system including a data provider and two servers. The first n layers of the neural network model are respectively deployed on the first server and the second server in an arithmetic sharding manner, and all layers above the nth layer of the neural network model are deployed on the second server. Figure 1 , the training method includes:

[0037] S1: The data providers perform attribute expansion on their respective sample data, split the attribute-expanded sample data into shards, and then send them to the first server and the second server;

[0038] S2: After the first server and the second server receive a certain number of sample shards, if a sample has a label, they perform arithmetic summation on the attributes of all shards of the sample and divide the sum by the number of shards to obtain a first vector as training data.

[0039] S3: The first server and the second server run the first n layers of the neural network model for training based on the training data according to a known two-party security protocol. In the forward propagation stage, when the neural network model runs to the n+1 layer, the first server sends the calculation result of its own n-th layer output to the second server. The second server performs arithmetic addition on the received calculation result and its own calculation result as the input of the n+1 layer and runs the subsequent neural network. In the reverse propagation stage, when the neural network model runs to the n-th layer, the second server performs arithmetic slicing on the calculated n-layer gradient and sends one of the gradient slices to the first server. Then the first server and the second server each complete the reverse propagation process of the gradient, and n is a positive integer.

[0040] Specifically, see Figure 2 , which is a system overall architecture diagram of the MPC-based vertical federated neural network training method provided in an embodiment of the present invention.

[0041] The overall system architecture includes a data provider and first and second servers P1 and P2. The value of n can be determined as needed, for example, n = 1 or 2. When n = 2, the first two layers of the neural network model are deployed on P1 and P2, respectively, in arithmetic slicing, while all layers above layer 2 are deployed on P2. The model parameters are assigned random initial values, and the entire neural network has m layers.

[0042] See Figure 3 The data provider expands the attributes of its sample data so that each sample contains all the characteristic attributes of all data providers' data. The attributes of each data provider are relatively concentrated in the expanded sample attributes. If a data provider's sample has a certain characteristic attribute, the attribute is directly assigned an actual value; otherwise, the attribute is assigned a value of 0. The data provider then performs arithmetic slicing on the expanded sample data. This approach makes the method of the present invention applicable to a wider range of application scenarios.

[0043] In one embodiment, if a sample has a label, the data provider needs to segment the label and send it to the first server and the second server.

[0044] The training process consists of a forward propagation phase and a backpropagation phase. The backpropagation phase primarily involves two servers calculating the difference (error) between the sample label and the output of the neural network's forward propagation. The derivative of this error with respect to each model parameter, i.e., the gradient, is then calculated using the chain rule. The model parameters are then updated based on the learning rate.

[0045] In one embodiment, the method further includes: when a sample fragment has no label, the corresponding sample fragment is retained for next batch processing; if a sample fragment has no label within a preset time, the corresponding sample fragment is deleted.

[0046] Specifically, after the first and second servers receive a certain number of the aforementioned sample shards, if a sample has a label, they perform an arithmetic sum of the attributes of all the sample shards and divide the sum by the number of shards (the result is a vector, each value of which represents a characteristic attribute, and the value is the average value of the attribute across the shards). Unlabeled sample shards are reserved for the next batch of processing. If a sample does not have a label within a preset time, the sample shard is deleted. In addition, the sample also includes a sample ID, which is a number that requires the ability to identify the sample.

[0047] It should be noted that the above result is one of two vectors obtained by performing arithmetic slicing on all attributes of a sample. However, since the attributes of this sample are provided by multiple providers, each provider provides a slicing, and these slicings need to be arithmetic summed and averaged.

[0048] Regarding the neural network after running, for P2, the neural network run by P2 can select a commonly used neural network model depending on the application scenario, and run it according to the rules of the normal neural network model. The result obtained is the output result of the neural network model.

[0049] Specifically, the model has weight parameters, and arithmetically sharing these weight parameters is model sharding. Model sharding is actually the random initialization of each independently. In order to speed up the convergence of the system, the initial weight parameters of the model can also use the weight parameters of similar existing models, and then perform arithmetic sharding. Gradient refers to the derivative of the error with respect to the model parameters, and arithmetically sharing this value is gradient sharding. Representation refers to the calculation results of each layer of the neural network, and arithmetically sharing these results is representation sharding.

[0050] For the case where there is only one label, during forward propagation, P1 sends the output representation of layer n to P2 at the end of layer n. P2 then adds this slice to its own slice and uses it as the input to the neural network in layer n+1. During backward propagation, the gradient from layer n+1 is sliced ​​and sent to layer n to calculate the gradient of layer n and update the model parameters.

[0051] For the case of multiple label parties, during forward propagation, P1 sends the output Representation slice of layer n to P2 at the end of layer n. P2 adds the slice and its own slice as the input of the n+1 layer neural network.

[0052] In one embodiment, when only one of the data providers provides a tag, the method further includes: the second server is operated by the data provider that provides the tag.

[0053] In one embodiment, when there are multiple data providers providing labels, the method further includes: splitting the model weight parameters of layers mk to m in the initialization phase, specifically:

[0054] In the forward propagation phase, when running to the mk layer, where k = 1 or 2, the second server splits the model at the last layer or two layers of the model. Specifically, the input calculation results of the mk layer are sliced, and then the first server and the second server respectively use their respective calculation results as input to run their respective neural networks; in the backward propagation phase, the first server and the second server respectively calculate the difference between the label slice and the output of their respective neural networks, use the chain rule to calculate the gradient of the difference with respect to each model parameter, and use the gradient to update the model parameters, where m represents the total number of layers of the neural network model;

[0055] In the back propagation stage, when the back propagation reaches the end of the mk layer, the first server sends the gradient slice propagated in this layer to the second server. The second server calculates the arithmetic sum of its own gradient slice and the received gradient slice, and then continues back propagation using the chain rule. When the back propagation reaches the n layer, the gradient of the n layer is arithmetically sliced, and one of the gradient slices is sent to the first server; then the first server and the second server each use the chain rule to calculate the gradient of the model parameters based on their own gradient slices, and use the gradient to update the model parameters.

[0056] Specifically, when there are multiple data providers providing labels (multiple label providers), they all have trust issues with P2. In order to protect the privacy of labels of all parties, the labels need to be sharded. Figure 2As shown in the dashed box, during the forward propagation phase, when the system reaches layer mk, P2 slices the input representation of layer mk (the computational results of each neural network layer) and passes one of the slices to P1. P1 and P2 then run their respective neural networks using their respective representations as input. During the backward propagation phase, P1 calculates the difference between the label slice and the output of P1's neural network, applies the chain rule to calculate the derivative (i.e., gradient) of this difference with respect to P1's model parameters, and uses this gradient to update the model parameters. P2's process is similar. When backpropagation reaches layer mk, P1 sends the gradient slice propagated from that layer to P2. P2 calculates the arithmetic sum of its own gradient slice and the received gradient slice, and then continues backpropagation using the chain rule. When backpropagation reaches layer n, the gradient of layer n needs to be arithmetically sliced, and one of the gradient slices is sent to P1. P1 and P2 then each calculate the gradient of the model parameters using the chain rule based on their own gradient slices and use this to update the model parameters. This completes one round of training.

[0057] It should be noted that model splitting involves performing arithmetic secret sharing on each model parameter: mod_var = mod_var1 + mod_var2, where mod_var1 and mod_var2 are the parameters of the P1 and P2 models, respectively. Representation = representation1 + representation2, where representation1 and representation2 are the input vectors of the P1 and P2 models, respectively. When there are multiple label providers, the high-level neural network is split into two models in the last one or two layers to protect label privacy.

[0058] Example 2

[0059] Based on the same inventive concept, this embodiment discloses an MPC-based vertical federated neural network training device, which is applied to a system including a data provider and two servers. The first n layers of the neural network model are respectively deployed on a first server and a second server in an arithmetic slicing manner, and all layers above the nth layer of the neural network model are deployed on the second server. The training device includes:

[0060] The sample attribute expansion module is used for data providers to expand the attributes of their respective sample data, and to split the attribute-expanded sample data into slices before sending them to the first server and the second server;

[0061] A training data acquisition module is configured to, when the first server and the second server receive a certain number of sample fragments, perform arithmetic summation of the attributes of all fragments of the sample, if a sample has a label, and divide the sum by the number of fragments to obtain a first vector as training data;

[0062] A training module is used for the first server and the second server to run the first n layers of the neural network model for training based on the training data according to a known two-party security protocol. In the forward propagation stage, when the neural network model runs to the n+1 layer, the first server sends the calculation result of its own n-th layer output to the second server, and the second server performs arithmetic addition on the received calculation result and its own calculation result as the input of the n+1 layer, and runs the subsequent neural network; in the reverse propagation stage, when the neural network model runs to the n-th layer, the second server performs arithmetic slicing on the calculated n-layer gradient and sends one of the gradient slices to the first server, and then the first server and the second server each complete the reverse propagation process of the gradient, and n is a positive integer.

[0063] Since the device described in Example 2 of the present invention is used to implement the MPC-based longitudinal federated neural network training method described in Example 1 of the present invention, those skilled in the art will be able to understand the specific structure and variations of the device based on the method described in Example 1 of the present invention, and therefore will not be described in detail here. All devices used in the method described in Example 1 of the present invention fall within the scope of protection of the present invention.

[0064] Example 3

[0065] Based on the same inventive concept, this embodiment further provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the method described in the first embodiment is implemented.

[0066] Since the computer-readable storage medium described in the third embodiment of the present invention is the computer-readable storage medium used to implement the MPC-based vertical federated neural network training method described in the first embodiment of the present invention, those skilled in the art will be able to understand the specific structure and variations of the computer-readable storage medium based on the method described in the first embodiment of the present invention, and therefore will not be described in detail here. All computer-readable storage media used in the method of the first embodiment of the present invention fall within the scope of protection of the present invention.

[0067] Example 4

[0068] This embodiment further provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method described in the first embodiment when executing the program.

[0069] Since the computer device introduced in the fourth embodiment of the present application is the computer device used for implementing the training method of the MPC-based vertical federated neural network in the first embodiment of the present application, the specific structure and deformation of the computer device can be understood by those skilled in the art based on the method introduced in the first embodiment of the present application, and thus will not be described here again. Any computer device used by the method of the first embodiment of the present application belongs to the scope of the present application.

[0070] Embodiment five

[0071] The present embodiment also provides a reasoning method of MPC-based vertical federated neural network, which uses the neural network model obtained by the training method of the first embodiment to perform reasoning calculation.

[0072] The process of reasoning calculation includes:

[0073] Data is provided by each data provider, the neural network model is run according to the forward propagation process, when there is only one data provider providing labels, the output of the neural network model is taken as the reasoning result; when there are multiple data providers providing labels, the output results of the first server and the second server are taken as the reasoning result.

[0074] In general, in terms of security and privacy, P1 of the present application does not obtain additional information compared with the neural network based on two-party secure calculation, so as long as the neural network based on two-party secure calculation is secure, P1 cannot initiate any reasoning attack on the label, including MC attack. For P2, it obtains representation = σ1(W1(σ0(W0(A||B), where A represents the data shard owned by P1, B represents the data shard sent by P2, W0 and W1 are the neural network front 2 layer network weights or convolution kernels owned by P1, and σ0 and σ1 are the corresponding activation functions. Since P2 does not know A, W0 and W1, it cannot obtain the information of the input data A from the representation, nor can it obtain the model parameters of P1. Moreover, P1 and P2 cannot obtain complete model information independently, and still need to cooperate to perform model reasoning.

[0075] In terms of speed, compared with the central model training, the present application only has an additional 1 layer or 2 layers of additional operation amount, if the number of neural network layers is relatively large, the present application will greatly improve the speed and achieve practical efficiency. For the multi-label scene, only 2-4 layers of additional operation amount are added.

[0076] Thanks to data feature engineering, this solution can be applied to a wider range of scenarios and better utilize data. For example, if multiple departments across multiple hospitals jointly model, multiple hospitals can provide labels. Current solutions generally assume that labels are provided by a single party.

[0077] Moreover, due to the centralized data processing of this solution, P1 and P2 can shuffle the data, naturally solving the non-independent and identically distributed problem caused by multi-party data, making the model achieve better accuracy.

[0078] It will be understood by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0079] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0080] Although preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they are aware of the basic inventive concepts. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention. Obviously, those skilled in the art may make various changes and modifications to the embodiments of the present invention without departing from the spirit and scope of the embodiments of the present invention. Thus, the present invention is intended to include such changes and modifications to the embodiments of the present invention if they fall within the scope of the claims and their equivalents.

Claims

1. A training method for a longitudinal federated neural network based on MPC, characterized in that: Applied to a system including a data provider and two servers, the first n layers of a neural network model are respectively deployed on a first server and a second server in an arithmetic sharding manner, and all layers above the nth layer of the neural network model are deployed on the second server. The training method includes: The data providers perform attribute expansion on their respective sample data, split the attribute-expanded sample data into slices, and then send the slices to the first server and the second server; After the first server and the second server receive a certain number of sample shards, if a sample has a label, the attributes of all shards of the sample are arithmetic summed and divided by the number of shards to obtain a first vector as training data; The first server and the second server run the first n layers of the neural network model for training based on the training data according to a known two-party security protocol. In the forward propagation stage, when the neural network model runs to the n+1 layer, the first server sends the calculation result of its own n-th layer output to the second server, and the second server performs arithmetic addition on the received calculation result and its own calculation result as the input of the n+1 layer, and runs the subsequent neural network; in the reverse propagation stage, when the neural network model runs to the n-th layer, the second server performs arithmetic slicing on the calculated n-layer gradient, and sends one of the gradient slices to the first server, and then the first server and the second server each complete the reverse propagation process of the gradient, where n is a positive integer.

2. The MPC-based longitudinal federated neural network training method according to claim 1, characterized in that: When a sample has a label, the method further includes the data provider sharding the label.

3. The MPC-based vertical federated neural network training method according to claim 2, characterized in that: The method further includes: when a sample fragment has no label, the corresponding sample fragment is retained for next batch processing; if a sample fragment has no label within a preset time, the corresponding sample fragment is deleted.

4. The MPC-based longitudinal federated neural network training method according to claim 1, wherein: When only one of the data providers provides a tag, the method further includes: the second server is operated by the data provider that provides the tag.

5. The training method of the MPC-based vertical federated neural network according to claim 1, characterized in that: When there are multiple data providers providing labels, the method further includes: splitting the model weight parameters of layers mk to m in the initialization phase, specifically: In the forward propagation phase, when running to the mk layer, where k = 1 or 2, the second server splits the model's calculation results in the last layer or two layers of the model. Specifically, the input result of the mk layer is sliced, and then the first server and the second server respectively use their respective slices as input to run their respective neural networks; in the backward propagation phase, the first server and the second server respectively calculate the difference between the label slice and the output of their respective neural networks, use the chain rule to calculate the gradient of the difference with respect to each model parameter, and use the gradient to update the model parameters, where m represents the total number of layers of the neural network model; In the back propagation stage, when the back propagation reaches the end of the mk layer, the first server sends the gradient slice propagated in this layer to the second server. The second server calculates the arithmetic sum of its own gradient slice and the received gradient slice, and then continues back propagation using the chain rule. When the back propagation reaches the n layer, the gradient of the n layer is arithmetically sliced, and one of the gradient slices is sent to the first server; then the first server and the second server each use the chain rule to calculate the gradient of the model parameters based on their own gradient slices, and use the gradient to update the model parameters.

6. A longitudinal federated neural network training device based on MPC, characterized in that: Applied to a system including a data provider and two servers, the first n layers of a neural network model are respectively deployed on a first server and a second server in an arithmetic slicing manner, and all layers above the nth layer of the neural network model are deployed on the second server, and the training device includes: The sample attribute expansion module is used for data providers to expand the attributes of their respective sample data, and to split the attribute-expanded sample data into slices before sending them to the first server and the second server; A training data acquisition module is configured to, when the first server and the second server receive a certain number of sample fragments, perform arithmetic summation of the attributes of all fragments of the sample, if a sample has a label, and divide the sum by the number of fragments to obtain a first vector as training data; A training module is used for the first server and the second server to run the first n layers of the neural network model for training based on the training data according to a known two-party security protocol. In the forward propagation stage, when the neural network model runs to the n+1 layer, the first server sends the calculation result of its own n-th layer output to the second server, and the second server performs arithmetic addition on the received calculation result and its own calculation result as the input of the n+1 layer, and runs the subsequent neural network; in the reverse propagation stage, when the neural network model runs to the n-th layer, the second server performs arithmetic slicing on the calculated n-layer gradient and sends one of the gradient slices to the first server, and then the first server and the second server each complete the reverse propagation process of the gradient, and n is a positive integer.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the training method of the MPC-based longitudinal federated neural network according to any one of claims 1 to 5 is implemented.

8. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the training method of the MPC-based longitudinal federated neural network according to any one of claims 1 to 5 is implemented.

9. An MPC-based longitudinal federated neural network reasoning method, characterized in that: The neural network model obtained according to any one of claims 1 to 5 is used to perform inference calculations.

10. The reasoning method of the MPC-based vertical federated neural network according to claim 9, characterized in that: Performing inference calculations using the neural network model obtained according to any one of claims 1 to 5, comprising: Each data provider provides data, and the neural network model is run according to the forward propagation process. When only one data provider provides a label, the output of the neural network model is used as the inference result; when multiple data providers provide labels, the output results of the first server and the second server are arithmetic summed as the inference result.

Images