Training method, device and storage medium of full-anonymity federated learning model

Abstract

This invention provides a training method, device, and storage medium for a fully anonymous federated learning model. The method includes: acquiring a first aligned feature matrix fragment and a first aligned index matrix fragment; inputting the first aligned feature matrix fragment and the first aligned index matrix fragment into a single-party sub-model in an improved MPC XGBoost model to train a fully anonymous single-party sub-model in the fully anonymous federated learning model. In this invention, intersection information exists in fragmented form. In the fully anonymous framework, intermediate data is also fragmented. The aligned feature matrix fragments of each participant include fragmented, densely-state common user information, and the row height of the matrix is ​​minimized. The aligned feature matrix fragments align the common user data of each participant, and in the dense-state calculation, the sum of the common user data remains unchanged, while the sum of the non-common user data is zero. The dense-state fragmentation also prevents participants from identifying the zeroed data of non-intersection users, thereby greatly improving the security of the entire process and preventing the exposure of any data.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a training method, apparatus and storage medium for a fully anonymous federated learning model. Background Technology

[0002] As information technology permeates all aspects of people's lives, more and more data is generated in various scenarios, such as shopping mall purchase records, taxi ride records, and medical records. This data is part of personal privacy, extremely sensitive and valuable. To prevent the leakage of this data, many organizations choose not to open it to the public and only use it internally, thus creating "data silos." However, the development of information technology depends on the reasonable open use of data. Combining data from multiple organizations can assist the development of many industries. For example, combining hospital patient case data with pharmaceutical manufacturers' production data can promote the research and development of new drugs; combining transaction data from multiple banks can assist in joint risk control; and combining insurance company policy data with hospital patient case data can facilitate efficient claims processing, and so on.

[0003] Federated learning is essentially a distributed machine learning framework that enables data sharing and collaborative modeling while ensuring data privacy, security, and legal compliance. The widespread use of federated learning technology can break down the "data silo" problem, allowing technology users to collaborate with other users on modeling (i.e., multi-party modeling) without leaving their data premises, thus jointly creating data value. Multi-party modeling first requires Secure Intersection (PSI). Secure intersection can be simply understood as having N parties, each with users from their own business (patients in a hospital, depositors in a bank, registered users of a mobile application, etc.). Secure intersection technology is used to obtain the common users of the N parties, and after the intersection is completed, no information about non-intersection users is exposed to the other parties. The matching information for the intersection can be unique identification information such as mobile phone numbers or ID cards.

[0004] However, secure intersection of user data sets is unsuitable in some sensitive scenarios. The main reason is that the final result is in plaintext, exposing the users at the intersection. For example, bank A has user set A, and hospital B has user set B. Securely intersecting the data sets A and B yields A∩B=C. Here, data set C is known in plaintext to both parties; that is, bank A knows that depositors in data set C have medical records at hospital B, and hospital B knows that patients in data set C have accounts at bank A. This is unacceptable for some organizations due to poor security and failure to protect user privacy, limiting the use of vertical federated learning in institutions or scenarios with high security requirements. Furthermore, in vertical federated learning, without intersection sharing, traditional algorithm design cannot effectively perform joint modeling. Summary of the Invention

[0005] The technical problem to be solved by the present invention is to overcome the defects of existing technologies that expose the intersection users, resulting in poor security and failure to protect user privacy, and to provide a training method, device and storage medium for a fully anonymous federated learning model.

[0006] The present invention solves the above-mentioned technical problems through the following technical solution:

[0007] This invention provides a training method for a fully anonymous federated learning model, comprising two participants: a first participant and a second participant. The training method is applied to the first participant and includes:

[0008] Obtain the first alignment feature matrix slice and the first alignment index matrix slice;

[0009] The first alignment feature matrix slice and the first alignment index matrix slice are input into the single-party sub-model in the improved MPC (Multi-Party Secure Computation) XGBoost (a federated learning model) model for training to obtain the fully anonymous single-party sub-model in the fully anonymous federated learning model;

[0010] In this process, multiple single-party sub-models are trained jointly. The first aligned feature matrix sharding includes information on common users in the fragmented dense state, and the row height of the matrix is ​​the same as the sample size with the least amount of data among the two participants. The first aligned feature matrix sharding aligns the feature shard data corresponding to common users in the first participant with the feature shard data corresponding to common users in the second participant, and in the dense state calculation, the sum of the feature shard data corresponding to common users remains unchanged, while the sum of the feature shard data corresponding to non-common users is zero.

[0011] Preferably, obtaining the first alignment feature matrix slice and the first alignment index matrix slice includes:

[0012] Obtain the first sample data of the first participant;

[0013] The first sample data is divided into a first feature matrix fragment and a second feature matrix fragment, and the second feature matrix fragment is sent to the second participant among the two participants.

[0014] Receive the third feature matrix fragment sent by the second participant;

[0015] Obtain the first intersection result partition;

[0016] The row heights of the first feature matrix slice and the third feature matrix slice are compared, and the row height with the smallest row height is used as the row height of the aligned feature matrix. The third intermediate feature matrix slice is obtained by multiplication based on the first intersection result slice and the third feature matrix slice through the MPC protocol.

[0017] The first alignment index matrix fragment is obtained by summing the first intersection result fragment by row.

[0018] The first intermediate feature matrix fragment is obtained by dot product based on the first feature matrix fragment and the first alignment index matrix fragment;

[0019] The first aligned feature matrix fragment is obtained by concatenating the first intermediate feature matrix fragment and the third intermediate feature matrix fragment.

[0020] Preferably, obtaining the first intersection result partition includes:

[0021] Obtain a first intersection data set, wherein the first intersection data set includes the user data of the first participant;

[0022] The first intersection data set is divided into a first fragment and a second fragment, wherein the first fragment and the second fragment retain a portion of the information of each data in the first intersection data set;

[0023] The second fragment is sent to the second participant, and the third fragment sent by the second participant is received, wherein the third fragment is a fragment of the second intersection data set, and the second intersection data set includes the user data of the second participant;

[0024] The first intersection result fragment is obtained by comparing the first fragment and the third fragment through the MPC protocol. The first intersection result fragment indicates the intersection user of the first participant and the second participant in the form of fragment information.

[0025] Preferably, the step of obtaining the first intersection result fragment based on the comparison of the first fragment and the third fragment using the MPC protocol includes:

[0026] The first intersection result segment is obtained by comparing the values ​​at each position in the first segment and the third segment pairwise to see if they are equal; wherein, the first segment is a matrix;

[0027] If the values ​​are equal, the corresponding position of the first intersection result piece is set to one; if the values ​​are not equal, the corresponding position of the first intersection result piece is set to zero.

[0028] Preferably, during training, the single-sided sub-model performs the following steps:

[0029] Obtain the random seed, the first predicted value slice, and the first label slice;

[0030] Execute iterations to build the tree until the iteration stopping condition is met.

[0031] Preferably, during the iteration of the tree construction, the single-component sub-model performs the following steps:

[0032] The first sampled feature matrix fragment is obtained by sampling the first aligned feature matrix fragment by row.

[0033] Based on the first predicted value slice and the first tag pagination, the first initial first derivative slice and the first initial second derivative slice are obtained through dense state calculation using the MPC protocol.

[0034] The first final first derivative piece and the first final second derivative piece are obtained by dot product of the first initial first derivative piece, the first initial second derivative piece, and the first aligned feature matrix piece through the MPC protocol.

[0035] Based on the first sampling feature matrix, the first maximum value transpose fragment and the first minimum value transpose fragment are obtained through dense state calculation using the MPC protocol;

[0036] Obtain the first auxiliary calculation matrix slice;

[0037] Based on the first maximum value transpose partition, the first minimum value transpose partition, and the first auxiliary calculation matrix partition, B buckets are divided, and the boundary of each bucket is determined by the dense state calculation of the MPC protocol to obtain the first bucket boundary partition.

[0038] Traverse each column of feature data in the first sampling feature matrix slice, extract column feature slices of each column of feature data, perform range comparison of MPC protocol based on the column feature slices and the first bucket boundary slices, and perform B2A (a matrix transformation method, i.e. Boolean to numerical, True to 1, False to 0) transformation to generate feature bucket matrix slices.

[0039] Concatenate all feature bucket matrices to obtain the final feature bucket sparse matrix fragment;

[0040] The first histogram slice is obtained by matrix multiplication using the MPC protocol based on the first final first derivative slice, the first final second derivative slice, and the final feature bucket sparse matrix slice.

[0041] Based on the first final first derivative slice and the first final second derivative slice, the nodes that have reached the stopping split condition are assigned values ​​through the dense state calculation of the MPC protocol, and the first node value slice of the node that has stopped splitting is obtained.

[0042] Update the tree structure;

[0043] Use the updated tree to predict the original data, and update the first predicted value shard.

[0044] Preferably, when the first participant is the labeler, the first sample data includes the y label;

[0045] The process of obtaining the random seed, the first predicted value slice, and the first label slice includes:

[0046] Generate the random seed, initialize the prediction value, and divide the prediction value and the y label into slices to obtain a first prediction value slice, a second prediction value slice, a first label slice, and a second label slice;

[0047] The random seed, the second predicted value fragment, and the second label fragment are sent to the second participant;

[0048] The step of obtaining the first auxiliary calculation matrix fragment includes:

[0049] Construct an auxiliary calculation matrix, and divide the auxiliary calculation matrix into a first auxiliary calculation matrix fragment and a second auxiliary calculation matrix fragment. Send the second auxiliary calculation matrix fragment to the second participant.

[0050] Before the step of assigning values ​​to nodes that meet the stopping splitting condition through dense state calculation using the MPC protocol based on the first final first-order derivative piece and the first final second-order derivative piece, the single-sided sub-model also performs the following steps:

[0051] The first histogram fragment is sent to the second participant, and the second histogram fragment sent by the second participant is received; wherein the second histogram fragment is a histogram fragment calculated by the second participant;

[0052] The plaintext histogram is obtained based on the first histogram fragmentation and the second histogram fragmentation;

[0053] For the plaintext of the histogram, the optimal segmentation point is calculated according to the optimal segmentation formula to obtain the optimal segmentation point of the node to be split;

[0054] Prior to the step of updating the tree structure, the single-component sub-model also performs the following steps:

[0055] The information of the optimal split point is sent to the second participant;

[0056] Following the step of updating the tree structure, the single-sided sub-model further performs the following steps:

[0057] The node information of the next level is sent to the second participant.

[0058] Preferably, when the first participant is a non-labeling participant, the step of obtaining the random seed, the first predicted value slice, and the first label slice includes:

[0059] The system receives the random seed, the first predicted value fragment, and the first label fragment sent by the second participant; wherein the first predicted value fragment is any one of a plurality of predicted value fragments obtained by the second participant by fragmenting the initialized predicted value, and the first label fragment is any one of a plurality of label fragments obtained by the second participant by fragmenting the y label;

[0060] The step of obtaining the first auxiliary calculation matrix fragment includes:

[0061] Receive the first auxiliary calculation matrix fragment sent by the second participant; wherein, the first auxiliary calculation matrix fragment is any one of multiple auxiliary calculation matrix fragments obtained by the second participant by fragmenting the constructed auxiliary calculation matrix;

[0062] Before the step of assigning values ​​to nodes that meet the stopping splitting condition through dense state calculation using the MPC protocol based on the first final first-order derivative piece and the first final second-order derivative piece, the single-sided sub-model also performs the following steps:

[0063] The first histogram is fragmented and sent to the second participant;

[0064] Prior to the step of updating the tree structure, the single-component sub-model also performs the following steps:

[0065] Receive the optimal split point information sent by the second participant;

[0066] Following the step of updating the tree structure, the single-sided sub-model further performs the following steps:

[0067] Receive the next-level node information sent by the second participant.

[0068] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the aforementioned training method for the fully anonymous federated learning model.

[0069] The present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the aforementioned training method for a fully anonymous federated learning model.

[0070] The positive and progressive effects of this invention are as follows: Based on MPC secret sharing, the intersection information of each participant exists in the form of fragments. When performing numerical calculations in a fully anonymous framework, the intermediate data is also calculated in fragment form. The alignment feature matrix fragment of each participant includes the information of common users in the fragmented state, and the row height of the matrix is ​​the same as the sample size with the least amount of data among the participants. The alignment feature matrix fragment of each participant aligns the feature fragment data corresponding to common users in the participant (i.e., the first participant) with the feature fragment data corresponding to common users in other participants (i.e., the second participant). In the state-encrypted calculation, the sum of the feature fragment data corresponding to common users remains unchanged, while the sum of the feature fragment data corresponding to non-common users is zero. The state-encrypted fragmentation also prevents the participants from identifying the zeroed data of non-intersecting users, thereby greatly improving the security of the entire process and preventing the exposure of any data. Attached Figure Description

[0071] Figure 1 This is a flowchart of the training method for the fully anonymous federated learning model in Embodiment 1 of the present invention.

[0072] Figure 2A This is a flowchart of a specific implementation of step S11 in the training method of the fully anonymous federated learning model of Embodiment 1 of the present invention.

[0073] Figure 2B This is a flowchart illustrating an example of obtaining aligned feature matrix slices and aligned index matrix slices in the training method of the fully anonymous federated learning model of Embodiment 1 of the present invention.

[0074] Figure 3A This is a flowchart of a specific implementation of step S114 in the training method of the fully anonymous federated learning model of Embodiment 1 of the present invention.

[0075] Figure 3B This is a flowchart illustrating a fully anonymous secure intersection example in the training method of the fully anonymous federated learning model according to Embodiment 1 of the present invention.

[0076] Figure 4 This is a flowchart illustrating a specific implementation of the training method for the fully anonymous federated learning model according to Embodiment 1 of the present invention.

[0077] Figure 5A This is a flowchart of a specific implementation of step S22 in the training method of the fully anonymous federated learning model of Embodiment 1 of the present invention.

[0078] Figure 5B1 The upper part of the flowchart is a training example in the training method of the fully anonymous federated learning model of Embodiment 1 of the present invention.

[0079] Figure 5B2The lower half of the flowchart is a training example in the training method of the fully anonymous federated learning model of Embodiment 1 of the present invention.

[0080] Figure 6 This is a flowchart of a specific implementation of step S21 in the training method of the fully anonymous federated learning model of Embodiment 1 of the present invention.

[0081] Figure 7 This is a flowchart of a specific implementation of step S2205 in the training method of the fully anonymous federated learning model of Embodiment 1 of the present invention.

[0082] Figure 8 This is a flowchart illustrating a specific implementation of the method for calculating the optimal split point in the training method of the fully anonymous federated learning model according to Embodiment 1 of the present invention.

[0083] Figure 9 This is a schematic diagram of the electronic device according to Embodiment 2 of the present invention. Detailed Implementation

[0084] The present invention will be further illustrated by way of embodiments below, but the present invention is not limited to the scope of the embodiments described herein.

[0085] It should be noted that in the description of this invention, the terms "first" and "second" are used for descriptive purposes only to distinguish data of the same type, and should not be construed as indicating or implying relative importance.

[0086] Example 1

[0087] This embodiment provides a training method for a fully anonymous federated learning model, with two participants: a first participant and a second participant. The training method is applied to the first participant, as described above. Figure 1 Training methods include:

[0088] S11. Obtain the first alignment feature matrix slice and the first alignment index matrix slice.

[0089] S12. Input the first aligned feature matrix slice and the first aligned index matrix slice into the single-component model in the improved MPC XGBoost model for training to obtain the fully anonymous single-component model in the fully anonymous federated learning model.

[0090] In this process, multiple single-party sub-models are trained jointly. The first aligned feature matrix sharding includes information about common users in the fragmented dense state, and the row height of the matrix is ​​the same as the sample size with the least data in the two participating parties. The first aligned feature matrix sharding aligns the feature shard data corresponding to common users in the first participating party with the feature shard data corresponding to common users in the second participating party, and in the dense state calculation, it ensures that the sum of the feature shard data corresponding to common users remains unchanged, while the sum of the feature shard data corresponding to non-common users is zero.

[0091] For example, the two participants are Guest and Host, and Guest has an aligned feature matrix slice. <cf1>And an aligned index matrix partition<s_c1> The host has an aligned feature matrix partition. <cf2>And an aligned index matrix partition<s_c2> .

[0092] For the Guest, the first participant is the Guest, the second participant is the Host, and the first aligned feature matrix slice is... <cf1>The first alignment index matrix partition is<s_c1> For the Host, the first participant is the Host, the second participant is the Guest, and the first aligned feature matrix slice is... <cf2>The first alignment index matrix partition is<s_c2> .

[0093] Alignment feature matrix slices <cf1>and <cf2>This includes information about shared users in the fragmented, dense state, and the row height of the matrix is ​​the same as the minimum sample size in both the Guest and Host sections. Alignment index matrix sharding.<s_c1> and<s_c2> This method aligns the feature matrices of the Guest and Host users, ensuring that the sum of feature slices for users with common data remains unchanged, while the sum of feature slices for users with non-common data is zero. This dense-state sharding also prevents participants from identifying the zeroed-out data of non-intersecting users. The row height of the matrix is ​​designed to ensure that user data is not exposed during training. The entire training process involves calculating the sharded states; neither party knows which data actually contributes to the training, and the user data of non-intersecting users that did not contribute are... <cf1>and <cf2>All data will be set to zero. Setting the data of non-intersecting users to zero will not affect model training. The reason why the zeroed data of non-intersecting users will not be exposed is that the fragmented data of intersecting users may be -6 on the Guest side and 6 on the Host side. The sum of the two is zero, so it is impossible to distinguish it from the data of non-intersecting users.

[0094] This embodiment is based on MPC secret sharing. The intersection information of each participant exists in the form of fragments. When performing numerical calculations in the fully anonymous framework, the intermediate data is also calculated in fragment form. The alignment feature matrix fragment of each participant includes the information of common users in the fragmented state, and the row height of the matrix is ​​the same as the sample size with the least amount of data among the participants. The alignment feature matrix fragment of each participant aligns the feature fragment data corresponding to common users in the participant (i.e., the first participant) with the feature fragment data corresponding to common users in other participants (i.e., the second participant). In the state-encrypted calculation, the sum of the feature fragment data corresponding to common users remains unchanged, and the sum of the feature fragment data corresponding to non-common users is zero. The state-encrypted fragmentation also prevents the participants from identifying the zeroed data of non-intersection users, thereby greatly improving the security of the entire process and preventing the exposure of any data.

[0095] In specific implementation, refer to Figure 2A Step S11 includes:

[0096] S111, Obtain the first sample data of the first participant.

[0097] S112. The first sample data is divided into first feature matrix fragments and second feature matrix fragments, and the second feature matrix fragment is sent to the second participant among the two participants.

[0098] S113, Receive the third feature matrix fragment sent by the second participant.

[0099] S114. Obtain the first intersection result fragment.

[0100] S115. Compare the row heights of the first feature matrix slice and the third feature matrix slice, and use the smallest row height as the row height of the aligned feature matrix. Based on the first intersection result slice and the third feature matrix slice, obtain the third intermediate feature matrix slice through multiplication using the MPC protocol.

[0101] S116. Sum the first intersection result slice by row to obtain the first aligned index matrix slice.

[0102] S117. Based on the first feature matrix slice and the first alignment index matrix slice, the first intermediate feature matrix slice is obtained by dot product. Among them, the unaligned data of the first participant is removed by dot product.

[0103] S118. The first intermediate feature matrix slice and the third intermediate feature matrix slice are concatenated to obtain the first aligned feature matrix slice.

[0104] in, Figure 2B This illustrates how the Guest and Host interact to obtain their respective aligned feature matrix slices. <cf1>and <cf2>and aligning index matrix sharding<s_c1> and<s_c2> The flowchart.

[0105] In the example above, the Guest's sample data is FA, where FA is the feature matrix with a shape of (m, r), where m is the number of Guest samples and r is the number of Guest features. The Guest's feature matrix is ​​divided into slices. <fa1>and <fa2>The intersection result of Guest is divided into fragments. <c1>The intermediate feature matrix of Guest is divided into pieces. <cfa1>and <cfb1>The alignment index matrix of Guest is partitioned as follows:<s_c1> The alignment feature matrix of Guest is divided into pieces. <cf1>The host's sample data is FB, where FB is the feature matrix with a shape of (n,t), where n is the number of samples in the host and t is the number of features in the host. The host's feature matrix is ​​divided into slices. <fb1>and <fb2>The intersection result of the Host is divided into partitions. <c2>The intermediate feature matrix of the Host is divided into segments. <cfa2>and <cfb2>The Host's alignment index matrix is ​​partitioned as follows:<s_c2> The alignment feature matrix of the Host is divided into pieces. <cf2>.

[0106] For the Guest, the first participant is the Guest, the second participant is the Host, the first sample data is FA, and the first feature matrix segment is... <fa1>The second feature matrix is ​​partitioned. <fa2>The third feature matrix is ​​segmented as follows: <fb1>The first intersection result is a partition. <c1>The first alignment index matrix partition is<s_c1> The first intermediate feature matrix is ​​partitioned as follows: <cfa1>The second intermediate feature matrix is ​​partitioned as follows: <cfa2>The third intermediate feature matrix is ​​segmented as follows: <cfb1>The fourth intermediate feature matrix partitioning is <cfb2>The first aligned feature matrix slice is <cf1>.

[0107] For the Host, the first participant is the Host, the second participant is the Guest, the first sample data is FB, and the first feature matrix shard is... <fb1>The second feature matrix is ​​partitioned. <fb2>The third feature matrix is ​​segmented as follows: <fa1>The first intersection result is a partition. <c2>The first alignment index matrix partition is<s_c2> The first intermediate feature matrix is ​​partitioned as follows: <cfb1>The second intermediate feature matrix is ​​partitioned as follows: <cfb2>The third intermediate feature matrix is ​​segmented as follows: <cfa1>The fourth intermediate feature matrix partitioning is <cfa2>The first aligned feature matrix slice is <cf2>.

[0108] In the diagram, the single-arrow dashed line represents the transmission of fragmented data, and the double-arrow dashed line represents the transmission of intermediate computation data in the operator computation of the MPC protocol (such as multiplication and dot product). The operator computation of the MPC protocol is existing technology and will not be described in detail here.

[0109] This example assumes that the number of samples in the Guest section is less than the number of samples in the Host section. If the number of aligned samples in the Host section is much smaller than that in the Guest section, the alignment index matrix can be recalculated by summing the columns and then partitioned, which can reduce the size of the aggregated feature matrix.

[0110] In specific implementation, refer to Figure 3A Step S114 includes:

[0111] S1141. Obtain the first intersection data set, wherein the first intersection data set includes the user data of the first participant.

[0112] S1142. The first intersection data set is divided into a first partition and a second partition, wherein the first partition and the second partition retain a portion of the information of each data in the first intersection data set.

[0113] S1143. Send the second fragment to the second participant and receive the third fragment sent by the second participant, wherein the third fragment is a fragment of the second intersection data set, and the second intersection data set includes the user data of the second participant.

[0114] S1144. Based on the comparison of the first fragment and the third fragment through the MPC protocol, the first intersection result fragment is obtained, wherein the first intersection result fragment indicates the intersection user of the first participant and the second participant in the form of fragment information.

[0115] in, Figure 3B The flowchart for finding the intersection of Guest and Host in a fully anonymous and secure manner is shown.

[0116] In the example above, the intersection dataset of Guest is A, and A is split into two parts. <a1>and <a2>The intersection result is divided into pieces. <c1>.

[0117] The host's intersection dataset is B. After partitioning B, two partitions are obtained. <b1>and <b2>The intersection result is divided into pieces. <c2>.

[0118] The methods for determining the data required by each Guest and Host (such as the first intersection dataset) are similar to those described above, and will not be repeated here.

[0119] In the diagram, the single-arrow dashed line represents the transmission of fragmented data, and the double-arrow dashed line represents the transmission of intermediate computation data in the operator computation of the MPC protocol (such as multiplication and dot product). The operator computation of the MPC protocol is existing technology and will not be described in detail here.

[0120] In specific implementation, step S1144 includes:

[0121] The first intersection result segment is obtained by comparing the values ​​at each position in the first and third segments pairwise to determine if they are equal. The first segment is a matrix.

[0122] If the values ​​are equal, the corresponding position of the first intersection result piece is set to one. If the values ​​are not equal, the corresponding position of the first intersection result piece is set to zero.

[0123] In specific implementation, refer to Figure 4 During training, the single-sided sub-model performs the following steps:

[0124] S21. Obtain the random seed, the first predicted value slice, and the first label slice.

[0125] S22. Execute the iteration of building the tree until the iteration stopping condition is met.

[0126] The random seed is used to align the data of the two participants when creating random samples later.

[0127] In specific implementation, refer to Figure 5A During the iteration of tree construction, the single-sided sub-model performs the following steps:

[0128] S2201. The first sampled feature matrix fragment is obtained by sampling the first aligned feature matrix fragment by row.

[0129] S2202. Based on the first predicted value segmentation and the first tag pagination, the first initial first derivative segmentation and the first initial second derivative segmentation are obtained through dense state calculation using the MPC protocol.

[0130] S2203. Based on the first initial first-order derivative slice, the first initial second-order derivative slice, and the first aligned feature matrix slice, the first final first-order derivative slice and the first final second-order derivative slice are obtained by dot product through the MPC protocol.

[0131] S2204. Based on the first sampling feature matrix, the first maximum value transpose fragment and the first minimum value transpose fragment are obtained through the dense state calculation of the MPC protocol.

[0132] S2205. Obtain the first auxiliary calculation matrix slice.

[0133] S2206. Based on the first maximum value transpose partition, the first minimum value transpose partition, and the first auxiliary calculation matrix partition, B buckets are divided, and the boundary of each bucket is determined by the dense state calculation of the MPC protocol to obtain the first bucket boundary partition.

[0134] S2207. Traverse each column of feature data in the first sampled feature matrix slice, extract column feature slices of each column of feature data, perform range comparison of MPC protocol based on column feature slices and first bucket boundary slices, and perform B2A transformation on the matrix content to generate feature bucket matrix slices.

[0135] S2208. Concatenate all feature bucket matrices to obtain the final feature bucket sparse matrix fragment.

[0136] S2209. The first histogram slice is obtained by matrix multiplication using the MPC protocol based on the first final first derivative slice, the first final second derivative slice, and the final feature bucket sparse matrix slice.

[0137] S2210. Based on the first final first-order derivative slice and the first final second-order derivative slice, the node value slice of the node that has reached the stopping split condition is assigned a value through the dense state calculation of the MPC protocol, and the first node value slice of the node that has stopped splitting is obtained.

[0138] S2211, Update the tree structure.

[0139] S2212. Use the updated tree to predict the original data and update the first predicted value shard.

[0140] Among them, sampling by row for aligning feature matrix slices is sample sampling, and sampling by column is feature sampling. The shape of the matrix is ​​changed through sampling.

[0141] Plaintext updates of predicted values ​​may allow the inference of overlapping content, exposing the overlapping samples. Therefore, in fully anonymous scenarios, predicted value sharding is used to avoid exposing overlapping samples. Label sharding is used to obtain first-order and second-order gradient shards through numerical computation using the MPC protocol.

[0142] Figure 5B1 and 5B2 The flowchart shows an example of Guest and Host jointly training a fully anonymous federated learning model. Figure 5B1 This is the upper part of the flowchart. Figure 5B2 This is the lower half of the flowchart, with the thick dashed line marking the boundary between the upper and lower halves. In the diagram, single-arrow dashed lines represent the transmission of fragmented data, while double-arrow dashed lines represent the transmission of intermediate computation data during operator computations (such as multiplication and dot product) in the MPC protocol. The operator computations in the MPC protocol are existing technology and will not be elaborated upon here.

[0143] Guest's predicted values ​​are sharded as follows <p1>Tag segmentation is <y1>The sampling feature matrix is ​​divided into pieces.<cf_sam1> The initial first derivative is piecewise.<g1′> The initial second derivative is piecewise. The final first derivative piecewise is<h1> <g1>And the final second derivative piecewise is <h1>The maximum value transpose is<f_max1> The minimum value transpose is<f_min1> The auxiliary calculation matrix is ​​partitioned into pieces.<B_range1> The bucket boundaries are divided into segments.<bin_split1> Feature segmentation is<cf_i1> The feature bucket matrix is ​​divided into segments.<bin_i1> The final feature bucket sparse matrix is ​​divided into pieces. <b1>Histogram segmentation<h_histo1> Node value sharding is <w1i>.

[0144] Host's predicted value sharding is <p2>Tag segmentation is <y2>The sampling feature matrix is ​​divided into pieces.<cf_sam2> The initial first derivative is piecewise.<g2′> The initial second derivative is piecewise. The final first derivative piecewise is <g2>And the final second derivative piecewise is <h2>The maximum value transpose is<f_max2> The minimum value transpose is<f_min2> The auxiliary calculation matrix is ​​partitioned into pieces.<B_range2> The bucket boundaries are divided into segments.<bin_split2> Feature segmentation is<cf_i2> The feature bucket matrix is ​​divided into segments.<bin_i2> The final feature bucket sparse matrix is ​​divided into pieces. <b2>Histogram segmentation<h_histo2> Node value sharding is <w2i>.

[0145] The methods for determining the data required by each Guest and Host (such as the first tag fragment) are similar to those described above, and will not be repeated here.

[0146] The calculation formula used in the example is as follows:

[0147]

[0148]

[0149]

[0150]

[0151]

[0152]

[0153]

[0154]

[0155]

[0156] The formulas above all represent the MPC dense state calculation process. This refers to MPC multiplication (multiplication in the MPC protocol). For MPC division (division in the MPC protocol), For MPC addition (addition in the MPC protocol), This is for MPC subtraction (subtraction in the MPC protocol). I represents the set of samples under the stopping split node, i represents the sequence number of the stopping split node, j represents each sample in set I, and e is a constant.

[0157] The histogram bucket boundaries of the partitioned states are constructed using maximum value transpose partitioning and minimum value transpose partitioning. Additionally, to construct the histogram bucket boundaries of the partitioned states, an auxiliary calculation matrix partitioning is also needed, with the content [0,1,2,3...,B], where B is the number of buckets.

[0158] The Boolean result for each feature value is calculated sequentially within different boundary ranges. This process also involves MPC comparisons. Finally, the Boolean result matrix is ​​converted into an arithmetic result.<bin_i> The shape is (m, B), where m is the total number of samples from the Guest side and B is the number of buckets.<bin_i> The original plaintext value of the record is a sparse 0 / 1 matrix.

[0159] All features<bin_i> The matrix is ​​concatenated to obtain the final feature bucket sparse matrix partitioning. The shape is (m, r*B+t*B), where m is the total number of samples in the Guest side, B is the number of buckets, r is the number of features in the Guest side, and t is the number of features in the Host side.

[0160] Gain represents gain calculation, G R G represents the first-order gradient of the split right-hand sample. L H represents the first-order gradient of the split left sample. R H represents the second-order gradient on the right side of the split and H. L The left-hand gradient of the split is represented by λ and γ, where λ and γ are calculated coefficients to prevent division by zero. The calculation of Gain is a prior art technique and will not be elaborated here.

[0161] In practice, when the first participant is the labeler, the first sample data includes the y label.

[0162] Reference Figure 6 Step S21 includes:

[0163] S211. Generate a random seed, initialize the prediction value, and divide the prediction value and y label into slices to obtain the first prediction value slice, the second prediction value slice, the first label slice, and the second label slice.

[0164] S212, Send the random seed, the second predicted value fragment, and the second label fragment to the second participant.

[0165] Reference Figure 7 Step S2205 includes:

[0166] S22051. Construct an auxiliary calculation matrix, and divide the auxiliary calculation matrix into a first auxiliary calculation matrix fragment and a second auxiliary calculation matrix fragment. Send the second auxiliary calculation matrix fragment to the second participant.

[0167] Reference Figure 8 Before step S2210, the unilateral sub-model also performs the following steps:

[0168] S31. Send the first histogram fragment to the second participant and receive the second histogram fragment sent by the second participant. The second histogram fragment is the histogram fragment calculated by the second participant.

[0169] S32. Obtain the plaintext histogram based on the first histogram slice and the second histogram slice.

[0170] S33. For plaintext histograms, calculate the optimal segmentation point according to the optimal segmentation formula to obtain the optimal segmentation point of the node to be split.

[0171] In this process, the plaintext histogram is obtained by partitioning the histograms of both parties, which in turn allows us to calculate the optimal split point for the node to be split and further construct the tree.

[0172] Prior to step S2211, the unilateral sub-model also performs the following steps:

[0173] The information of the optimal split point is sent to the second participant. This optimal split point information is the node splitting information.

[0174] After step S2211, the unilateral sub-model also performs the following steps:

[0175] Send the node information of the next level to the second participant.

[0176] The tag provider needs to provide y-tag data.

[0177] In this embodiment, based on MPC secret sharing, the tagger needs to provide y-label data. The intersection information of each participant exists in the form of fragments. When performing numerical calculations in the fully anonymous framework, the intermediate data is also calculated in fragment form. The alignment feature matrix fragment of each participant includes the information of common users in the fragmented state, and the row height of the matrix is ​​the same as the sample size with the least amount of data among the participants. The alignment feature matrix fragment of each participant aligns the feature fragment data corresponding to common users in this participant (i.e., the first participant) with the feature fragment data corresponding to common users in other participants (i.e., the second participant). In the state-encrypted calculation, the sum of the feature fragment data corresponding to common users remains unchanged, and the sum of the feature fragment data corresponding to non-common users is zero. The state-encrypted fragmentation also avoids the participants from identifying the zeroed data of non-intersecting users, thereby greatly improving the security of the entire process and not exposing any data.

[0178] In practice, when the first participant is a non-labeling party, refer to... Figure 6 Step S21 includes:

[0179] S213. Receive the random seed, the first predicted value fragment, and the first label fragment sent by the second participant. The first predicted value fragment is any one of multiple predicted value fragments obtained by the second participant by fragmenting the initialized predicted value, and the first label fragment is any one of multiple label fragments obtained by the second participant by fragmenting the y label.

[0180] Reference Figure 7 Step S2205 includes:

[0181] S22052. Receive the first auxiliary computation matrix fragment sent by the second participant. The first auxiliary computation matrix fragment is any one of multiple auxiliary computation matrix fragments obtained by the second participant by fragmenting the constructed auxiliary computation matrix.

[0182] Prior to step S2210, the unilateral sub-model also performs the following steps:

[0183] The first histogram fragment is sent to the second participant.

[0184] Prior to step S2211, the unilateral sub-model also performs the following steps:

[0185] Receive the optimal split point information sent by the second participant. The optimal split point information is the node splitting information.

[0186] After step S2211, the unilateral sub-model also performs the following steps:

[0187] Receive the next-level node information sent by the second participant.

[0188] Non-labeled parties do not need to provide y-label data.

[0189] In this embodiment, based on MPC secret sharing, the tagger needs to provide y-tag data and share tag fragments with other participants. The intersection information of each participant exists in the form of fragments. When performing numerical calculations in the fully anonymous framework, the intermediate data is also calculated in fragment form. The alignment feature matrix fragment of each participant includes the information of common users in the fragmented state, and the row height of the matrix is ​​the same as the sample size with the least amount of data among the participants. The alignment feature matrix fragment of each participant aligns the feature fragment data corresponding to common users in this participant (i.e., the first participant) with the feature fragment data corresponding to common users in other participants (i.e., the second participant). In the state-encrypted calculation, the sum of the feature fragment data corresponding to common users remains unchanged, and the sum of the feature fragment data corresponding to non-common users is zero. The state-encrypted fragmentation also avoids the participants from identifying the zeroed data of non-intersecting users, thereby greatly improving the security of the entire process and not exposing any data.

[0190] Example 2

[0191] Figure 9 This is a schematic diagram of an electronic device provided in Embodiment 2 of the present invention. The electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the training method of the fully anonymous federated learning model in Embodiment 1. Figure 9 The electronic device 30 shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of the present invention.

[0192] The electronic device 30 may be in the form of a general-purpose computing device, such as a server device. The components of the electronic device 30 may include, but are not limited to: at least one processor 31, at least one memory 32, and a bus 33 connecting different system components (including memory 32 and processor 31).

[0193] Bus 33 includes a data bus, an address bus, and a control bus.

[0194] The memory 32 may include volatile memory, such as random access memory (RAM) 321 and / or cache memory 322, and may further include read-only memory (ROM) 323.

[0195] The memory 32 may also include a program / utility 325 having a set (at least one) of program modules 324, including but not limited to: an operating system, one or more application programs, other program modules, and program data, each or some combination of these examples may include an implementation of a network environment.

[0196] The processor 31 executes various functional applications and data processing by running computer programs stored in the memory 32, such as the training method of the fully anonymous federated learning model in Embodiment 1 of the present invention.

[0197] Electronic device 30 can also communicate with one or more external devices 34 (e.g., buttons, pointing devices, etc.). This communication can be performed via input / output (I / O) interface 35. Furthermore, the model-generated electronic device 30 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 36. As shown in the figure, network adapter 36 communicates with other modules of the model-generated electronic device 30 via bus 33. It should be understood that, although not shown in the figure, other hardware and / or software modules can be used in conjunction with the model-generated electronic device 30, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID (disk array) systems, tape drives, and data backup storage systems.

[0198] It should be noted that although several modules / modules or sub-modules / modules of the electronic device have been mentioned in the detailed description above, this division is merely exemplary and not mandatory. In fact, according to embodiments of the present invention, the features and functions of two or more modules / modules described above can be embodied in one module / module; conversely, the features and functions of one module / module described above can be further divided into multiple modules / modules for embodiment.

[0199] Example 3

[0200] This embodiment provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the training method for the fully anonymous federated learning model in Embodiment 1.

[0201] The readable storage medium may be more specifically adopted, including but not limited to: portable disk, hard disk, random access memory, read-only memory, erasable programmable read-only memory, optical storage device, magnetic storage device, or any suitable combination thereof.

[0202] In a possible implementation, the present invention can also be implemented as a program product comprising program code, which, when the program product is run on a terminal device, is used to cause the terminal device to execute a training method for implementing the fully anonymous federated learning model in Embodiment 1.

[0203] The program code for executing the present invention can be written in any combination of one or more programming languages. The program code can be executed entirely on the user device, partially on the user device, as a standalone software package, partially on the user device and partially on a remote device, or entirely on a remote device.

[0204] While specific embodiments of the present invention have been described above, those skilled in the art should understand that these are merely illustrative examples, and the scope of protection of the present invention is defined by the appended claims. Those skilled in the art can make various changes or modifications to these embodiments without departing from the principles and essence of the present invention, but all such changes and modifications fall within the scope of protection of the present invention. < / h2> < / h1>

Claims

1. A training method for a fully anonymous federated learning model, characterized in that, The two participants include a first participant and a second participant. The training method is applied to the first participant, and the training method includes: Obtain the first alignment feature matrix slice and the first alignment index matrix slice; The first alignment feature matrix slice and the first alignment index matrix slice are input into the single-component sub-model in the improved MPC XGBoost model for training to obtain the fully anonymous single-component model in the fully anonymous federated learning model. In this process, multiple single-party sub-models are trained jointly. The first aligned feature matrix sharding includes information on common users in the fragmented dense state, and the row height of the matrix is ​​the same as the sample size with the least amount of data among the two participants. The first aligned feature matrix sharding aligns the feature shard data corresponding to common users in the first participant with the feature shard data corresponding to common users in the second participant, and in the dense state calculation, the sum of the feature shard data corresponding to common users remains unchanged, while the sum of the feature shard data corresponding to non-common users is zero. The step of obtaining the first alignment feature matrix slice and the first alignment index matrix slice includes: Obtain the first sample data of the first participant; The first sample data is divided into a first feature matrix fragment and a second feature matrix fragment, and the second feature matrix fragment is sent to the second participant among the two participants. Receive the third feature matrix fragment sent by the second participant; Obtain the first intersection result partition; The row heights of the first feature matrix slice and the third feature matrix slice are compared, and the row height with the smallest row height is used as the row height of the aligned feature matrix. Based on the first intersection result slice and the third feature matrix slice, the third intermediate feature matrix slice is obtained by multiplication using the MPC protocol. The first alignment index matrix fragment is obtained by summing the first intersection result fragment by row. The first intermediate feature matrix fragment is obtained by dot product of the first feature matrix fragment and the first alignment index matrix fragment. The first aligned feature matrix fragment is obtained by concatenating the first intermediate feature matrix fragment and the third intermediate feature matrix fragment.

2. The training method of the full-anonymity federated learning model according to claim 1, wherein, The process of obtaining the first intersection result fragment includes: Obtain a first intersection data set, wherein the first intersection data set includes the user data of the first participant; The first intersection data set is divided into a first fragment and a second fragment, wherein the first fragment and the second fragment retain a portion of the information of each data in the first intersection data set; The second fragment is sent to the second participant, and the third fragment sent by the second participant is received, wherein the third fragment is a fragment of the second intersection data set, and the second intersection data set includes the user data of the second participant; The first intersection result fragment is obtained by comparing the first fragment and the third fragment through the MPC protocol. The first intersection result fragment indicates the intersection user of the first participant and the second participant in the form of fragment information.

3. The training method of the full-anonymity federated learning model according to claim 2, wherein, The process of obtaining the first intersection result fragment based on the comparison of the first fragment and the third fragment using the MPC protocol includes: The first intersection result segment is obtained by comparing the values ​​at each position in the first segment and the third segment pairwise to see if they are equal; wherein, the first segment is a matrix; If the values ​​are equal, the corresponding position of the first intersection result piece is set to one; if the values ​​are not equal, the corresponding position of the first intersection result piece is set to zero.

4. The training method of a full-anonymity federated learning model according to claim 1, wherein, During training, the single-sided sub-model performs the following steps: Obtain the random seed, the first predicted value slice, and the first label slice; Execute iterations to build the tree until the iteration stopping condition is met.

5. The training method of a full-anonymity federated learning model according to claim 4, wherein, In the iteration of the tree construction, the single-component sub-model performs the following steps: The first sampled feature matrix fragment is obtained by sampling the first aligned feature matrix fragment by row. Based on the first predicted value slice and the first tag slice, the first initial first derivative slice and the first initial second derivative slice are obtained through the dense state calculation of the MPC protocol; The first final first derivative piece and the first final second derivative piece are obtained by dot product of the first initial first derivative piece, the first initial second derivative piece, and the first aligned feature matrix piece through the MPC protocol. Based on the first sampling feature matrix, the first maximum value transpose fragment and the first minimum value transpose fragment are obtained through dense state calculation using the MPC protocol; Obtain the first auxiliary calculation matrix slice; Based on the first maximum value transpose partition, the first minimum value transpose partition, and the first auxiliary calculation matrix partition, B buckets are divided, and the boundary of each bucket is determined by the dense state calculation of the MPC protocol to obtain the first bucket boundary partition. Traverse each column of feature data in the first sampling feature matrix slice, extract column feature slices of each column of feature data, perform range comparison of MPC protocol based on the column feature slices and the first bucket boundary slices, and perform B2A transformation on the matrix content to generate feature bucket matrix slices; Concatenate all feature bucket matrices to obtain the final feature bucket sparse matrix fragment; The first histogram slice is obtained by matrix multiplication using the MPC protocol based on the first final first derivative slice, the first final second derivative slice, and the final feature bucket sparse matrix slice. Based on the first final first derivative slice and the first final second derivative slice, the nodes that have reached the stopping split condition are assigned values ​​through the dense state calculation of the MPC protocol, and the first node value slice of the node that has stopped splitting is obtained. Update the tree structure; Use the updated tree to predict the original data, and update the first predicted value shard.

6. The training method of a full-anonymity federated learning model according to claim 5, wherein, When the first participant is the labeler, the first sample data includes the y label; The process of obtaining the random seed, the first predicted value slice, and the first label slice includes: Generate the random seed, initialize the prediction value, and divide the prediction value and the y label into slices to obtain a first prediction value slice, a second prediction value slice, a first label slice, and a second label slice; The random seed, the second predicted value fragment, and the second label fragment are sent to the second participant; The step of obtaining the first auxiliary calculation matrix fragment includes: Construct an auxiliary calculation matrix, and divide the auxiliary calculation matrix into a first auxiliary calculation matrix fragment and a second auxiliary calculation matrix fragment. Send the second auxiliary calculation matrix fragment to the second participant. Before the step of assigning values ​​to nodes that meet the stopping splitting condition through dense state calculation using the MPC protocol based on the first final first-order derivative piece and the first final second-order derivative piece, the single-sided sub-model also performs the following steps: The first histogram fragment is sent to the second participant, and the second histogram fragment sent by the second participant is received; wherein the second histogram fragment is a histogram fragment calculated by the second participant; The plaintext histogram is obtained based on the first histogram fragmentation and the second histogram fragmentation; For the plaintext of the histogram, the optimal segmentation point is calculated according to the optimal segmentation formula to obtain the optimal segmentation point of the node to be split; Prior to the step of updating the tree structure, the single-component sub-model also performs the following steps: The information of the optimal split point is sent to the second participant; Following the step of updating the tree structure, the single-sided sub-model further performs the following steps: The node information of the next level is sent to the second participant.

7. The training method for the fully anonymous federated learning model as described in claim 5, characterized in that, When the first participant is a non-labeled party, obtaining the random seed, the first predicted value slice, and the first label slice includes: The system receives the random seed, the first predicted value fragment, and the first label fragment sent by the second participant; wherein the first predicted value fragment is any one of a plurality of predicted value fragments obtained by the second participant by fragmenting the initialized predicted value, and the first label fragment is any one of a plurality of label fragments obtained by the second participant by fragmenting the y label; The step of obtaining the first auxiliary calculation matrix fragment includes: Receive the first auxiliary calculation matrix fragment sent by the second participant; wherein, the first auxiliary calculation matrix fragment is any one of multiple auxiliary calculation matrix fragments obtained by the second participant by fragmenting the constructed auxiliary calculation matrix; Before the step of assigning values ​​to nodes that meet the stopping splitting condition through dense state calculation using the MPC protocol based on the first final first-order derivative piece and the first final second-order derivative piece, the single-sided sub-model also performs the following steps: The first histogram is fragmented and sent to the second participant; Prior to the step of updating the tree structure, the single-component sub-model also performs the following steps: Receive the optimal split point information sent by the second participant; Following the step of updating the tree structure, the single-sided sub-model further performs the following steps: Receive the next-level node information sent by the second participant.

8. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes a computer program, it implements the training method of the fully anonymous federated learning model according to any one of claims 1 to 7.

9. A computer readable storage medium having stored thereon a computer program, characterized in that, When the computer program is executed by the processor, it implements the training method of the fully anonymous federated learning model according to any one of claims 1 to 7.

Images