Federal learning training and data processing method, device, equipment, medium and program

Abstract

The embodiment discloses a federated learning training and data processing method, device, equipment, medium and program wherein the federated learning training method comprises: obtaining a filter of a first node and a first position index sent by a second node, the first position index representing a position not mapped by sample data in the filter of the second node, the filter is used for representing the position where the sample data is inserted and the position where the sample data is not inserted through an array; filtering sample data corresponding to the first position index in a filter of the first node to obtain an updated filter of the first node; obtaining a first sample data set corresponding to the filter after the first node is updated; obtaining a second sample data set corresponding to the filter after the second node is updated; and jointly training a second sub-model and a third sub-model based on the first sample data set and the second sample data set to obtain a first model, the first model being composed of the second sub-model deployed by the first node and the third sub-model deployed by the second node.

Description

Technical Field

[0001] The present application belongs to the field of federated learning technology, and in particular, relates to a method, apparatus, device, medium and program for federated learning training and data processing. Background Art

[0002] In the process of federated learning, the two nodes participating in the training first use Private Set Intersection (PSI) to obtain the user ID in the intersection by the first node and the second node, and then extract the sample data corresponding to the user ID in the intersection according to the user ID to perform model training. When a node does not want to disclose its local user ID to another node, the traditional PSI intersection method cannot meet the needs of obtaining the intersection data between different nodes. Summary of the invention

[0003] The embodiments of the present application provide a method, apparatus, device, medium and program for federated learning training and data processing.

[0004] The present application embodiment provides a federated learning training method, which is applied to a first node, and the method includes:

[0005] Obtaining a filter of the first node and a first position index sent by the second node, wherein the first position index represents a position in the filter of the second node that is not mapped by sample data, and the filter is used to represent a position where sample data is inserted and a position where sample data is not inserted through an array;

[0006] Filtering out the sample data corresponding to the first position index from the filter of the first node to obtain an updated filter of the first node;

[0007] Obtaining a first sample data set corresponding to the updated filter of the first node;

[0008] Sending the second position index of the first node to the second node, so that the filter of the second node filters out the sample data corresponding to the second position index, and obtains the updated filter of the second node; obtaining the second sample data set corresponding to the updated filter of the second node; the second position index represents the position in the filter of the first node that is not mapped by the sample data;

[0009] The second sub-model and the third sub-model are jointly trained based on the first sample data set and the second sample data set to obtain a first model, wherein the first model is composed of the second sub-model deployed on the first node and the third sub-model deployed on the second node.

[0010] In some embodiments, obtaining the filter of the first node includes: initializing the filter; obtaining a sample data set of the first node; processing each user identifier in the sample data set of the first node to obtain a third position index of each user identifier in the filter; and inserting the sample data corresponding to the user identifier into the filter according to the third position index.

[0011] It can be seen that, through the method provided in this embodiment, the position index of the filter can be updated through the user identifier, the updated filter can be obtained, and the sample data can be inserted into the filter.

[0012] In some embodiments, the third position index of each user identifier in the filter is composed of a binary array of length N, wherein at least one element of the binary array is 1, and N is an integer greater than 2; inserting the sample data corresponding to the user identifier into the filter according to the third position index includes: dividing the filter into N buckets; arbitrarily selecting a bucket with an element of 1 in the third position index, and inserting the sample data corresponding to each user identifier into the filter; dividing the sample data set of the first node into the N buckets of the filter, wherein some of the N buckets have empty data.

[0013] It can be seen that based on the method provided in this embodiment, it is possible to implement bucketing of sample data, which is beneficial to improving sample data query efficiency and optimizing sample data management.

[0014] In some embodiments, obtaining the first sample data set corresponding to the updated filter of the first node includes: receiving a first position index sent by the second node, the first position index indicating the bucket number of the N buckets in the filter of the second node that has no sample data; the filter of the first node eliminates the sample data of the corresponding bucket number according to the bucket number involved in the first position index to obtain the remaining sample data, and the remaining sample data serves as the first sample data set of the first node.

[0015] It can be seen that the first position index in this embodiment indicates the bucket number without sample data. Through the method provided in this embodiment, the same bucket number in two nodes can be obtained, and the sample data set with the same bucket number can be obtained.

[0016] In some embodiments, the joint training of the second sub-model based on the first sample data set and the second sample data set includes: agreeing with the second node on model training parameters and bucket numbers selected in each round of model iteration; determining, based on the number of iterations of the first model, sample data corresponding to the bucket number corresponding to the number of iterations; and training the second sub-model based on the sample data corresponding to the bucket number corresponding to the number of iterations, in combination with the sample data corresponding to the bucket number corresponding to the second node.

[0017] It can be seen that since the data in the buckets of different nodes are randomly inserted, the data in the corresponding buckets is determined by the number of iterations, and the second sub-model is trained. This is beneficial to improving the diversity of data participating in model training in the two nodes under the same number of iterations, and further helps to improve the accuracy of model training.

[0018] In some embodiments, the second sub-model is trained based on the sample data corresponding to the bucket number corresponding to the iteration number, combined with the sample data corresponding to the bucket number corresponding to the second node, including: receiving the second model gradient corresponding to the third sub-model corresponding to the iteration number sent by the second node; obtaining the first model gradient corresponding to the second sub-model corresponding to the iteration number; and updating the parameters involved in the second sub-model based on the first model gradient and the second model gradient.

[0019] It can be seen that updating the parameters of the second sub-model through the first model gradient and the second model gradient is beneficial to improving the training accuracy of the second sub-model.

[0020] In some embodiments, the method also includes: generating a first public key, a first private key corresponding to the first public key, and a first random number; sending the first public key to the second node, and receiving the second public key sent by the second node; the receiving the second model gradient corresponding to the third sub-model corresponding to the number of iterations sent by the second node includes: receiving a second collection, the second collection is the second node using the first public key to encrypt the second model gradient to obtain the encrypted second model gradient, and then using the user identifier corresponding to the sample data for generating the second model gradient as a key to encrypt the first random number, and the encrypted second model gradient is encrypted twice, the first random number is the random number encrypted by the first node using the second public key; the updating of the parameters involved in the second sub-model based on the first model gradient and the second model gradient includes: based on any one user identifier in the bucket corresponding to the number of iterations, encrypting the first random number in the second set; The method comprises the steps of: decrypting the first random number and the second model gradient encrypted by the first public key, and obtaining the plaintext of the first random number in the second set by using the first private key corresponding to the first public key to obtain the second model gradient; extracting the user identifier of the successfully decrypted random number, the corresponding first model gradient, and the second model gradient; merging the first model gradient and the second model gradient, and updating the parameters of the second sub-model based on the merged model gradient; encrypting the second random number and the updated model gradient sent by the second node by using the user identifier of the successfully decrypted random number, and sending them to the second node, so that the second node decrypts by using any user identifier in the bucket corresponding to the number of iterations and the second private key corresponding to the second public key. When the decryption is successful, the updated model gradient is obtained, and the third sub-model is updated by the updated model gradient. The second random number is the random number encrypted by the second node with the first public key.

[0021] It can be seen that by adopting the symmetric encryption and asymmetric encryption methods to encrypt the data exchanged between the first node and the second node, it is beneficial to protect the security of the exchanged data.

[0022] The embodiment of the present application also provides a federated learning training method, which is applied to a second node, and the method includes:

[0023] Acquire the filter of the second node and the second position index sent by the first node, wherein the second position index represents a position in the filter of the first node that is not mapped by the sample data, and the filter is used to represent the position where the sample data is inserted and the position where the sample data is not inserted through an array;

[0024] Filter out the sample data corresponding to the second position index from the filter of the second node to obtain an updated filter of the second node;

[0025] Obtain a second sample data set corresponding to the updated filter of the second node;

[0026] Sending the first position index of the second node to the first node, so that the filter of the first node filters out the sample data corresponding to the first position index, and obtains the updated filter of the first node; obtaining the first sample data set corresponding to the updated filter of the first node; the first position index represents the position in the filter of the second node that is not mapped by the sample data;

[0027] The second sub-model and the third sub-model are jointly trained based on the first sample data set and the second sample data set to obtain a first model, wherein the first model is composed of the second sub-model deployed on the first node and the third sub-model deployed on the second node.

[0028] In some embodiments, obtaining the filter of the second node includes: initializing the filter; obtaining a sample data set of the second node; processing each user identifier in the sample data set of the second node to obtain a fourth position index of each user identifier in the filter; and inserting the sample data corresponding to the user identifier into the filter according to the fourth position index.

[0029] It can be seen that, through the method provided in this embodiment, the position index of the filter can be updated through the user identifier to obtain the updated filter of the second node.

[0030] In some embodiments, the fourth position index of each user identifier in the filter is composed of a binary array of length N, wherein at least one element of the binary array is 1, and N is an integer greater than 2; inserting the sample data corresponding to the user identifier into the filter according to the fourth position index includes: dividing the filter into N buckets; arbitrarily selecting a bucket with an element of 1 in the fourth position index, and inserting the sample data corresponding to each user identifier into the filter; dividing the sample data set of the second node into the N buckets of the filter, wherein some of the N buckets have empty data.

[0031] It can be seen that, based on the method provided in this embodiment, bucketing the sample data is beneficial to improving the query efficiency of the sample data and optimizing the management of the sample data.

[0032] In some embodiments, obtaining the second sample data set corresponding to the updated filter of the second node includes: receiving a second position index sent by the first node, the second position index indicating the bucket number of the N buckets in the filter of the first node that has no sample data; the filter of the second node eliminates the sample data of the corresponding bucket number according to the bucket number involved in the second position index to obtain the remaining sample data, and the remaining sample data serves as the second sample data set of the second node.

[0033] It can be seen that the second position index in this embodiment indicates the bucket number without sample data. Through the method provided in this embodiment, the same bucket number in two nodes can be obtained, and the sample data set with the same bucket number can be obtained.

[0034] In some embodiments, the joint training of the third sub-model based on the first sample data set and the second sample data set includes: agreeing with the first node on model training parameters and bucket numbers selected in each round of model iteration; determining, based on the number of iterations of the first model, sample data corresponding to the bucket number corresponding to the number of iterations; and training the third sub-model based on the sample data corresponding to the bucket number corresponding to the number of iterations, in combination with the sample data corresponding to the bucket number corresponding to the first node.

[0035] It can be seen that since the data in the buckets of different nodes are randomly inserted, the data in the corresponding buckets is determined by the number of iterations, and the third sub-model is trained. This is beneficial to improving the diversity of data participating in model training in the two nodes under the same number of iterations, and further helps to improve the model training accuracy.

[0036] In some embodiments, the third sub-model is trained based on the sample data corresponding to the bucket number corresponding to the iteration number, combined with the sample data corresponding to the bucket number corresponding to the first node, including: receiving the first model gradient corresponding to the second sub-model corresponding to the iteration number sent by the first node; obtaining the second model gradient corresponding to the third sub-model corresponding to the iteration number; and updating the parameters involved in the third sub-model based on the first model gradient and the second model gradient.

[0037] It can be seen that updating the parameters of the third sub-model through the first model gradient and the second model gradient is conducive to improving the training accuracy of the third sub-model.

[0038] In some embodiments, the method also includes: generating a second public key, a second private key corresponding to the second public key, and a second random number; sending the second public key to the first node, and receiving the first public key sent by the first node; the receiving of the first model gradient corresponding to the second sub-model corresponding to the number of iterations sent by the first node includes: receiving a first collection, the first collection is the first node using the second public key to encrypt the first model gradient to obtain the encrypted first model gradient, and then using the user identifier corresponding to the sample data for generating the first model gradient as a key to encrypt the second random number, and the encrypted first model gradient is obtained by double-encrypting, the second random number is the random number encrypted by the second node using the first public key; the updating of the parameters involved in the third sub-model based on the first model gradient and the second model gradient includes: based on any one user identifier in the bucket corresponding to the number of iterations, encrypting the second random number in the first set. The method comprises the following steps: decrypting the first model gradient encrypted by the first public key and the second random number, and decrypting the plaintext of the second random number in the first set by the second private key corresponding to the second public key to obtain the first model gradient; extracting the user identifier of the successfully decrypted random number, the corresponding first model gradient, and the second model gradient; merging the first model gradient and the second model gradient, and updating the parameters of the third sub-model based on the merged model gradient; encrypting the first random number and the updated model gradient sent by the first node by using the user identifier of the successfully decrypted random number, and sending them to the first node, so that the first node decrypts them by using any user identifier in the bucket corresponding to the number of iterations and the first private key corresponding to the first public key. When the decryption is successful, the updated model gradient is obtained, and the second sub-model is updated by the updated model gradient. The first random number is the random number encrypted by the first node with the second public key.

[0039] It can be seen that by adopting the symmetric encryption and asymmetric encryption methods to encrypt the data exchanged between the first node and the second node, it is beneficial to protect the security of the exchanged data.

[0040] The present application also provides a data processing method, which includes:

[0041] Get the data to be processed;

[0042] The data to be processed is processed using the trained first model, where the first model is obtained by performing federated learning training on the second sub-model and the third sub-model based on the above method.

[0043] The embodiment of the present application further provides a federated learning training device, which is applied to a first node, and the device includes:

[0044] A first processing module is configured to obtain a filter of the first node and a first position index sent by the second node, wherein the first position index indicates a position in the filter of the second node that is not mapped by sample data, and the filter is configured to represent positions where sample data is inserted and positions where sample data is not inserted through an array; the sample data corresponding to the first position index is filtered out from the filter of the first node to obtain an updated filter of the first node;

[0045] The second processing module is used to obtain a first sample data set corresponding to the updated filter of the first node; send the second position index of the first node to the second node, so that the filter of the second node filters out the sample data corresponding to the second position index, and obtain the updated filter of the second node; obtain a second sample data set corresponding to the updated filter of the second node; the second position index represents a position in the filter of the first node that is not mapped by the sample data;

[0046] The first training module is used to jointly train the second sub-model and the third sub-model based on the first sample data set and the second sample data set to obtain a first model, wherein the first model is composed of the second sub-model deployed on the first node and the third sub-model deployed on the second node.

[0047] The embodiment of the present application also provides a federated learning training device, which is applied to a second node, and the device includes:

[0048] A third processing module is used to obtain the filter of the second node and the second position index sent by the first node, wherein the second position index represents the position in the filter of the first node that is not mapped by the sample data, and the filter is used to represent the position where the sample data is inserted and the position where the sample data is not inserted through an array; the sample data corresponding to the second position index is filtered out from the filter of the second node to obtain an updated filter of the second node;

[0049] The fourth processing module is used to obtain a second sample data set corresponding to the updated filter of the second node; send the first position index of the second node to the first node, so that the filter of the first node filters out the sample data corresponding to the first position index, and obtain the updated filter of the first node; obtain the first sample data set corresponding to the updated filter of the first node; the first position index represents the position in the filter of the second node that is not mapped by the sample data;

[0050] The second training module is used to jointly train the second sub-model and the third sub-model based on the first sample data set and the second sample data set to obtain a first model, wherein the first model is composed of the second sub-model deployed on the first node and the third sub-model deployed on the second node.

[0051] The present application also provides a data processing device, the device comprising:

[0052] An acquisition module is used to acquire data to be processed;

[0053] The fifth processing module is used to process the data to be processed using the trained first model, where the first model is obtained by performing federated learning training on the second sub-model and the third sub-model based on the above method.

[0054] An embodiment of the present application provides an electronic device, the electronic device comprising a processor and a memory for storing a computer program that can be run on the processor; wherein:

[0055] The processor is used to run the computer program to execute any one of the above-mentioned federated learning training methods implemented by the first node, or the federated learning training method implemented by the second node, or the data processing method.

[0056] An embodiment of the present application provides a computer storage medium having a computer program stored thereon. When the computer program is executed by a processor, the computer program implements any one of the above-mentioned federated learning training methods implemented by the first node, or the federated learning training method implemented by the second node, or the data processing method.

[0057] An embodiment of the present application provides a computer program product, including a computer program, which, when executed by a processor, implements any of the above-mentioned federated learning training methods implemented by the first node, or the federated learning training method implemented by the second node, or the data processing method.

[0058] The embodiments of the present application provide a federated learning training and data processing method, apparatus, device, medium and program. Through the federated learning training method provided in the embodiments of the present application, it is possible to determine the intersecting data between different nodes through the elements corresponding to the position index, without the need for user identification between the interacting nodes, thereby improving the security of user information. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 A flow chart of a federated learning training method provided in an embodiment of the present application;

[0060] Figure 2 A flowchart of another federated learning training method provided in an embodiment of the present application;

[0061] Figure 3 A flow chart of a data processing method provided in an embodiment of the present application;

[0062] Figure 4 A schematic diagram of the structure of a federated learning training device provided in an embodiment of the present application;

[0063] Figure 5 A schematic diagram of the structure of another federated learning training device provided in an embodiment of the present application;

[0064] Figure 6 A schematic diagram of the structure of a data processing device provided in an embodiment of the present application;

[0065] Figure 7 An interactive schematic diagram of a model training system provided in an embodiment of the present application;

[0066] Figure 8 A flowchart of another federated learning training method provided in an embodiment of the present application;

[0067] Fig. 9 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0068] Traditional federated learning methods are mainly based on the privacy intersection of two nodes, using the sample data of the intersection for model training, and updating the local models corresponding to different nodes with interactive batch encrypted gradients until the model is stable. In the federated learning process, the two nodes participating in the training first use PSI to intersect, and the first and second nodes obtain the user ID in the intersection, and then extract the data corresponding to the user ID in the intersection based on the user ID to train the model. When a node does not want to disclose its local user ID to another node, the traditional PSI intersection method cannot meet the demand for obtaining intersection data. For example, when a bank and an operator jointly make a high-value user identification model, the bank has the label corresponding to the user, while the operator only has the corresponding features of the user. Under the premise that the bank does not want the operator to know which users it has, the method based on PSI intersection cannot complete the training of the joint model between the bank and the operator.

[0069] In order to obtain the intersection of data in different nodes without exchanging user identifiers, the embodiments of the present application provide a method, device, equipment, medium and program for federated learning training and data processing. The federated learning training method provided in the embodiments of the present application can exchange the positions in the local filters of different nodes that are not mapped by sample data, and update the elements corresponding to the position index of the filter to screen the local sample data of different nodes, obtain data for federated learning training, and complete the model training of federated learning without exchanging user identifiers.

[0070] The following is a further detailed description of the embodiments of the present application in conjunction with the accompanying drawings and examples. It should be understood that the embodiments provided herein are only used to explain the embodiments of the present application and are not intended to limit the embodiments of the present application. In addition, the embodiments provided below are partial embodiments for implementing the present application, rather than providing all embodiments for implementing the present application. In the absence of conflict, the technical solutions recorded in the embodiments of the present application can be implemented in any combination.

[0071] It should be noted that, in the embodiments of the present application, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a method or device including a series of elements includes not only the elements explicitly recorded, but also includes other elements not explicitly listed, or also includes elements inherent to the implementation of the method or device. In the absence of further restrictions, an element defined by the sentence "includes..." does not exclude the presence of other related elements (such as steps in the method or units in the device, such as a unit in the device may be a part of a circuit, a part of a processor, a part of a program or software, etc.) in the method or device including the element.

[0072] The federated learning training and data processing method provided in the embodiment of the present application includes a series of steps, but the federated learning training and data processing method provided in the embodiment of the present application is not limited to the recorded steps. Similarly, the federated learning training and data processing device provided in the embodiment of the present application includes a series of modules, but the device provided in the embodiment of the present application is not limited to including the modules explicitly recorded, and may also include modules required to obtain relevant information or perform processing based on information.

[0073] This application embodiment provides a federated learning training method, such as Figure 1 As shown, Figure 1 A flow chart of a federated learning training method is shown, applied to a first node, Figure 1 The federated learning training method shown includes:

[0074] Step 101: Obtain the filter of the first node and the first position index sent by the second node, the first position index represents the position in the filter of the second node that is not mapped by the sample data, and the filter is used to represent the position where the sample data is inserted and the position where the sample data is not inserted through an array.

[0075] The first node and the second node are two different nodes in the federated learning training. For ease of explanation, the number of nodes of the federated learning involved in this application is taken as two as an example. In practical applications, when there are more than two nodes in the federated learning training, the federated learning training can also be performed based on the method given in the embodiment of the present application. Both the first node and the second node include models that can be trained independently. For example, the first node can represent a bank, and the second node can represent an operator.

[0076] Assuming that the first node is a bank, the sample data set in the first node can be represented as S1, and S1 can be (ID1, X1, Y1), where ID1 represents the user identification in the sample data set of the first node, and ID1 can specifically be a user's mobile phone number, a user identifier (Identification, ID), etc., X1 represents the feature field corresponding to the sample data set of the first node, such as age, consumption amount, etc., and Y1 is the business label corresponding to the sample data set of the first node. When it comes to anti-fraud business, the business label can be marked as whether it is a blacklist, etc. The sample data set in the second node can be represented as S2, and S2 can be (ID2, X2), where ID2 represents the user identification in the sample data set of the second node, and ID2 can specifically be a user's mobile phone number, a user ID, etc., and X2 is the feature field corresponding to ID2 in the sample data set of the second node, such as age, consumption amount, etc.

[0077] In an embodiment of the present application, filters are provided in both the first node and the second node for implementing storage of sample data in the nodes. The position index of the filter in the first node and the position index of the filter in the second node can be represented by an array or by a specific identifier. When the position index of the filter is represented by an array, different numerical values ​​in the array represent positions where sample data is inserted and positions where sample data is not inserted. For example, the array corresponding to the filter can be an array represented by binary data, such as an array consisting of element 1 and element 0, where element 1 is defined to represent the position where sample data is inserted, and element 0 is defined to represent the position where sample data is not inserted.

[0078] In order to accurately obtain the intersection of sample data in the first node and the second node, the filter of the first node and the filter of the second node may be initialized first so that the length of the position index of the first node filter is the same as the length of the position index of the second node filter.

[0079] Step 102: Filter out the sample data corresponding to the first position index from the filter of the first node to obtain an updated filter of the first node.

[0080] Since the index of the filter is composed of an array, the elements in the array all have corresponding positions. In one example, the position index of the filter D1 of the first node can be [1,1,1,1,0,0]. Corresponding to D1, the position index of the filter D2 of the second node is [1,1,1,0,1,1]. At this time, the second node can send the 4th position (i.e., the first position index) in [1,1,1,0,1,1] to the first node. When the first node obtains the 4th position in [1,1,1,0,1,1], it can filter out the data at the 4th position to obtain the updated filter of the first node; the position index of the updated filter D1 of the first node becomes [1,1,1,0,0,0].

[0081] Step 103: Obtain a first sample data set corresponding to the updated filter of the first node.

[0082] After obtaining the updated filter through the above steps, the first sample data set corresponding to the updated filter of the first node can be obtained. That is, the sample data corresponding to element 1 in the position index of the updated filter of the first node is the first sample data set.

[0083] Step 104: Send the second position index of the first node to the second node, so that the filter of the second node filters out the sample data corresponding to the second position index to obtain the updated filter of the second node; obtain the second sample data set corresponding to the updated filter of the second node; the second position index represents the position in the filter of the first node that is not mapped by the sample data.

[0084] Similar to step 101 and step 102, the first node will also send the second position index in the filter of the first node that is not mapped by the sample data to the second node, so that the filter of the second node filters out the sample data corresponding to the position index that is the same as the second position index, obtains the updated filter of the second node, and obtains the second sample data set corresponding to the updated filter of the second node.

[0085] Step 105: jointly train the second sub-model and the third sub-model based on the first sample data set and the second sample data set to obtain a first model, wherein the first model is composed of the second sub-model deployed on the first node and the third sub-model deployed on the second node.

[0086] The second sub-model is deployed in the first node, and the third sub-model is deployed in the second node. The first model can be obtained by performing federated learning training on the second sub-model and the third sub-model.

[0087] The first node trains its local second sub-model based on the first sample data set. At the same time, the second node trains its local third sub-model based on the second sample data set. Based on the trained second sub-model and third sub-model, the joint training of the first model is completed.

[0088] An embodiment of the present application provides a federated learning training method. The first node and the second node can filter out data that are not intersections in the first node and the second node by exchanging positions in the local filter that are not mapped by the sample data, thereby downgrading the computational complexity of processing the sample data set and not requiring user identifiers in the sample data set corresponding to the exchange node, thereby improving the security of user data in the local node.

[0089] In practical applications, steps 101 to 105 can be implemented based on a processor, and the processor can be at least one of an application specific integrated circuit (ASIC), a digital signal processor (DSP), a digital signal processing device (DSPD), a programmable logic device (PLD), a field programmable gate array (FPGA), a central processing unit (CPU), a controller, a microcontroller, and a microprocessor.

[0090] In some embodiments, the above-mentioned acquisition of the filter of the first node includes: initializing the filter; acquiring a sample data set of the first node; processing each user identifier in the sample data set of the first node to obtain a third position index of each user identifier in the filter; and inserting the sample data corresponding to the user identifier into the filter according to the third position index.

[0091] In this embodiment, the sample data set in the first node represents all the sample data included in the first node. This embodiment provides a method for constructing a filter. First, the filter is initialized in the first node. The position index of the initial filter is composed of elements representing positions that are not mapped by sample data. Specifically, the position index of the initial filter can be composed of 0 in a binary array, wherein the position of 0 in the position index of the initial filter represents a position that is not mapped by sample data. The length of the position index of the initial filter can be set to N.

[0092] After constructing the initial filter, each user identifier in the sample data set of the first node is processed to obtain the third position index corresponding to each user identifier, wherein the number of elements in the third position index is the same as the number of elements in the position index of the initial filter. For example, when the elements in the position index of the initial filter are set to be composed of a binary array, the elements in the third position index are also composed of a binary array. In practical applications, the user identifier can be a user ID or a user's mobile phone number, etc. After obtaining the user identifier, the binary array corresponding to the user identifier can be obtained by substituting the user identifier into the relevant g functions for calculation, where the binary array corresponding to the user identifier is consistent with the length of the third position index.

[0093] After calculating the third position index corresponding to the user identifier, randomly select a position index where element 1 is located in the third position index and insert it into the bucket number of the filter as the corresponding sample data.

[0094] For example, assume that the sample data set S1 of the first node includes three sample data, namely (mobile phone number 12, network fee 34, label 0), (mobile phone number 129, network fee 349, label 0), (mobile phone number 1221, network fee 34, label 1), and the position index of the initialized filter of the first node is [0,0,0,0,0,0]. Substitute the user identifiers (i.e., mobile phone numbers) of the above three sample data into three hash functions respectively to obtain the third position index. The third position indexes corresponding to the user identifiers of the three sample data are [1,1,1,0,0,0], [1,1,0,1,0,0] and [1,1,0,0,0,0]. For the first sample data of the above three sample data, a position (for example, the first position) can be randomly selected from the first position to the third position in the third position index [1,1,1,0,0,0], and the first sample data can be inserted into the bucket of the filter of the first node.

[0095] In some embodiments, the third position index of each user identifier in the filter is composed of a binary array of length N, where at least one element of the binary array is 1, and N is an integer greater than 2; the above-mentioned inserting the sample data corresponding to the user identifier into the filter according to the third position index includes: dividing the filter into N buckets; arbitrarily selecting a bucket with an element of 1 in the third position index, and inserting the sample data corresponding to each user identifier into the filter; dividing the sample data set of the first node into the N buckets of the filter, and some of the N buckets have empty data.

[0096] According to the method given in the above embodiment, after the filter D1 is obtained, the filter D1 is divided into N buckets according to the length N of the position index of D1. It can be seen that the corresponding elements in the N buckets are 1 or 0. A bucket with an element of 1 in the third position index is selected, and the sample data corresponding to each user identifier is randomly inserted into the filter.

[0097] After the sample data is inserted into the filter, according to the multiple buckets divided by the filter D1, the sample data in the sample data set in the first node is divided into N buckets corresponding to the filter D1, thereby implementing bucketing of the data.

[0098] For example, when the third position indexes corresponding to the user identifiers of the three sample data are [1,1,1,0,0,0], [1,1,0,1,0,0] and [1,1,0,0,0,0] respectively, the three sample data can be respectively inserted into the buckets of the filter of the first node with reference to the above-mentioned contents, thereby completing the data bucketing. Since the sample data is randomly inserted, there may be a situation where the data in some buckets is empty.

[0099] Similarly, the second node can also bucket the sample data of the second node through the method provided in this embodiment.

[0100] In some embodiments, the above-mentioned acquisition of the first sample data set corresponding to the updated filter of the first node includes: receiving a first position index sent by the second node, the first position index indicating the bucket number without sample data among the N buckets in the filter of the second node; the filter of the first node eliminates the sample data corresponding to the bucket number involved in the first position index to obtain the remaining sample data, and the remaining sample data serves as the first sample data set of the first node.

[0101] Based on the method given in the above embodiment, the first node and the second node can locally complete the bucketing of local sample data. Therefore, corresponding to the method given in the above steps 101 to 103, this embodiment provides a method for obtaining the first sample data set after data bucketing.

[0102] After completing the bucketing of the local sample data set of the node, the second node sends the first position index to the first node. At this time, the first position index can also represent the bucket number without sample data in the N buckets in the second node filter. Specifically, the first position index can be the bucket number of the element corresponding to the bucket in the second node is 0.

[0103] After receiving the first position index, the first node matches the first position index with the bucket number in the first node, removes the sample data in the bucket number corresponding to the first position index, and uses the remaining sample data as the first sample data set of the first node.

[0104] For example, the sample data sets S1 are (mobile phone number 12, internet fee 34, label 0), (mobile phone number 129, internet fee 349, label 0) and (mobile phone number 1221, internet fee 34, label 1). After inserting each sample data in the sample data set S1 into the bucket, the first position index can be received. For example, in the position index of D1 [1,1,1,1,0,0], the first position index received by the first node is: the 4th position in the position index of D2 [1,1,1,0,1,1]. At this time, the first node can delete the data in bucket number 4 after bucketing to obtain the first sample data set S11. For example, the first sample data set S11 is (mobile phone number 12, internet fee 34, label 0), (mobile phone number 1221, internet fee 34, label 1).

[0105] Similarly, the first node will also send the second position index to the second node, and the second node can obtain the bucket number of the N buckets of the first node that does not have sample data through the second position index. Correspondingly, the second node removes the sample data in the bucket number corresponding to the second position index in the second node according to the second position index, and uses the remaining sample data after removal as the second sample data set.

[0106] In some embodiments, the above-mentioned joint training of the second sub-model based on the first sample data set and the second sample data set includes: agreeing with the second node on model training parameters and the bucket number selected in each round of model iteration; determining the sample data corresponding to the bucket number corresponding to the number of iterations based on the number of iterations of the first model; and training the second sub-model based on the sample data corresponding to the bucket number corresponding to the number of iterations, in combination with the sample data corresponding to the bucket number corresponding to the second node.

[0107] Before conducting federated learning training, the first node and the second node each initialize a local model. The first node and the second node agree to set model parameters that may include one or more of the following: learning rate, number of iterations, stochastic gradient descent algorithm, parameters of the initial model, etc., and based on the number of model iterations, the first node and the second node jointly agree on the bucket number corresponding to the number of model iterations, wherein the bucket number corresponding to the number of model iterations is randomly generated. In order to ensure the diversity of model training data, the bucket number corresponding to each iteration can also be set to be different. The value range of the bucket number corresponding to each round of iteration is the bucket number corresponding to the updated filter corresponding to the first node, and the bucket number corresponding to the updated filter corresponding to the second node.

[0108] When training the first model, the sample data corresponding to the bucket number corresponding to the agreed number of iterations can be determined by the number of iterations of the first model. The first node trains the second sub-model based on the sample data corresponding to the local bucket number corresponding to the number of iterations, and the second node trains the third sub-model based on the sample data corresponding to the local bucket number corresponding to the number of iterations, thereby achieving joint training for the first model.

[0109] In some embodiments, the above-mentioned sample data corresponding to the bucket number corresponding to the number of iterations, combined with the sample data corresponding to the bucket number corresponding to the second node, is used to train the second sub-model, including: receiving the second model gradient corresponding to the third sub-model corresponding to the number of iterations sent by the second node; obtaining the first model gradient corresponding to the second sub-model corresponding to the number of iterations; and updating the parameters involved in the second sub-model based on the first model gradient and the second model gradient.

[0110] After starting model training, based on the number of iterations of each iteration, the first model gradient corresponding to the second sub-model in this iteration is obtained, and the first error value corresponding to the second sub-model in this iteration can also be obtained. At the same time, the second model gradient corresponding to the third sub-model in this iteration sent by the second node is received, and the second error value corresponding to the third sub-model in this iteration can also be obtained. The first node updates the model parameters of the second sub-model in this iteration through the first model gradient, the first error value, the second model gradient, and the second error value.

[0111] Corresponding to the training method of the second sub-model, the first node will also send the first model gradient and the first error value corresponding to this iteration to the second node, so that when the second node iterates the third sub-model, based on the first model gradient, the first error value, the second model gradient, and the second error value corresponding to this iteration, the parameters of the third sub-model are updated, and multiple rounds of iterations are carried out until the sub-model in each node is stable.

[0112] In some embodiments, the method further includes: generating a first public key, a first private key corresponding to the first public key, and a first random number; sending the first public key to the second node, and receiving the second public key sent by the second node; the receiving of the second model gradient corresponding to the third sub-model corresponding to the number of iterations sent by the second node includes: receiving a second collection, the second collection is the second node using the first public key to encrypt the second model gradient to obtain the encrypted second model gradient, and then using the user identifier corresponding to the sample data for generating the second model gradient as the key to encrypt the first random number, and the encrypted second model gradient is encrypted twice, the first random number is the random number encrypted by the first node using the second public key; the updating of the parameters involved in the second sub-model based on the first model gradient and the second model gradient includes: based on any one of the user identifiers in the bucket corresponding to the number of iterations, encrypting the first random number and the first public key in the second set. The encrypted second model gradient is decrypted, and when the decryption is successful, the plaintext of the first random number in the second set is obtained; the encrypted second model gradient in the second set is decrypted by the first private key corresponding to the first public key to obtain the second model gradient; the user identifier of the successfully decrypted random number, the corresponding first model gradient, and the second model gradient are extracted; the first model gradient and the second model gradient are merged, and the parameters of the second sub-model are updated based on the merged model gradient; the second random number and the updated model gradient sent by the second node are encrypted by the user identifier of the successfully decrypted random number, and sent to the second node, so that the second node uses any user identifier in the bucket corresponding to the number of iterations and the second private key corresponding to the second public key for decryption. When the decryption is successful, the updated model gradient is obtained, and the third sub-model is updated by the updated model gradient. The second random number is the random number encrypted by the second node with the first public key.

[0113] After starting the model training, in each round of model iteration, an anonymous method based on symmetric encryption and asymmetric encryption is used to update the gradient and random number of the model in each node, and multiple rounds of iterations are performed until the model is stable.

[0114] After determining the number of iterations j, the first node obtains the bucket number k corresponding to the number of iterations j according to the agreed correspondence, and obtains the sample data corresponding to the bucket number k. The second sub-model in the first node trains the sample data corresponding to the bucket number k in the first node to obtain the first model gradient corresponding to the second sub-model. Similarly, the second node obtains the bucket number k corresponding to the number of iterations j in the second node according to the number of iterations j, and obtains the sample data corresponding to the bucket number k in the second node. The third sub-model in the second node trains the sample data corresponding to the bucket number k in the second node to obtain the second model gradient corresponding to the third sub-model.

[0115] The first node generates a first public key, a first private key corresponding to the first public key, and a first random number; the second node generates a second public key, a second private key corresponding to the second public key, and a second random number. After that, the first node and the second node exchange public keys, the first node obtains the second public key sent by the second node, and the second node receives the first public key sent by the first node.

[0116] The first node encrypts the first random number of the first node through the second public key sent by the second node, and sends the encrypted first random number to the second node. After receiving the encrypted first random number, the second node decrypts the first random number through the second private key corresponding to the second public key to obtain the decrypted first random number. Here, the first node and the second node can know the user ID and other information of the sample data corresponding to the current iteration number through the first random number.

[0117] Afterwards, the second node first encrypts the second model gradient of the third sub-model through the first public key sent by the first node to obtain the encrypted second model gradient. Then, the second node uses the user identifier corresponding to the sample data for generating the second model gradient as the key to perform secondary encryption on the encrypted second model gradient to obtain the secondary encrypted second model gradient; the second node then uses the user identifier corresponding to the sample data for generating the second model gradient as the key to encrypt the first random number to obtain the encrypted first random number. The user identifier corresponding to the sample data for generating the second model gradient is used as the key, and can be specifically encrypted by the hash value corresponding to the user identifier. The second node constructs the secondary encrypted second model gradient and the encrypted first random number to obtain a second set, and sends the second set to the first node. For example, the second set can be {[random number [[rb]] of the first node encrypted by the hash value of user ID1, second model gradient corresponding to user ID1 encrypted by the first public key encrypted by the hash value of user ID1] corresponding to the second node; [random number [[rb]] of the first node encrypted by the hash value of user ID2, second model gradient corresponding to user ID2 encrypted by the first public key encrypted by the hash value of user ID2]; ...}.

[0118] After the first node receives the second set, the first node decrypts the second set according to any user identifier in the corresponding bucket, and obtains the plaintext of the first random number and the encrypted second model gradient if the decryption is successful. Afterwards, the first node decrypts the encrypted second model gradient using the first private key corresponding to the first public key to obtain the second model gradient. In summary, the first node obtains the user identifier that can successfully decrypt the random number, the second model gradient, and the first model gradient corresponding to the second sub-model of the first node.

[0119] Afterwards, the first node merges the first model gradient with the second model gradient to obtain a merged gradient, and updates the parameters of the second sub-model based on the merged gradient. Here, the updated parameters of the second sub-model include the updated model gradient of the second sub-model. Here, the updated model gradient of the second sub-model can be the updated first model gradient.

[0120] Corresponding to the above implementation, the second node encrypts the second random number of the second node by using the first public key sent by the first node, and sends the encrypted second random number to the first node.

[0121] After receiving the encrypted second random number, the first node decrypts the second random number using the first private key corresponding to the first public key to obtain the decrypted second random number.

[0122] Afterwards, the first node encrypts the updated first model gradient through the second public key sent by the second node to obtain the encrypted updated first model gradient. Then, the first node uses the user identifier that successfully decrypted the random number to re-encrypt the encrypted updated first model gradient to obtain the re-encrypted updated first model gradient; the first node then uses the user identifier that successfully decrypted the random number to encrypt the second random number to obtain the encrypted second random number. The first node constructs the first set through the re-encrypted updated first model gradient and the encrypted second random number, and sends the first set to the second node.

[0123] After receiving the first set sent by the first node, the second node uses any user identifier in the bucket corresponding to the number of iterations to decrypt the first set. If the decryption is successful, the encrypted updated first model gradient and the plain text of the second random number are obtained. Afterwards, the second node decrypts the encrypted updated first model gradient using the second private key corresponding to the second public key to obtain the updated first model gradient. The second node updates the local third sub-model based on the updated first model gradient and the second model gradient, and further obtains the updated second model gradient of the third sub-model. Further based on the above method, the second node encrypts the updated second model gradient through the above method, so that the first node updates the parameters of the second sub-model based on the updated second model gradient until each sub-model in the federated learning is stable.

[0124] Through the method provided in the above embodiment, by first using a filter to filter out redundant data, bucketing the data based on the user identifier, symmetric encryption and asymmetric encryption of the corresponding model gradients and random numbers after the sample data in each bucket is trained, the parameters of the sub-model in each node are updated and then the model training is performed. This can reduce the number of non-intersecting sample data in different nodes and is suitable for large-scale model training scenarios.

[0125] In the related art, the label vector under the secret sharing state is calculated by the Model Predictive Control (MPC) algorithm. During the training process, the full amount of data and the label vector are simultaneously involved in the model training, resulting in a large amount of non-intersection data also participating in the calculation process, thereby causing a large communication and calculation overhead, especially for the smaller sample data in the intersection part, which will cause greater redundant calculation. However, the method provided in the embodiment of the present application first uses a filter to filter the redundant sample data, uses the position index to bucket the data, and updates the gradient of the model in the node based on the anonymous method of symmetric encryption and asymmetric encryption for the training results corresponding to the sample data in each bucket, which reduces the amount of calculation and makes the model training faster.

[0126] Through the method provided in the embodiment of the present application, the user identification with the intersection of the first node and the second node is first obtained, the sample data corresponding to the intersection is extracted, and then the model training is performed. In the case where a certain node does not want to disclose the user ID to another node, the method provided in the embodiment of the present application can achieve the completion of model training in the case where a certain node does not want to disclose the user ID.

[0127] Those skilled in the art will appreciate that, in the above method of specific implementation, the order in which the steps are written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of the steps should be determined by their functions and possible internal logic.

[0128] Based on the federated learning training given in the above embodiment, Figure 2 Another flow chart of a federated learning training method is shown, which is applied to a second node and includes:

[0129] Step 201: Obtain the filter of the second node and the second position index sent by the first node, the second position index represents the position in the filter of the first node that is not mapped by the sample data, and the filter is used to represent the position where the sample data is inserted and the position where the sample data is not inserted through an array.

[0130] Step 202: Filter out the sample data corresponding to the second position index from the filter of the second node to obtain an updated filter of the second node.

[0131] Step 203: Obtain a second sample data set corresponding to the updated filter of the second node.

[0132] Step 204: Send the first position index of the second node to the first node, so that the filter of the first node filters out the sample data corresponding to the first position index to obtain the updated filter of the first node; obtain the first sample data set corresponding to the updated filter of the first node; the first position index represents the position in the filter of the second node that is not mapped by the sample data.

[0133] Step 205: jointly train the second sub-model and the third sub-model based on the first sample data set and the second sample data set to obtain a first model, wherein the first model is composed of the second sub-model deployed on the first node and the third sub-model deployed on the second node.

[0134] The method given in this embodiment corresponds to the method given in the above steps 101 to 105. The second node can screen the sample data in the second node based on the second position index sent by the first node to obtain a second sample data set, and further train the third sub-model deployed in the second node through the second sample data set, combined with the training of the second sub-model deployed in the first node by the first node through the first sample data set, to achieve joint training of the first model.

[0135] In practical applications, steps 201 to 205 can be implemented based on a processor, and the processor can be at least one of an application specific integrated circuit (ASIC), a digital signal processor (DSP), a digital signal processing device (DSPD), a programmable logic device (PLD), a field programmable gate array (FPGA), a central processing unit (CPU), a controller, a microcontroller, and a microprocessor.

[0136] In some embodiments, the above-mentioned acquisition of the filter of the second node includes: initializing the filter; acquiring a sample data set of the second node; processing each user identifier in the sample data set of the second node to obtain a fourth position index of each user identifier in the filter; and inserting the sample data corresponding to the user identifier into the filter according to the fourth position index.

[0137] Corresponding to the method of inserting sample data into the filter in the first node given in the above embodiment, the second node can obtain the fourth position index based on the user identifier in the sample data set of the second node, and insert the sample data set of the second node into the filter of the second node based on the fourth position index.

[0138] For example, assume that the sample data set S2 of the second node includes three sample data, namely (mobile phone number 12, consumption amount 34), (mobile phone number 122, consumption amount 344), (mobile phone number 123, consumption amount 346), and set the length of the position index of the initial filter to 6, for example, the position index of the initial filter is [0,0,0,0,0,0]. Substitute the user identifier, that is, the mobile phone number, in the sample data set S2 into three hash functions respectively, and obtain the fourth position indexes of [1,1,1,0,0,0], [1,1,0,1,0,0], [1,1,0,0,0,1]. Insert the sample data corresponding to the user identifier at the position where the element is 1 in the fourth position index to insert the sample data into the filter.

[0139] In some embodiments, the fourth position index of each user identifier in the filter is composed of a binary array of length N, where at least one element of the binary array is 1, and N is an integer greater than 2; the above-mentioned inserting the sample data corresponding to the user identifier into the filter according to the fourth position index includes: dividing the filter into N buckets; arbitrarily selecting a bucket with an element of 1 in the fourth position index, and inserting the sample data corresponding to each user identifier into the filter; dividing the sample data set of the second node into the N buckets of the filter, and some of the N buckets have empty data.

[0140] Corresponding to the method of bucketing data for the first node given in the above embodiment, based on the method given in this embodiment, the second node buckets the filter according to the fourth position index, and can insert the sample data into the filter based on the bucket with the corresponding element being 1, and further realize the bucketing of the sample data set in the second node by dividing the sample data set of the second node into N buckets corresponding to the filter.

[0141] In some embodiments, the above-mentioned acquisition of the second sample data set corresponding to the updated filter of the second node includes: receiving the second position index sent by the first node, the second position index indicating the bucket number of the N buckets in the filter of the first node that has no sample data; the filter of the second node eliminates the sample data of the corresponding bucket number according to the bucket number involved in the second position index, and obtains the remaining sample data, and the remaining sample data serves as the second sample data set of the second node.

[0142] Corresponding to the method for obtaining the first sample data set from the first node given in the above embodiment, after the data is bucketed for the second node filter, the second position index can also be the bucket number in which there is no sample data in the first node. By corresponding the bucket numbers, the sample data in the bucket number corresponding to the second position index is eliminated, thereby implementing the screening of the sample data set of the second node and obtaining the second sample data set.

[0143] For example, in the second node, after inserting each sample data in the sample data set S2 into the bucket, according to the received second position index, assuming that the second position index is 5 and 6, the second node deletes the corresponding data in bucket number 5 and bucket number 6, and obtains the second sample data set S21 as (mobile phone number 12 hash value, consumption amount 34), (mobile phone number 122 hash value, consumption amount 344).

[0144] In some embodiments, the above-mentioned joint training of the third sub-model based on the first sample data set and the second sample data set includes: agreeing with the first node on model training parameters and the bucket number selected in each round of model iteration; determining the sample data corresponding to the bucket number corresponding to the number of iterations based on the number of iterations of the first model; training the third sub-model based on the sample data corresponding to the bucket number corresponding to the number of iterations, combined with the sample data corresponding to the bucket number corresponding to the first node.

[0145] Corresponding to the method of training the sub-model of the first node given in the above embodiment, the second node can train the third sub-model in the second node according to the sample data corresponding to the bucket number corresponding to the agreed number of iterations, and combine the training of the second sub-model by the first node to realize joint training of the first model.

[0146] In some embodiments, the above-mentioned sample data corresponding to the bucket number corresponding to the number of iterations, combined with the sample data corresponding to the bucket number corresponding to the first node, is used to train the third sub-model, including: receiving the first model gradient corresponding to the second sub-model corresponding to the number of iterations sent by the first node; obtaining the second model gradient corresponding to the third sub-model corresponding to the number of iterations; and updating the parameters involved in the third sub-model based on the first model gradient and the second model gradient.

[0147] Corresponding to the method for the first node to update the parameters of the second sub-model given in the above embodiment, the second node can update the parameters involved in the third sub-model through the first model gradient of the first node and the second model gradient of the local third sub-model.

[0148] In some embodiments, the method further includes: generating a second public key, a second private key corresponding to the second public key, and a second random number; sending the second public key to the first node, and receiving the first public key sent by the first node; the receiving of the first model gradient corresponding to the second sub-model corresponding to the number of iterations sent by the first node includes: receiving a first collection, the first collection is the first node using the second public key to encrypt the first model gradient to obtain the encrypted first model gradient, and then using the user identifier corresponding to the sample data for generating the first model gradient as the key to encrypt the second random number, and the encrypted first model gradient is encrypted twice, the second random number is the random number encrypted by the second node using the first public key; the updating of the parameters involved in the third sub-model based on the first model gradient and the second model gradient includes: based on any one user identifier in the bucket corresponding to the number of iterations, encrypting the second random number and the second public key in the first set. The encrypted first model gradient is decrypted, and when the decryption is successful, the plain text of the second random number in the first set is obtained; the encrypted first model gradient in the first set is decrypted by the second private key corresponding to the second public key to obtain the first model gradient; the user identifier of the successfully decrypted random number, the corresponding first model gradient, and the second model gradient are extracted; the first model gradient and the second model gradient are merged, and the parameters of the third sub-model are updated based on the merged model gradient; the first random number and the updated model gradient sent by the first node are encrypted by the user identifier of the successfully decrypted random number, and sent to the first node, so that the first node uses any user identifier in the bucket corresponding to the number of iterations and the first private key corresponding to the first public key for decryption. When the decryption is successful, the updated model gradient is obtained, and the second sub-model is updated by the updated model gradient. The first random number is the random number encrypted by the first node with the second public key.

[0149] Corresponding to the above embodiment, the second node can receive the updated first gradient model and the second random number sent by the first node, and update the parameters of the third sub-model based on the first gradient model updated by the first node and the second gradient model. After completing the parameter update, the second node can obtain the updated second model gradient, and based on the encryption method given in this embodiment, send the encrypted updated second model gradient and the first random number to the first node, so that the first node implements the parameter update of the second sub-model according to the latest second model gradient and the first model gradient of the second sub-model.

[0150] The above embodiment provides a method for implementing federated learning training based on the second node, which corresponds to the method for implementing federated learning training based on the first node. The specific implementation method can refer to the implementation method for implementing federated learning training based on the first node.

[0151] Based on the federated learning training method given in the above embodiment, Figure 3A data processing method flow chart is shown, comprising:

[0152] Step 301: Obtain data to be processed.

[0153] Step 302: Use the trained first model to process the data to be processed.

[0154] Here, the first model is obtained by jointly training the second sub-model and the third sub-model based on the method given in the above embodiment.

[0155] Based on the federated learning training method proposed in the above embodiment, the present application embodiment also provides a federated learning training device, which is applied to the first node, such as Figure 4 As shown, the federated learning training device includes:

[0156] The first processing module 401 is used to obtain the filter of the first node and the first position index sent by the second node, the first position index represents the position in the filter of the second node that is not mapped by the sample data, and the filter is used to represent the position where the sample data is inserted and the position where the sample data is not inserted through an array; the sample data corresponding to the first position index is filtered out from the filter of the first node to obtain the updated filter of the first node.

[0157] The second processing module 402 is used to obtain the first sample data set corresponding to the updated filter of the first node; send the second position index of the first node to the second node, so that the filter of the second node filters out the sample data corresponding to the second position index to obtain the updated filter of the second node; obtain the second sample data set corresponding to the updated filter of the second node; the second position index represents the position in the filter of the first node that is not mapped by the sample data.

[0158] The first training module 403 is used to jointly train the second sub-model and the third sub-model based on the first sample data set and the second sample data set to obtain a first model, where the first model is composed of the second sub-model deployed on the first node and the third sub-model deployed on the second node.

[0159] In practical applications, the first processing module 401, the second processing module 402, and the first training module 403 can be implemented based on a processor and a communication device.

[0160] In some embodiments, the first processing module 401 is specifically used to initialize the filter; obtain a sample data set of the first node; process each user identifier in the sample data set of the first node to obtain a third position index of each user identifier in the filter; and insert the sample data corresponding to the user identifier into the filter according to the third position index.

[0161] In some embodiments, the third position index of each user identifier in the filter is composed of a binary array of length N, where at least one element of the binary array is 1, and N is an integer greater than 2; the first processing module 401 is specifically used to divide the filter into N buckets; according to any one bucket with an element of 1 selected in the third position index, the sample data corresponding to each user identifier is inserted into the filter; the sample data set of the first node is divided into the N buckets of the filter, and some of the N buckets have empty data.

[0162] In some embodiments, the first processing module 401 is also used to receive a first position index sent by the second node, where the first position index represents the bucket number of the N buckets in the filter of the second node that has no sample data; the filter of the first node eliminates the sample data of the corresponding bucket number based on the bucket number involved in the first position index to obtain the remaining sample data, and the remaining sample data serves as the first sample data set of the first node.

[0163] In some embodiments, the first training module 403 is specifically used to agree with the second node on model training parameters and the bucket number selected in each round of model iteration; based on the number of iterations of the first model, determine the sample data corresponding to the bucket number corresponding to the number of iterations; based on the sample data corresponding to the bucket number corresponding to the number of iterations, combined with the sample data corresponding to the bucket number corresponding to the second node, train the second sub-model.

[0164] In some embodiments, the first training module 403 is specifically used to receive the second model gradient corresponding to the third sub-model corresponding to the iteration number sent by the second node; obtain the first model gradient corresponding to the second sub-model corresponding to the iteration number; and update the parameters involved in the second sub-model based on the first model gradient and the second model gradient.

[0165] In some embodiments, the first training module 403 is specifically used to generate a first public key, a first private key corresponding to the first public key, and a first random number; send the first public key to the second node, and receive the second public key sent by the second node; the first training module 403 is specifically used to receive a second collection, the second collection is the second node using the first public key to encrypt the second model gradient to obtain the encrypted second model gradient, and then use the user identifier corresponding to the sample data for generating the second model gradient as the key to encrypt the first random number, and the encrypted second model gradient is encrypted twice, the first random number is the random number encrypted by the first node using the second public key; the first training module 403 is specifically used to, based on any one user identifier in the bucket corresponding to the number of iterations, decrypt the first random number in the second set and the second model gradient encrypted by the first public key, and in the decrypted In the case of success, the plaintext of the first random number in the second set is obtained; the encrypted second model gradient in the second set is decrypted by the first private key corresponding to the first public key to obtain the second model gradient; the user identifier of the successfully decrypted random number, the corresponding first model gradient, and the second model gradient are extracted; the first model gradient and the second model gradient are merged, and the parameters of the second sub-model are updated based on the merged model gradient; the second random number and the updated model gradient sent by the second node are encrypted by the user identifier of the successfully decrypted random number, and sent to the second node, so that the second node uses any user identifier in the bucket corresponding to the number of iterations and the second private key corresponding to the second public key for decryption. In the case of successful decryption, the updated model gradient is obtained, and the third sub-model is updated by the updated model gradient. The second random number is the random number encrypted by the second node with the first public key.

[0166] It should be noted that the description of the above device embodiment is similar to the description of the above method embodiment, and has similar beneficial effects as the same method embodiment. For technical details not disclosed in the device embodiment of the present application, please refer to the description of the method embodiment of the present application for understanding.

[0167] Based on the data processing method proposed in the above embodiment, the embodiment of the present application also provides another data processing device, which is applied to the second node, such as Figure 5 As shown, the data processing device includes:

[0168] The third processing module 501 is used to obtain the filter of the second node and the second position index sent by the first node, where the second position index represents the position in the filter of the first node that is not mapped by the sample data, and the filter is used to represent the position where the sample data is inserted and the position where the sample data is not inserted through an array; the sample data corresponding to the second position index is filtered out from the filter of the second node to obtain the updated filter of the second node.

[0169] The fourth processing module 502 is used to obtain the second sample data set corresponding to the updated filter of the second node; send the first position index of the second node to the first node, so that the filter of the first node filters out the sample data corresponding to the first position index to obtain the updated filter of the first node; obtain the first sample data set corresponding to the updated filter of the first node; the first position index represents the position in the filter of the second node that is not mapped by the sample data.

[0170] The second training module 503 is used to jointly train the second sub-model and the third sub-model based on the first sample data set and the second sample data set to obtain a first model, where the first model is composed of the second sub-model deployed on the first node and the third sub-model deployed on the second node.

[0171] In practical applications, the third processing module 501, the fourth processing module 502, and the second training module 503 can be implemented based on a processor and a communication device.

[0172] In some embodiments, the third processing module 501 is specifically used to initialize the filter; obtain a sample data set of the second node; process each user identifier in the sample data set of the second node to obtain a fourth position index of each user identifier in the filter; and insert the sample data corresponding to the user identifier into the filter according to the fourth position index.

[0173] In some embodiments, the fourth position index of each user identifier in the filter is composed of a binary array of length N, where at least one element of the binary array is 1, and N is an integer greater than 2; the third processing module 501 is specifically used to divide the filter into N buckets; according to any one bucket with an element of 1 selected in the fourth position index, the sample data corresponding to each user identifier is inserted into the filter; the sample data set of the second node is divided into the N buckets of the filter, and some of the N buckets have empty data.

[0174] In some embodiments, the third processing module 501 is also used to receive a second position index sent by the first node, where the second position index represents the bucket number of the N buckets in the filter of the first node that has no sample data; the filter of the second node eliminates the sample data of the corresponding bucket number based on the bucket number involved in the second position index to obtain the remaining sample data, and the remaining sample data serves as the second sample data set of the second node.

[0175] In some embodiments, the second training module 503 is specifically used to agree with the first node on model training parameters and the bucket number selected in each round of model iteration; based on the number of iterations of the first model, determine the sample data corresponding to the bucket number corresponding to the number of iterations; based on the sample data corresponding to the bucket number corresponding to the number of iterations, combined with the sample data corresponding to the bucket number corresponding to the first node, train the third sub-model.

[0176] In some embodiments, the second training module 503 is specifically used to receive the first model gradient corresponding to the second sub-model corresponding to the iteration number sent by the first node; obtain the second model gradient corresponding to the third sub-model corresponding to the iteration number; and update the parameters involved in the third sub-model based on the first model gradient and the second model gradient.

[0177] In some embodiments, the second training module 503 is specifically used to generate a second public key, a second private key corresponding to the second public key, and a second random number; send the second public key to the first node, and receive the first public key sent by the first node; the second training module 503 is specifically used to receive a first collection, the first collection is the first node using the second public key to encrypt the first model gradient to obtain the encrypted first model gradient, and then use the user identifier corresponding to the sample data for generating the first model gradient as the key to encrypt the second random number, and the encrypted first model gradient is encrypted twice, the second random number is the random number encrypted by the second node using the first public key; the second training module 503 is specifically used to, based on any one user identifier in the bucket corresponding to the number of iterations, decrypt the second random number in the first set and the first model gradient encrypted with the second public key, and in the decrypted If the decryption succeeds, the plaintext of the second random number in the first set is obtained; the encrypted first model gradient in the first set is decrypted by the second private key corresponding to the second public key to obtain the first model gradient; the user identifier of the successfully decrypted random number, the corresponding first model gradient, and the second model gradient are extracted; the first model gradient and the second model gradient are merged, and the parameters of the third sub-model are updated based on the merged model gradient; the first random number and the updated model gradient sent by the first node are encrypted by the user identifier of the successfully decrypted random number, and sent to the first node, so that the first node uses any user identifier in the bucket corresponding to the number of iterations and the first private key corresponding to the first public key for decryption. If the decryption is successful, the updated model gradient is obtained, and the second sub-model is updated by the updated model gradient. The first random number is the random number encrypted by the first node with the second public key.

[0178] It should be noted that the description of the above device embodiment is similar to the description of the above method embodiment, and has similar beneficial effects as the same method embodiment. For technical details not disclosed in the device embodiment of the present application, please refer to the description of the method embodiment of the present application for understanding.

[0179] Based on the data processing method proposed in the above embodiment, the present application embodiment also provides a data processing device, such as Figure 6 As shown, the data processing device includes:

[0180] The acquisition module 601 is used to acquire data to be processed.

[0181] The fifth processing module 602 is used to process the data to be processed using the trained first model, where the first model is obtained by jointly training the second sub-model of the first node and the third model of the first and second nodes based on the training method given in the above embodiment.

[0182] In practical applications, the acquisition module 601 and the fifth processing module 602 can be implemented based on a processor and a communication device.

[0183] In practical applications, Figure 4 In the federated learning training device shown, the first processing module 401 can specifically be a data processing module of the first node, and the first training module 403 can specifically be divided into a data encryption and calculation module, and an interaction module. Figure 5 In the federated learning training device shown, the third processing module 501 can be specifically a data processing module of the second node, and the second training module 503 can be specifically divided into a data encryption and calculation module, and an interaction module. Based on the federated learning training method given in the above embodiment, Figure 7 Given various modules based on the first node and the second node, the method for implementing federated learning training may specifically include the following steps:

[0184] Step 701: The data processing module of the first node initializes the filter D1 and updates D1 according to the user identifier of the sample data.

[0185] The data processing module of the first node constructs an initial filter D1, wherein the initial filter D1 is composed of elements representing positions that are not mapped by the sample data. The data processing module of the first node obtains a third position index based on the user identifier in the sample data set in the first node, and updates the initial filter D1 using the method given in the above embodiment. Based on the example given in the above embodiment, the position index of the filter D1 of the first node is obtained as [1,1,1,1,0,0], and the sample data corresponding to the user identifier of the first node is inserted into the filter D1. The data processing module of the first node can also bucket the data according to the filter D1.

[0186] Step 702: The data processing module of the second node initializes the filter D2 and updates D2 according to the user identifier of the sample data.

[0187] The data processing module of the second node constructs an initial filter D2, wherein the initial filter D2 is composed of elements representing positions that are not mapped by the sample data. The data processing module of the second node obtains a fourth position index according to the user identifier in the sample data set in the second node, updates the initial filter D2 using the same method as updating D1, and inserts the sample data corresponding to the user identifier of the second node into the filter D2. The data processing module of the second node can also perform data bucketing according to the filter D2.

[0188] For example, assume that the sample data set S2 of the second node includes 3 records: (mobile phone number 12, consumption amount 34), (mobile phone number 122, consumption amount 344), (mobile phone number 123, consumption amount 346). Based on the method given in the above embodiment, the position index of filter D2 is [1,1,1,0,1,1]. After obtaining filter D2, the sample data corresponding to the user identifier of the second node is inserted into filter D2. The inserted sample data can be {hash value of user identifier, X2}.

[0189] Step 702 and step 701 may be performed simultaneously in corresponding nodes.

[0190] Step 703: The interaction module of the first node exchanges position indexes with the interaction module of the second node.

[0191] After the data is bucketed, the interaction module of the first node sends the second position index to the second node; the interaction module of the second node sends the first position index to the first node. The first position index indicates the bucket number of the N buckets in the filter of the second node that does not have sample data; the second position index indicates the bucket number of the N buckets in the filter of the first node that does not have sample data.

[0192] Based on the above example, the first position index includes bucket number 4, and the second position index includes bucket number 5 and bucket number 6.

[0193] Step 704: The data processing module of the first node updates the sample data based on the location index and performs data bucketing.

[0194] The data processing module of the first node finds the bucket corresponding to position 4 of the position index [1,1,1,1,0,0] of D1 based on the first position index, updates element 1 corresponding to position 4 to 0, and uses the sample data corresponding to the remaining bucket numbers as the first sample data set.

[0195] Step 705: The data processing module of the second node updates the sample data based on the location index and performs data bucketing.

[0196] The data processing module of the second node obtains the updated position index [1,1,1,0,0,0] of the filter D2 based on the second position index, and uses the sample data corresponding to the remaining bucket numbers as the second sample data set S21.

[0197] For example, the second sample data set S21 corresponding to the updated position index [1,1,1,0,0,0] of D2 is (mobile phone number 12 hash value, consumption amount 34), (mobile phone number 122 hash value, consumption amount 344).

[0198] Step 705 and step 704 may be performed simultaneously in corresponding nodes.

[0199] Step 706: The data encryption and calculation module of the first node generates a first asymmetric key and a random number.

[0200] Here, the first asymmetric key includes a first public key and a first private key, and the random number generated by the first node is a first random number.

[0201] Step 707: The data encryption and calculation module of the second node generates a second asymmetric key and a random number.

[0202] Here, the second asymmetric key includes a second public key and a second private key, and the random number generated by the second node is a second random number.

[0203] Step 707 and step 706 may be performed simultaneously in corresponding nodes.

[0204] Step 708: The interaction module of the first node and the interaction module of the second node exchange public keys.

[0205] The interaction module of the first node sends the first public key to the second node, and the interaction module of the second node sends the second public key to the first node.

[0206] Step 709: The data encryption and computing module of the first node initializes the model and determines the bucket number for each round of iteration.

[0207] The data encryption and calculation module of the first node initializes the first model, and setting the first model parameters may include one or more of the following: learning rate, number of iterations, stochastic gradient descent algorithm, parameters of the initial model, etc. According to the number of iterations, the bucket numbers of each round of iteration of the two nodes are randomly generated, wherein the value range of the bucket number of each round is the bucket number corresponding to S11 and S21.

[0208] Step 710: The data encryption and computing module of the second node initializes the model and determines the bucket number for each round of iteration.

[0209] Corresponding to step 709, the data encryption and calculation module of the second node initializes the second model, and the second model parameters may include one or more of the following: learning rate, number of iterations, stochastic gradient descent algorithm, parameters of the initial model, etc. According to the number of iterations, the bucket numbers of each round of iteration of the two nodes are randomly generated, wherein the bucket number of each round ranges from the bucket number corresponding to S11 and S21. The bucket number of each round of iteration is agreed upon by the first node and the second node.

[0210] Step 710 and step 709 may be performed simultaneously in corresponding nodes.

[0211] Step 711: The interaction module of the first node and the interaction module of the second node exchange set data based on symmetric encryption and asymmetric encryption.

[0212] Based on the method provided in the above embodiment, the interaction module of the second node sends the second set to the first node, and correspondingly, the interaction module of the first node sends the first set to the second node.

[0213] Step 712: The data encryption and computing module of the first node performs multiple rounds of training until the model is stable.

[0214] The data encryption and calculation module of the first node updates the parameters of the second sub-model based on the data in the second set and the first model gradient and the first error value corresponding to the second sub-model, and iterates the second sub-model multiple times based on the above steps until the second sub-model is stable.

[0215] Step 713: The data encryption and computing module of the second node performs multiple rounds of training until the model is stable.

[0216] The data encryption and calculation module of the second node updates the parameters of the third sub-model based on the data in the first set and the second model gradient and the second error value corresponding to the third sub-model, and iterates the third sub-model multiple times based on the above steps until the third sub-model is stable.

[0217] exist Figure 7 Based on the model training system flow chart shown, Figure 8 Another federated learning training flow chart is shown, including:

[0218] Step 801: The data encryption and calculation module of the second node generates a second set based on the k-th bucket sample data.

[0219] Based on the method given in the above embodiment, the data encryption and calculation module of the second node determines the corresponding bucket number k based on the number of iterations j, obtains the second model gradient and second error value of the third sub-model based on the sample data corresponding to the bucket number k, and encrypts the second model gradient and second error value with the first public key sent by the first node to obtain the encrypted second model gradient and second error value. At the same time, corresponding to the method given in the above embodiment, the data encryption and calculation module of the second node uses the user identifier corresponding to the sample data for generating the second model gradient as a key, encrypts the first random number corresponding to the first node, and obtains the encrypted first random number; the data encryption and calculation module of the second node also uses the user identifier corresponding to the sample data for generating the second model gradient as a key, and performs secondary encryption on the encrypted second model gradient and the second error value, and finally obtains the second set corresponding to the second node, and the second set can be expressed as {[the first random number [[rb]] of the first node encrypted by the hash value of user ID1, the second model gradient corresponding to user ID1 encrypted by the first public key encrypted by the hash value of user ID1, and the second error value corresponding to user ID1 encrypted by the first public key encrypted by the hash value of user ID1]; [the first random number [[rb]] of the first node encrypted by the hash value of user ID2, the second model gradient corresponding to user ID2 encrypted by the first public key encrypted by the hash value of user ID1, and the second error value corresponding to user ID2 encrypted by the first public key encrypted by the hash value of user ID1]; ...}.

[0220] Step 802: The data encryption and calculation module of the first node generates a first set based on the k-th bucket sample data.

[0221] The data encryption and calculation module of the first node determines the corresponding bucket number k based on the number of iterations j, and obtains the first model gradient and the first error value of the second sub-model model based on the sample data corresponding to the bucket number k.

[0222] Step 803: The interaction module of the second node sends the encrypted second set to the first node.

[0223] Step 804: The data encryption and calculation module of the first node decrypts the encrypted second set and updates the second sub-model.

[0224] After the interaction module of the first node receives the second set sent by the interaction module of the second node, the encryption and calculation module of the first node extracts the bucket number k corresponding to the iteration number j, and arbitrarily selects a user identifier corresponding to a sample data from the sample data corresponding to the bucket number k, and uses the hash value of the user ID of the selected sample data to decrypt the first random number in the encrypted second set. If the decrypted value does not contain a random number, a user identifier corresponding to a sample data is randomly selected from other sample data in bucket number k, and decrypted using the above method until the decrypted value is the first random number. The data encryption and calculation module of the first node decrypts the encrypted second model gradient sent by the second node through the first private key, and merges it with the corresponding first model gradient of the first node, and updates the second sub-model based on the merged model gradient.

[0225] Step 805: The data encryption and calculation module of the first node updates the first set.

[0226] After updating the second sub-model, the first node obtains the parameters of the updated second sub-model, wherein the parameters of the updated second sub-model include the updated first model gradient. Based on the updated first model gradient, the data in the first set is updated and encrypted to obtain the encrypted first set.

[0227] Step 806: The interaction module of the first node sends the updated first set to the second node.

[0228] Based on the method provided in the above embodiment, the data encryption and calculation module of the first node encrypts the updated first set and sends the updated first set to the second node.

[0229] Step 807: The data encryption and calculation module of the second node decrypts the updated first set and updates the third sub-model.

[0230] Steps 801 to 807 are repeated until the second sub-model and the third sub-model are stable.

[0231] It should be noted that in the embodiment of the present application, the federated learning training method applied to the second node is the same as the federated learning training method applied to the first node, and the specific implementation method can refer to the method given in the above embodiment.

[0232] It should be noted that in the embodiments of the present application, if the above method is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium, including a number of instructions to enable a computer device (which can be a terminal, a server, etc.) to execute all or part of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a U disk, a mobile hard disk, a read-only memory (ROM), a disk or an optical disk. In this way, the embodiments of the present application are not limited to any specific combination of hardware and software.

[0233] An embodiment of the present application also provides an electronic device. Fig. 9 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application is shown in FIG. Fig. 9 As shown, the electronic device 90 may include:

[0234] The memory 901 is used to store executable instructions.

[0235] The processor 902 is used to implement any one of the above-mentioned federated learning training methods implemented by the first node, or the federated learning training method implemented by the second node, or the data processing method when executing the executable instructions stored in the memory 901.

[0236] The processor 902 may be at least one of an ASIC, a DSP, a DSPD, a PLD, a FPGA, a CPU, a controller, a microcontroller, and a microprocessor.

[0237] The above-mentioned computer-readable storage medium or memory 901 can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic random access memory (FRAM), a flash memory (Flash Memory), a magnetic surface memory, an optical disk, or a compact disc read-only memory (CD-ROM) and other memories; it can also be various terminals including one or any combination of the above-mentioned memories, such as mobile phones, computers, tablet devices, personal digital assistants, etc.

[0238] An embodiment of the present application further provides a computer storage medium, on which computer executable instructions are stored, and the computer executable instructions are used to implement any one of the federated learning training methods implemented by the first node, or the federated learning training method implemented by the second node, or the data processing method provided in the above embodiments.

[0239] Correspondingly, an embodiment of the present application further provides a computer program product, which includes computer executable instructions, and the computer executable instructions are used to implement any one of the federated learning training methods implemented by the first node, or the federated learning training method implemented by the second node, or the data processing method provided in the above embodiments.

[0240] In some embodiments, the functions or modules included in the device provided in the embodiments of the present application can be used to execute the method described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments. For the sake of brevity, it will not be repeated here.

[0241] The above description of various embodiments tends to emphasize the differences between the various embodiments. The same or similar aspects can be referenced to each other, and for the sake of brevity, they will not be repeated herein.

[0242] The methods disclosed in the various method embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments.

[0243] The features disclosed in the various product embodiments provided in this application can be arbitrarily combined without conflict to obtain new product embodiments.

[0244] The features disclosed in the various method or device embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments or device embodiments.

[0245] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0246] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. A federated learning training method, characterized in that: Applied to the first node, the method comprises: Obtaining a filter of the first node and a first position index sent by the second node, wherein the first position index represents a position in the filter of the second node that is not mapped by sample data, and the filter is used to represent a position where sample data is inserted and a position where sample data is not inserted through an array; Filtering out the sample data corresponding to the first position index from the filter of the first node to obtain an updated filter of the first node; Obtaining a first sample data set corresponding to the updated filter of the first node; Sending the second position index of the first node to the second node, so that the filter of the second node filters out the sample data corresponding to the second position index, and obtains the updated filter of the second node; obtaining the second sample data set corresponding to the updated filter of the second node; the second position index represents the position in the filter of the first node that is not mapped by the sample data; The second sub-model and the third sub-model are jointly trained based on the first sample data set and the second sample data set to obtain a first model, wherein the first model is composed of the second sub-model deployed on the first node and the third sub-model deployed on the second node.

2. The method according to claim 1, characterized in that The obtaining the filter of the first node includes: Initialize the filter; Obtaining a sample data set of the first node; Each user identifier in the sample data set of the first node is processed to obtain a third position index of each user identifier in the filter; and sample data corresponding to the user identifier is inserted into the filter according to the third position index.

3. The method according to claim 2, characterized in that: The third position index of each user identifier in the filter is composed of a binary array of length N, wherein at least one element of the binary array is 1, and N is an integer greater than 2; and inserting the sample data corresponding to the user identifier into the filter according to the third position index includes: Divide the filter into N buckets; According to any bucket with an element of 1 selected in the third position index, the sample data corresponding to each user identifier is inserted into the filter; The sample data set of the first node is divided into N buckets of the filter, and some of the N buckets have empty data.

4. The method according to claim 3, characterized in that The obtaining of a first sample data set corresponding to the updated filter of the first node includes: Receive a first position index sent by the second node, where the first position index indicates a bucket number without sample data among the N buckets in the filter of the second node; The filter of the first node removes the sample data of the corresponding bucket number according to the bucket number involved in the first position index to obtain the remaining sample data, and the remaining sample data is used as the first sample data set of the first node.

5. The method according to claim 4, characterized in that: The jointly training the second sub-model based on the first sample data set and the second sample data set includes: Agree with the second node on the model training parameters and the bucket number to be selected in each round of model iteration; Based on the number of iterations of the first model, determine the sample data corresponding to the bucket number corresponding to the number of iterations; The second sub-model is trained based on the sample data corresponding to the bucket number corresponding to the iteration number and the sample data corresponding to the bucket number corresponding to the second node.

6. The method according to claim 5, characterized in that The training of the second sub-model based on the sample data corresponding to the bucket number corresponding to the iteration number and the sample data corresponding to the bucket number corresponding to the second node includes: receiving a second model gradient corresponding to the third sub-model corresponding to the number of iterations sent by the second node; Obtaining a first model gradient corresponding to a second sub-model corresponding to the number of iterations; Based on the first model gradient and the second model gradient, parameters involved in the second sub-model are updated.

7. The method according to claim 6, characterized in that The method further comprises: Generate a first public key, a first private key corresponding to the first public key, and a first random number; Sending a first public key to the second node, and receiving a second public key sent by the second node; The receiving the second model gradient corresponding to the third sub-model corresponding to the iteration number sent by the second node includes: receiving a second collection, where the second collection is obtained by encrypting the second model gradient by the second node using the first public key to obtain an encrypted second model gradient, then using the user identifier corresponding to the sample data for generating the second model gradient as a key to encrypt the first random number, and performing secondary encryption on the encrypted second model gradient, where the first random number is a random number encrypted by the first node using the second public key; The updating of parameters involved in the second sub-model based on the first model gradient and the second model gradient includes: Decrypting the first random number in the second set and the second model gradient encrypted by the first public key based on any one user identifier in the bucket corresponding to the number of iterations, and obtaining the plain text of the first random number in the second set when the decryption is successful; Decrypting the encrypted second model gradient in the second set by using the first private key corresponding to the first public key to obtain the second model gradient; Extracting the user identifier of the successfully decrypted random number, the corresponding first model gradient, and the second model gradient; Merging the first model gradient and the second model gradient, and updating the parameters of the second sub-model based on the merged model gradient; The second random number and the updated model gradient sent by the second node are encrypted using the user identifier that successfully decrypted the random number, and sent to the second node, so that the second node uses any user identifier in the bucket corresponding to the number of iterations and the second private key corresponding to the second public key to decrypt them. When the decryption is successful, the updated model gradient is obtained, and the third sub-model is updated by the updated model gradient. The second random number is the random number encrypted by the second node using the first public key.

8. A federated learning training method, characterized in that: Applied to the second node, the method comprises: Acquire the filter of the second node and the second position index sent by the first node, wherein the second position index represents a position in the filter of the first node that is not mapped by the sample data, and the filter is used to represent the position where the sample data is inserted and the position where the sample data is not inserted through an array; Filter out the sample data corresponding to the second position index from the filter of the second node to obtain an updated filter of the second node; Obtain a second sample data set corresponding to the updated filter of the second node; Sending the first position index of the second node to the first node, so that the filter of the first node filters out the sample data corresponding to the first position index, and obtains the updated filter of the first node; obtaining the first sample data set corresponding to the updated filter of the first node; the first position index represents the position in the filter of the second node that is not mapped by the sample data; The second sub-model and the third sub-model are jointly trained based on the first sample data set and the second sample data set to obtain a first model, wherein the first model is composed of the second sub-model deployed on the first node and the third sub-model deployed on the second node.

9. The method according to claim 8, characterized in that The obtaining the filter of the second node includes: Initialize the filter; Obtaining a sample data set of the second node; Each user identifier in the sample data set of the second node is processed to obtain a fourth position index of each user identifier in the filter; and sample data corresponding to the user identifier is inserted into the filter according to the fourth position index.

10. The method according to claim 9, characterized in that: The fourth position index of each user identifier in the filter is composed of a binary array of length N, at least one element of the binary array is 1, and N is an integer greater than 2; inserting the sample data corresponding to the user identifier into the filter according to the fourth position index includes: Divide the filter into N buckets; According to the fourth position index, any bucket with an element of 1 is selected, and the sample data corresponding to each user identifier is inserted into the filter; The sample data set of the second node is divided into N buckets of the filter, and some of the N buckets have empty data.

11. The method according to claim 10, characterized in that The obtaining of a second sample data set corresponding to the updated filter of the second node includes: Receive a second position index sent by the first node, where the second position index indicates a bucket number without sample data among the N buckets in the filter of the first node; The filter of the second node removes the sample data of the corresponding bucket number according to the bucket number involved in the second position index to obtain the remaining sample data, and the remaining sample data is used as the second sample data set of the second node.

12. The method according to claim 11, characterized in that: The jointly training a third sub-model based on the first sample data set and the second sample data set includes: Agree with the first node on model training parameters and the bucket number to be selected for each round of model iteration; Based on the number of iterations of the first model, determine the sample data corresponding to the bucket number corresponding to the number of iterations; The third sub-model is trained based on the sample data corresponding to the bucket number corresponding to the iteration number and the sample data corresponding to the bucket number corresponding to the first node.

13. The method according to claim 12, characterized in that The training of the third sub-model based on the sample data corresponding to the bucket number corresponding to the iteration number and the sample data corresponding to the bucket number corresponding to the first node includes: Receive a first model gradient corresponding to a second sub-model corresponding to an iteration number sent by the first node; Obtain a second model gradient corresponding to the third sub-model corresponding to the number of iterations; Based on the first model gradient and the second model gradient, parameters involved in the third sub-model are updated.

14. The method according to claim 13, characterized in that The method further comprises: Generate a second public key, a second private key corresponding to the second public key, and a second random number; Sending a second public key to the first node, and receiving a first public key sent by the first node; The receiving the first model gradient corresponding to the second sub-model corresponding to the iteration number sent by the first node includes: Receive a first collection, where the first collection is obtained by encrypting the first model gradient by the first node using the second public key to obtain an encrypted first model gradient, then using the user identifier corresponding to the sample data for generating the first model gradient as a key to encrypt a second random number, and performing secondary encryption on the encrypted first model gradient, where the second random number is a random number encrypted by the second node using the first public key; The updating of parameters involved in the third sub-model based on the first model gradient and the second model gradient includes: Decrypting the first model gradient encrypted by the second random number and the second public key in the first set based on any one user identifier in the bucket corresponding to the number of iterations, and obtaining the plain text of the second random number in the first set when the decryption is successful; Decrypting the encrypted first model gradient in the first set by using the second private key corresponding to the second public key to obtain the first model gradient; Extracting the user identifier of the successfully decrypted random number, the corresponding first model gradient, and the second model gradient; Merging the first model gradient and the second model gradient, and updating the parameters of the third sub-model based on the merged model gradient; The first random number and the updated model gradient sent by the first node are encrypted using the user identifier that successfully decrypted the random number, and sent to the first node, so that the first node uses any user identifier in the bucket corresponding to the number of iterations and the first private key corresponding to the first public key to decrypt them. When the decryption is successful, the updated model gradient is obtained, and the second sub-model is updated through the updated model gradient. The first random number is the random number encrypted by the first node using the second public key.

15. A data processing method, characterized in that: The method comprises: Get the data to be processed; The data to be processed is processed using a trained first model, wherein the first model is trained based on the training method described in any one of claims 1 to 7 or any one of claims 8 to 14.

16. A federated learning training device, characterized in that: Applied to a first node, the device comprises: A first processing module is configured to obtain a filter of the first node and a first position index sent by the second node, wherein the first position index indicates a position in the filter of the second node that is not mapped by sample data, and the filter is configured to represent positions where sample data is inserted and positions where sample data is not inserted through an array; the sample data corresponding to the first position index is filtered out from the filter of the first node to obtain an updated filter of the first node; The second processing module is used to obtain a first sample data set corresponding to the updated filter of the first node; send the second position index of the first node to the second node, so that the filter of the second node filters out the sample data corresponding to the second position index, and obtain the updated filter of the second node; obtain a second sample data set corresponding to the updated filter of the second node; the second position index represents a position in the filter of the first node that is not mapped by the sample data; The first training module is used to jointly train the second sub-model and the third sub-model based on the first sample data set and the second sample data set to obtain a first model, wherein the first model is composed of the second sub-model deployed on the first node and the third sub-model deployed on the second node.

17. A federated learning training device, characterized in that: Applied to a second node, the device comprises: A third processing module is used to obtain the filter of the second node and the second position index sent by the first node, wherein the second position index represents the position in the filter of the first node that is not mapped by the sample data, and the filter is used to represent the position where the sample data is inserted and the position where the sample data is not inserted through an array; the sample data corresponding to the second position index is filtered out from the filter of the second node to obtain an updated filter of the second node; The fourth processing module is used to obtain a second sample data set corresponding to the updated filter of the second node; send the first position index of the second node to the first node, so that the filter of the first node filters out the sample data corresponding to the first position index, and obtain the updated filter of the first node; obtain the first sample data set corresponding to the updated filter of the first node; the first position index represents the position in the filter of the second node that is not mapped by the sample data; The second training module is used to jointly train the second sub-model and the third sub-model based on the first sample data set and the second sample data set to obtain a first model, wherein the first model is composed of the second sub-model deployed on the first node and the third sub-model deployed on the second node.

18. A data processing device, characterized in that: The device comprises: An acquisition module is used to acquire data to be processed; A fifth processing module is used to process the data to be processed using a trained first model, wherein the first model is trained based on the training method described in any one of claims 1 to 7 or any one of claims 8 to 14.

19. An electronic device, characterized in that: The electronic device comprises a processor and a memory for storing a computer program that can be run on the processor; wherein, The processor is configured to run the computer program to perform the method of any one of claims 1 to 7, or any one of claims 8 to 14, or claim 15.

20. A computer storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7, or any one of claims 8 to 14, or claim 15 is implemented.

21. A computer program product, comprising a computer program, characterized in that The computer program implements the method of any one of claims 1 to 7, any one of claims 8 to 14, or claim 15 when executed by a processor.

Images