A federated learning method, device, central server and data terminal

Abstract

The present invention provides a federated learning method, apparatus, central server, and data terminal. The federated learning method comprises: performing a fusion calculation on first training model parameters sent by multiple data terminals to obtain a first global model parameter, and sending the first global model parameter and a first threshold to each data terminal; receiving a second training model parameter recalculated by each data terminal in a second round of federated learning to obtain a second global model parameter; and determining the second global model parameter as a converged global model parameter when the difference between the second global model parameter and the first global model parameter is less than a preset value. The present invention sets a threshold so that the data terminal decides whether to participate in the next round of federated learning based on the corresponding threshold. This can eliminate abnormal data terminals, thereby ensuring the final model effect of federated learning while reducing communication overhead.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence, and in particular to a federated learning method, device, central server and data terminal. Background Art

[0002] Federated learning involves users jointly training a global model without sharing their data, effectively addressing data privacy and security concerns. Typically, the implementation of a federated learning algorithm requires the data client to send complete model parameter updates to a central server during each global model training cycle. As existing models increasingly employ complex deep neural networks, these require multiple rounds of iterative updates, resulting in significant communication overhead. Especially for large global models, network bandwidth limitations and the increasing number of data client nodes can exacerbate the communication bottleneck in federated learning. Improving training efficiency and reducing communication costs while maintaining model performance has become a hot research topic in federated learning.

[0003] Currently, methods for addressing the communication overhead of federated learning typically include reducing the number of communications, reducing the content of communication transmissions, optimizing algorithms, and optimizing networks. Existing methods for addressing communication overhead primarily include the following: 1. Model compression: During federated model training, participants can first define the structure of the model to be transmitted. Model transmission is then performed only according to this defined structure, such as by quantizing and encoding the model. Model compression can effectively reduce the amount of parameter transmission, improve the training efficiency of federated learning, and, to a certain extent, protect the model's original parameters from being leaked, thereby enhancing model security. 2. Reducing the number of communications: This typically involves reducing the frequency of client feedback communications, requiring clients to perform local updates and periodically send updated information to the central server at intervals or after a fixed number of local model iterations to aggregate the model. This periodic averaging scheme reduces the number of communications between the central server and clients, thereby reducing the overall communication cost of training the model.

[0004] The first method mentioned above speeds up communication by compressing the model and transmitting content. This approach often sacrifices model accuracy and is often affected by network latency and interruptions, failing to address the primary issue. The second method, which reduces communication frequency by reducing the number of communications, often involves performing multiple rounds of model transmission locally. This method is highly likely to cause local convergence, impacting the final model performance.

[0005] Therefore, it is necessary to design a federated learning method to reduce communication overhead while ensuring the final model effect. Summary of the Invention

[0006] The embodiments of the present invention provide a federated learning method, device, central server and data terminal to solve the problem in the prior art that it is impossible to both ensure the final model effect of federated learning and reduce communication overhead.

[0007] In order to solve the above technical problems, the embodiments of the present invention provide the following technical solutions:

[0008] In a first aspect, an embodiment of the present invention provides a federated learning method, applied to a central server, the method comprising:

[0009] Receiving a model training result sent by each of the multiple data terminals; the model training result includes a first training model parameter;

[0010] Performing a fusion calculation on the plurality of first training model parameters to obtain global model parameters for a first round of federated learning, and sending the first global model parameters and a first threshold for the first round of federated learning to each of the data terminals;

[0011] Receiving second training model parameters recalculated based on the first global model parameters of the first round of federated learning and the amount of data on the data end, and performing a fusion calculation on the second training model parameters to obtain second global model parameters for the second round of federated learning; wherein whether to participate in the second round of federated learning is determined by the data end based on the first global model parameters, the first training model parameters, and the first threshold of the first round of federated learning;

[0012] When the difference between the second global model parameter and the first global model parameter is smaller than a preset value, the second global model parameter is determined to be a converged global model parameter.

[0013] Optionally, the method further includes:

[0014] When the difference between the second global model parameter and the first global model parameter is greater than or equal to the preset value, sending the second global model parameter and the second threshold of the second round of federated learning to the data end participating in the second round of federated learning;

[0015] Receiving third training model parameters sent by each data end participating in the third round of federated learning, which are recalculated based on the second global model parameters of the second round of federated learning and the amount of data on the data end, and performing a fusion calculation on the third training model parameters to obtain third global model parameters for the third round of federated learning; wherein whether to participate in the third round of federated learning is determined by the data end based on the second global model parameters, the second training model parameters, and the second threshold of the second round of federated learning;

[0016] When the difference between the third global model parameter and the second global model parameter is smaller than the preset value, the third global model parameter is determined to be the converged global model parameter.

[0017] Optionally, both the first threshold and the second threshold are determined according to a round of federated learning.

[0018] Optionally, the model training result further includes: a first training model parameter sending timestamp, data end data volume, and physical machine information;

[0019] Receiving second training model parameters recalculated based on the first global model parameters and the data amount of the data end in the first round of federated learning, sent by each data end participating in the second round of federated learning, including:

[0020] During the waiting time of the second round of federated learning, receive the second training model parameters recalculated based on the first global model parameters and the data amount of the data terminals in the first round of federated learning, sent by each data terminal participating in the second round of federated learning;

[0021] Among them, the waiting time of the second round of federated learning is determined according to the first training model parameter sending timestamp corresponding to the data end participating in the first round of federated learning, the first training model parameter reaching timestamp, the data amount of the data end, and the physical machine information of the data end.

[0022] Optionally, the method further includes:

[0023] If the second training model parameters sent by the first data end are not received within the waiting time of the second round of federated learning, and the second training model parameters sent by the first data end are received within the waiting time of the Nth round of federated learning, consistency calculation is performed on the second training model parameters and the global model parameters obtained in the (N-1)th round of federated learning to obtain a first alignment degree;

[0024] When the first alignment degree is less than a threshold corresponding to the Nth round of federated learning, performing a fusion calculation on the second training model parameters sent by the first data end and the second training model parameters sent by each data end participating in the Nth round of federated learning to obtain global model parameters for the Nth round of federated learning;

[0025] Sending the global model parameters and corresponding thresholds of the Nth round of federated learning to each data end participating in the Nth round of federated learning and the first data end; wherein the first data end is one of the data ends participating in the second round of federated learning;

[0026] N is a positive integer greater than or equal to 3.

[0027] Optionally, the method further includes:

[0028] Determine the feedback duration corresponding to each data end according to the first training model parameter arrival timestamp, the first training model parameter sending timestamp, the data end data volume, and the data end physical machine information corresponding to each data end participating in the first round of federated learning;

[0029] Determine the maximum duration of the feedback durations corresponding to each of the data terminals as the waiting duration for the second round of federated learning.

[0030] In a second aspect, an embodiment of the present invention further provides a federated learning method, applied to a data end, the method comprising:

[0031] Performing initial model training and sending model training results to a central server; the model training results include first training model parameters;

[0032] Receiving a first global model parameter and a first threshold of a first round of federated learning of the central server sent by the central server;

[0033] When participation in the second round of federated learning is determined based on the first global model parameters, the first training model parameters and the first threshold of the first round of federated learning, the received first global model parameters and the amount of data on the data end are recalculated to obtain the second training model parameters and sent to the central server so that the central server can obtain the converged global model parameters.

[0034] Optionally, when determining to participate in the second round of federated learning based on the first global model parameters, the first training model parameters, and the first threshold of the first round of federated learning, before recalculating the received first global model parameters and the amount of data on the data end to obtain the second training model parameters and sending them to the central server, the method further includes:

[0035] Performing consistency calculation on the first global model parameters and the first training model parameters to obtain a second alignment;

[0036] When the second alignment degree is greater than the first threshold, it is determined to participate in a second round of federated learning.

[0037] In a third aspect, an embodiment of the present invention further provides a federated learning device, applied to a central server, comprising:

[0038] A first receiving module is configured to receive a model training result sent by each of a plurality of data terminals; the model training result includes a first training model parameter;

[0039] a first processing module, configured to perform a fusion calculation on the plurality of first training model parameters to obtain global model parameters for a first round of federated learning, and send the first global model parameters and a first threshold for the first round of federated learning to each of the data terminals;

[0040] A second processing module is configured to receive, from each data end participating in the second round of federated learning, second training model parameters recalculated based on the first global model parameters of the first round of federated learning and the amount of data on the data end, and to perform a fusion calculation on the second training model parameters to obtain second global model parameters for the second round of federated learning; wherein whether to participate in the second round of federated learning is determined by the data end based on the first global model parameters, the first training model parameters, and the first threshold of the first round of federated learning;

[0041] The first determining module is configured to determine that the second global model parameter is a converged global model parameter when a difference between the second global model parameter and the first global model parameter is less than a preset value.

[0042] Optionally, the device further comprises:

[0043] a first sending module, configured to send the second global model parameter and the second threshold of the second round of federated learning to the data terminal participating in the second round of federated learning when the difference between the second global model parameter and the first global model parameter is greater than or equal to the preset value;

[0044] a parameter receiving module, configured to receive, from each data end participating in the third round of federated learning, third training model parameters recalculated based on the second global model parameters of the second round of federated learning and the amount of data on the data end, and to perform a fusion calculation on the third training model parameters to obtain third global model parameters for the third round of federated learning; wherein whether to participate in the third round of federated learning is determined by the data end based on the second global model parameters, the second training model parameters, and the second threshold of the second round of federated learning;

[0045] The second determining module is configured to determine that the third global model parameter is the converged global model parameter when a difference between the third global model parameter and the second global model parameter is less than a preset value.

[0046] Optionally, both the first threshold and the second threshold are determined according to a round of federated learning.

[0047] Optionally, the model training result further includes: a first training model parameter sending timestamp, data end data volume, and physical machine information;

[0048] The second processing module includes:

[0049] A first receiving unit is configured to receive, within a waiting period of the second round of federated learning, second training model parameters recalculated based on the first global model parameters of the first round of federated learning and the amount of data on the data terminals, sent by each data terminal participating in the second round of federated learning;

[0050] Among them, the waiting time of the second round of federated learning is determined according to the first training model parameter sending timestamp corresponding to the data end participating in the first round of federated learning, the first training model parameter reaching timestamp, the data amount of the data end, and the physical machine information of the data end.

[0051] Optionally, the second processing module further includes:

[0052] The first processing unit is configured to, if the second training model parameters sent by the first data end are not received within the waiting period of the second round of federated learning, and the second training model parameters sent by the first data end are received within the waiting period of the Nth round of federated learning, perform consistency calculation on the second training model parameters with the global model parameters obtained in the (N-1)th round of federated learning to obtain a first alignment;

[0053] a second processing unit, configured to, when the first alignment degree is less than a threshold corresponding to the Nth round of federated learning, fuse the second training model parameters sent by the first data end with the second training model parameters sent by each data end participating in the Nth round of federated learning to obtain global model parameters for the Nth round of federated learning;

[0054] A first sending unit, configured to send the global model parameters and corresponding thresholds of the Nth round of federated learning to each data terminal participating in the Nth round of federated learning and the first data terminal;

[0055] The first data end is one of the data ends participating in the second round of federated learning;

[0056] N is a positive integer greater than or equal to 3.

[0057] Optionally, the second processing module further includes:

[0058] The first determining unit is used to

[0059] Determine the feedback duration corresponding to each data end according to the first training model parameter arrival timestamp, the first training model parameter sending timestamp, the data end data volume, and the data end physical machine information corresponding to each data end participating in the first round of federated learning;

[0060] The second determining unit is configured to determine a maximum duration among the feedback durations corresponding to each of the data terminals as a waiting duration for a second round of federated learning.

[0061] In a fourth aspect, an embodiment of the present invention further provides a federated learning device, applied to a data end, comprising:

[0062] A third processing module is used to perform initial model training and send the model training results to the central server; the model training results include the first training model parameters;

[0063] A second receiving module is configured to receive the first global model parameter and the first threshold of the first round of federated learning of the central server sent by the central server;

[0064] The fourth processing module is used to recalculate the received first global model parameters and the amount of data on the data end to obtain the second training model parameters and send them to the central server when determining to participate in the second round of federated learning based on the first global model parameters, the first training model parameters and the first threshold of the first round of federated learning, so that the central server can obtain the converged global model parameters.

[0065] Optionally, the device further comprises:

[0066] a fifth processing module, configured to perform consistency calculation on the first global model parameters and the first training model parameters to obtain a second alignment degree;

[0067] The third determining module is configured to determine whether to participate in a second round of federated learning if the second alignment degree is greater than the first threshold.

[0068] In a fifth aspect, an embodiment of the present invention further provides a central server comprising: a processor, a memory, and a program stored on the memory and executable on the processor, wherein when the program is executed by the processor, the steps of the federated learning method as described in any one of the first aspects are implemented.

[0069] In a sixth aspect, an embodiment of the present invention further provides a data terminal, comprising: a processor, a memory, and a program stored on the memory and runnable on the processor, wherein when the program is executed by the processor, the steps of the federated learning method as described in any one of the second aspects are implemented.

[0070] In the seventh aspect, an embodiment of the present invention further provides a readable storage medium, on which a program is stored. When the program is executed by a processor, the steps of the federated learning method as described in any one of the first aspects are implemented, or the steps of the federated learning method as described in any one of the second aspects are implemented.

[0071] The beneficial effects of the present invention are:

[0072] The solution of the present invention sets a threshold in each round of federated learning, so that the data end determines whether to participate in the next round of federated learning based on the global model parameters, training model parameters and corresponding thresholds of the previous round of federated learning. Data end nodes with anomalies or inconsistent with the overall data situation can be excluded from the data end, thereby ensuring the final model effect of federated learning and reducing communication overhead. BRIEF DESCRIPTION OF THE DRAWINGS

[0073] Figure 1 A flowchart showing a federated learning method applied to a central server provided by an embodiment of the present invention;

[0074] Figure 2 A flowchart showing a federated learning method applied to a data end provided by an embodiment of the present invention;

[0075] Figure 3 A flowchart showing the first round of federated learning provided by an embodiment of the present invention;

[0076] Figure 4 A flowchart showing a federated learning cyclic training according to an embodiment of the present invention;

[0077] Figure 5 A flowchart showing a federated learning method provided by an embodiment of the present invention;

[0078] Figure 6 A schematic diagram showing the structure of a federated learning device provided by an embodiment of the present invention;

[0079] Figure 7 A schematic diagram showing the structure of a federated learning device provided by an embodiment of the present invention;

[0080] Figure 8 A schematic diagram showing the structure of a central server provided in an embodiment of the present invention;

[0081] Figure 9 A schematic diagram showing the structure of a data terminal provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0082] In order to make the objectives, technical solutions and advantages of the present invention more clear, the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0083] The present invention addresses the problem in the prior art that it is impossible to both ensure the final model effect of federated learning and reduce communication overhead, and provides a federated learning method, device, central server and data terminal.

[0084] like Figure 1 As shown, an embodiment of the present invention provides a federated learning method, which is applied to a central server. The method includes:

[0085] Step 101: Receive a model training result sent by each of a plurality of data terminals; the model training result includes a first training model parameter.

[0086] It should be noted that the central server provided in the embodiment of the present invention is connected to multiple data terminals.

[0087] The process of the first round of federated learning at the data end is as follows: each data end performs initial model training locally and obtains its own local model training results, which include the first training model parameters G i 1 , where i is the i-th data terminal.

[0088] After each data terminal obtains its own local model training results, it sends the first training model parameter G in its own model training results to the central server. i 1 In the first round of federated learning, the central server receives the first training model parameter G sent by each data end in turn. i 1 .

[0089] Step 102: Perform fusion calculation on multiple first training model parameters to obtain first global model parameters of the first round of federated learning, and send the first global model parameters and first threshold of the first round of federated learning to each of the data terminals.

[0090] During the first round of federated learning, the central server receives the first training model parameters G sent by each data end in turn. i 1 After that, all the first training model parameters G i 1 Perform integrated calculations to obtain the first global model parameter G of the first round of federated learning 1 The first global model parameter G is sent to each data terminal participating in the first round of federated learning. 1 And the initial threshold V1 (first threshold). The first threshold is determined by the central server, so that each data terminal can determine whether to participate in the second round of federated learning based on the first threshold.

[0091] The function of the global model parameters is:

[0092] G j =F(G1 j ,G2 j ,...,G n j )

[0093] Among them, j is the jth federated learning round, and the global model parameters are determined by the training model parameters G1 of each data end j ,G2j ,...,G n j The fusion calculation shows that the first global model parameter in the first round is G 1 The first global model parameters are calculated by fusing the first training model parameters of each data end.

[0094] Step 103: Receive second training model parameters recalculated based on the first global model parameters of the first round of federated learning and the data amount of the data end, and perform a fusion calculation on the second training model parameters to obtain second global model parameters for the second round of federated learning; wherein, whether to participate in the second round of federated learning is determined by the data end based on the first global model parameters, the first training model parameters, and the first threshold of the first round of federated learning.

[0095] It should be noted that whether to participate in the second round of federated learning is determined by the data end based on the first global model parameters, first training model parameters and first threshold of the first round of federated learning. In the third round of federated learning and subsequent rounds of federated learning, whether to participate in the next round of federated learning is determined by the data end based on the global model parameters, training model parameters and corresponding thresholds of the previous round of federated learning.

[0096] Specifically, after the first round of federated learning, the data end and the central server respectively perform federated learning cycle training. At the data end, each data end participating in the first round of federated learning receives the first global model parameter G 1 , the first global model parameter G of the first round of federated learning 1 With the first training model parameters G i 1 Perform consistency calculation to obtain the second alignment, and compare the second alignment with the initial threshold V1. If the second alignment of a data end is less than the initial threshold V1, it is considered that there is a large difference between the data of the local data end and the data of other data ends, which can be regarded as abnormal data. Therefore, in the second round of federated learning, the data end does not need to feedback the updated second training model parameters to the central server; if the second alignment of a data end is greater than the initial threshold V1, it is considered that the data of the local data end and the data of other data ends are similar in overall distribution, and the data situation is similar, so it should continue to participate in the second round of federated learning. Each data end participating in the second round of federated learning combines the first global model parameter G 1 With the local data volume m i (data end data volume) is recalculated to obtain the second training model parameter G updated in the second round of federated learning i 2 , and update the second training model parameters G i2 The central server receives the updated second training model parameter G sent by the data terminal participating in the second round of federated learning. i 2 .

[0097] The consistency calculation function used by the data end to calculate the consistency between the global model parameters and the training model parameters is:

[0098]

[0099] Where N is the total number of parameters, r j The consistency calculation function works by counting how many parameters in two updates have the same sign and normalizing the result by the total number of parameters N. The percentage of parameters with the same sign in the two updates is calculated, and this percentage is used to measure the alignment of the local updates.

[0100] The consistency calculation function provided by the embodiment of the present invention can effectively determine whether the data end updates to follow the writing direction or are just outliers, and compare parameter signs to directly measure the consistency with the global model parameters without considering the learning rate and the size of the local data set.

[0101] It should also be noted that in the entire federated learning process, one federated learning round starts from calculating the training model parameters on the local data end and ends when the central server feeds back the global model parameters to each data end.

[0102] The central server receives the updated second training model parameter G sent by the data terminal participating in the second round of federated learning i 2 , update the second training model parameters G sent by all data ends i 2 Perform fusion calculation to obtain the second global model parameter G of the second round of federated learning 2 The second global model parameter G is sent to the data end participating in the second round of federated learning. 2 , and the data terminal and the central server repeat the corresponding steps in step 103 respectively.

[0103] In order to reduce the number of federated learning, an embodiment of the present invention adds a consistency calculation function for judging the global model parameters and the training model parameters at the data end. If the inconsistency is extremely large, the data at the data end is regarded as abnormal data, so that the data end does not continue to participate in federated learning, which can reduce the number of federated learning communications. Specifically, the embodiment of the present invention calculates and counts how many parameters have the same sign in the two updates, and normalizes the results by the total number of parameters, calculates the percentage of parameters with the same sign in the two updates, measures the consistency of the local update with the global update (the second alignment) based on the percentage, and compares it with the threshold to determine whether it needs to be proposed, thereby excluding data sets that are not consistent with the overall data situation and reducing unnecessary communications.

[0104] Step 104: When the difference between the second global model parameter and the first global model parameter is smaller than a preset value, determine that the second global model parameter is a converged global model parameter.

[0105] It should be noted that after each round of federated learning, the central server needs to calculate whether the difference between the global model parameters of this round of federated learning and the global model parameters of the previous round of federated learning is less than the preset value, that is, whether the obtained global model parameters are converged. If the difference is less than the preset value (global model parameters converge), the federated learning is terminated. That is, when the difference between the second global model parameter and the first global model parameter is less than the preset value, it is determined that the second global model parameter converges, and the second global model parameter is the converged global model parameter.

[0106] In an embodiment of the present invention, by setting a threshold in each round of federated learning, the data end determines whether to participate in the next round of federated learning based on the global model parameters, training model parameters and corresponding thresholds of the previous round of federated learning. Data end nodes that are abnormal or inconsistent with the overall data situation can be excluded from the data end, thereby ensuring the final model effect of federated learning and reducing communication overhead.

[0107] Optionally, the method further includes:

[0108] When the difference between the second global model parameter and the first global model parameter is greater than or equal to the preset value, sending the second global model parameter and the second threshold of the second round of federated learning to the data end participating in the second round of federated learning;

[0109] Receiving third training model parameters sent by each data end participating in the third round of federated learning, which are recalculated based on the second global model parameters of the second round of federated learning and the amount of data on the data end, and performing a fusion calculation on the third training model parameters to obtain third global model parameters for the third round of federated learning; wherein whether to participate in the third round of federated learning is determined by the data end based on the second global model parameters, the second training model parameters, and the second threshold of the second round of federated learning;

[0110] When the difference between the third global model parameter and the second global model parameter is smaller than the preset value, the third global model parameter is determined to be the converged global model parameter.

[0111] It should also be noted that if, after each round of federated learning, the central server calculates that the difference between the global model parameters of this round of federated learning and the global model parameters of the previous round of federated learning is greater than or equal to the preset value, that is, the obtained global model parameters do not converge, then the global model parameters and corresponding thresholds of this round of federated learning are sent to the data end participating in this round of federated learning, and the next round of federated learning is continued, that is, step 103 is repeated until the global model parameters converge.

[0112] Specifically, when the difference between the second global model parameter and the first global model parameter is greater than or equal to the preset value, the second global model parameter does not converge, then the second global model parameter and the second threshold of the second round of federated learning are sent to the data end participating in the second round of federated learning, and the data end obtains the second alignment of the second round of federated learning based on the second global model parameter and the second training model parameter. If the second alignment is greater than the second threshold, it is determined to participate in the third round of federated learning, and the data end recalculates the third training model parameter based on the second global model parameter of the second round of federated learning and the data amount of the data end, and performs a fusion calculation on the third training model parameter to obtain the third global model parameter of the third round of federated learning. If the difference between the third global model parameter and the second global model parameter is less than the preset value, the federated learning is terminated, and the third global model parameter is determined to be a converged global model parameter. Otherwise, the fourth round of federated learning is continued until the global model parameter converges.

[0113] As a preferred embodiment, the first threshold and the second threshold are both determined according to rounds of federated learning.

[0114] That is, the first threshold is determined according to the learning round of the first round of federated learning, and the second threshold is determined according to the learning round of the second round of federated learning.

[0115] Specifically, the corresponding threshold for each round of federated learning is determined by the central server according to the following threshold function model:

[0116]

[0117] Where j is the jth federated learning round. In the first round of federated learning, the threshold is set to 0.5. As the number of federated learning rounds increases, the threshold gradually increases.

[0118] Preferably, the model training result further includes: a first training model parameter sending timestamp, data end data volume, and physical machine information;

[0119] Receiving second training model parameters recalculated based on the first global model parameters and the data amount of the data end in the first round of federated learning, sent by each data end participating in the second round of federated learning, including:

[0120] During the waiting time of the second round of federated learning, receive the second training model parameters recalculated based on the first global model parameters and the data amount of the data terminals in the first round of federated learning, sent by each data terminal participating in the second round of federated learning;

[0121] Among them, the waiting time of the second round of federated learning is determined according to the first training model parameter sending timestamp corresponding to the data end participating in the first round of federated learning, the first training model parameter reaching timestamp, the data amount of the data end, and the physical machine information of the data end.

[0122] That is, the model training results also include: the training model parameters sending timestamp t i 、Data volume on the data end m i And physical machine information H i In each round of federated learning, the central server sends a timestamp t according to the training model parameters sent by each data terminal. i , training model parameters reaches timestamp T i 、Data volume on the data end m i And the data end physical machine information H i Calculate and measure the feedback time S of the updated training model parameters at each data end i , according to the feedback time S of the updated training model parameters of each data end i , determine the waiting time, and in the next round of federated learning, collect the updated training model parameters sent by each data terminal participating in the next round of federated learning within the waiting time, and then continue to fuse them in the process of this round of federated learning to obtain the global model parameters of this round of federated learning.

[0123] Exemplarily, during the second round of federated learning, the waiting time for the second round of federated learning is determined based on the first training model parameter sending timestamp corresponding to the data end participating in the first round of federated learning, the first training model parameter arrival timestamp, the data end data volume, and the data end physical machine information. During the waiting time for the second round of federated learning, the first global model parameters and the data end data volume sent by each data end participating in the second round of federated learning are received to recalculate the second training model parameters.

[0124] Furthermore, the method further comprises:

[0125] If the second training model parameters sent by the first data end are not received within the waiting time of the second round of federated learning, and the second training model parameters sent by the first data end are received within the waiting time of the Nth round of federated learning, consistency calculation is performed on the second training model parameters and the global model parameters obtained in the (N-1)th round of federated learning to obtain a first alignment degree;

[0126] When the first alignment degree is less than a threshold corresponding to the Nth round of federated learning, performing a fusion calculation on the second training model parameters sent by the first data end and the second training model parameters sent by each data end participating in the Nth round of federated learning to obtain global model parameters for the Nth round of federated learning;

[0127] Sending the global model parameters and corresponding thresholds of the Nth round of federated learning to each data end participating in the Nth round of federated learning and the first data end; wherein the first data end is one of the data ends participating in the second round of federated learning;

[0128] N is a positive integer greater than or equal to 3.

[0129] It should be noted that during a round of federated learning, if the data end returns the updated training model parameters to the central server within the waiting time S, the signal of the data end is considered normal. The central server receives the updated training model parameters during this round of federated learning, and performs fusion calculations on the updated training model parameters to obtain the global model parameters for this round of federated learning, and sends the global model parameters to the data end; if the data end does not return the updated training model parameters to the central server within the waiting time S, the signal of the data end is considered to be interrupted. During this round of federated learning, the central server does not send the global model parameters to the data end after obtaining the global model parameters for this round of federated learning.

[0130] Furthermore, during a round of federated learning, if the central server receives updated training model parameters sent by the data end whose signal is interrupted, the central server compares the alignment of the updated training model parameters with the already integrated global model parameters to obtain a first alignment. If the first alignment is less than the corresponding threshold of this round of federated learning, the updated training model parameters sent by the data end are integrated and calculated, and the global model parameters of this round of federated learning are fed back to the data end. This is regarded as the data end being reconnected during the next round of federated learning, that is, the communication of the data end is restored.

[0131] For example, during the second round of federated learning, if the central server does not receive the updated second training model parameters sent by the first data terminal within the waiting time of the second round of federated learning, and receives the second training model parameters sent by the first data terminal within the waiting time corresponding to the fourth round (N is 4) of federated learning, the central server calculates the consistency of the second training model parameters sent by the first data terminal with the global model parameters of the third round of federated learning to obtain a first alignment. If the first alignment is less than the corresponding threshold V4 of the fourth round of federated learning, the second training model parameters sent by the first data terminal are fused with the updated training model parameters sent by each other data terminal participating in the fourth round of federated learning to obtain the global model parameters of the fourth round of federated learning. The global model parameters and the corresponding threshold V4 are sent to each data terminal participating in the fourth round of federated learning and the first data terminal, which is regarded as the re-access of the first data terminal.

[0132] As a preferred embodiment, the method further includes:

[0133] Determine the feedback duration corresponding to each data end according to the first training model parameter arrival timestamp, the first training model parameter sending timestamp, the data end data volume, and the data end physical machine information corresponding to each data end participating in the first round of federated learning;

[0134] Determine the maximum duration of the feedback durations corresponding to each of the data terminals as the waiting duration for the second round of federated learning.

[0135] That is, the central server determines the feedback duration S corresponding to each data end based on the training model parameter arrival timestamp, training model parameter sending timestamp, data end data volume, and data end physical machine information sent by each data end received in this round. i , select the feedback time S of each data end iThe largest value is the waiting time S for the next round of federated learning. During the next round of federated learning, the central server collects the updated training model parameters sent by each data terminal participating in the next round of federated learning within this waiting time S. That is, the waiting time S represents the deadline for the central server to collect the updated training model parameters fed back by each data terminal in subsequent rounds of federated learning.

[0136] Specifically, during the first round of federated learning on the central server, the central server receives the model training results sent by each data terminal in turn, and sends the timestamp t according to the first training model parameters of each data terminal. i , the first training model parameters reach timestamp T i And the data end physical machine information H of each data end i 、Data volume on the data end m i Calculate and measure the updated second training model parameter feedback time S of each data end i , determine the updated second training model parameter feedback time S of each data end i The largest value is the waiting time S.

[0137] The formula for the feedback duration of each data terminal is as follows:

[0138] S i =T i -t i +f(m i +M i ),i=1,...,n

[0139] Among them, n is the number of data terminals in the first round of federated learning, f(m i +M i ) is the model calculation time, S i The feedback duration of each data terminal.

[0140] Based on the feedback time of each data terminal, the formula for determining the waiting time is as follows:

[0141] S=max[S1,S2,...,S n ]

[0142] Among them, S1, S2, ..., S n is the feedback time of each data end, and S is the waiting time.

[0143] In an embodiment of the present invention, the central server time waiting mechanism eliminates data terminals that are in the process of communication or communication interruption, thereby reducing unnecessary communication feedback and improving the learning efficiency of federated learning. In addition, a federated learning re-admission mechanism is added to the central server to restore data terminal nodes that have communication delays and are helpful to the global model. Specifically, by measuring the model calculation time and transmission time of the data terminal, the maximum waiting time is obtained. Data terminals that exceed the maximum waiting time are not allowed to participate in this round of federated learning, and data terminal nodes with communication delays are eliminated, solving the problem of long waiting time for communication during the federated learning process. In addition, a re-access mechanism for data terminal nodes is also added. If the data terminal node has slow feedback due to communication reasons, the central server can perform model parameter consistency calculation when receiving the feedback from the data terminal to obtain the first alignment. If the first alignment is greater than the current threshold, the communication of the data terminal is restored and the global model parameters are fed back to the data terminal node.

[0144] like Figure 2 As shown, an embodiment of the present invention further provides a federated learning method, which is applied to a data end, and the method includes:

[0145] Step 201: Perform initial model training and send the model training results to the central server; the model training results include first training model parameters.

[0146] It should be noted that the central server is connected to multiple data terminals, and each data terminal executes the federated learning method in the embodiment of the present invention.

[0147] During the first round of federated learning, the data end first performs initial model training to obtain its own local model training results, which include the first training model parameters. After each data end obtains its own local first model training results, it sends the first training model parameters in its own model training results to the central server.

[0148] Step 202: Receive the first global model parameter and the first threshold of the first round of federated learning of the central server sent by the central server.

[0149] During the first round of federated learning, the central server sequentially receives the first training model parameters from each data terminal, integrates and calculates all the first training model parameters, and obtains the first global model parameters for the first round of federated learning. The central server then sends these first global model parameters and an initial threshold (the first threshold) to each data terminal participating in the first round of federated learning. The initial threshold is determined by the central server so that each data terminal can subsequently determine whether to participate in the next round of federated learning based on the initial threshold.

[0150] It should be noted that the model training results also include the first training model parameter sending timestamp, the data volume of the data end, and the physical machine information of the data end. The central server determines the waiting time based on the first training model parameter arrival timestamp, the first training model parameter sending timestamp, the data volume of the data end, and the physical machine information of the data end. The waiting time indicates the deadline for the central server to collect the updated training model parameters fed back by each data end in subsequent rounds of federated learning.

[0151] In an embodiment of the present invention, a threshold is set in each round of federated learning, so that the data end decides whether to participate in the next round of federated learning based on the corresponding threshold of the previous round of federated learning. This can eliminate data end nodes with abnormalities or data that is inconsistent with the overall data, thereby ensuring the final model effect of federated learning while reducing communication overhead.

[0152] The following combination Figure 3 , taking the central server connecting to two data terminals (data terminal 1 and data terminal 2) as an example, the process of the first round of federated learning is explained.

[0153] Step 1: Data end 1 and data end 2 perform initial model training respectively to obtain model training results, which include training model parameters (first training model parameters), training model parameter sending timestamp (first training model parameter sending timestamp), data end data volume and data end physical machine information; Step 2: Data end 1 and data end 2 send model training results to the central server respectively; Step 3: The central server calculates the waiting time based on the training model parameter sending timestamp, data end data volume, data end physical machine information and training model parameter arrival timestamp in the training model results, integrates the training model parameters, obtains global model parameters (first global model parameters), and calculates the initial threshold (first threshold); Step 4: The central server sends the global model parameters and initial threshold to data end 1 and data end 2 respectively.

[0154] Step 203: When participation in the second round of federated learning is determined based on the first global model parameters, the first training model parameters, and the first threshold of the first round of federated learning, the received first global model parameters and the amount of data on the data end are recalculated to obtain the second training model parameters and sent to the central server so that the central server can obtain the converged global model parameters.

[0155] That is, after receiving the global model parameters of the previous round of federated learning, the data end determines whether to participate in the next round of federated learning based on the global model parameters, training model parameters and corresponding thresholds. If it is determined to participate in the next round of federated learning, the updated training model parameters are recalculated based on the global model parameters and the amount of data on the data end, and the updated training model parameters are sent to the central server so that the central server can determine whether to proceed with the next round of federated learning. The central server needs to calculate whether the difference between the global model parameters of this round of federated learning and the global model parameters of the previous round of federated learning is less than the preset value, that is, whether the obtained global model parameters converge. If the difference is less than the preset value (global model parameters converge), the federated learning is terminated. If the difference is greater than or equal to the preset value (global model parameters do not converge), the global model parameters and corresponding thresholds of this round of federated learning are sent to the data end participating in this round of federated learning, and the next round of federated learning is continued. The above steps are repeated, that is, federated learning cycle training is performed until the global model parameters converge.

[0156] Exemplarily, when participating in the second round of federated learning is determined based on the first global model parameters, the first training model parameters and the first threshold, the received first global model parameters and the amount of data on the data end are recalculated to obtain the second training model parameters and sent to the central server. The central server obtains the second global model parameters based on the second training model parameters. If the central server determines that the difference between the second global model parameters and the first global model parameters is less than the preset value, the federated learning is terminated and the second global model parameters are determined to be the converged global model parameters. If the central server determines that the difference between the second global model parameters and the first global model parameters is greater than or equal to the preset value, the third round of federated learning is continued until the global model parameters converge.

[0157] As a preferred embodiment,

[0158] When determining to participate in the second round of federated learning based on the first global model parameters, the first training model parameters, and the first threshold of the first round of federated learning, before recalculating the received first global model parameters and the amount of data on the data end to obtain the second training model parameters and sending the second training model parameters to the central server, the method further includes:

[0159] Performing consistency calculation on the first global model parameters and the first training model parameters to obtain a second alignment;

[0160] When the second alignment degree is greater than the first threshold, it is determined to participate in a second round of federated learning.

[0161] Specifically, after receiving the global model parameters of the first round of federated learning, the data end determines whether to participate in the next round of federated learning based on the first global model parameters, the first training model parameters and the first threshold. The process is as follows: the first global model parameters of the first round of federated learning are calculated to be consistent with the first training model parameters to obtain a second alignment, and the second alignment is compared with the first threshold. If the second alignment of a data end is less than the first threshold, it is considered that there is a large difference between the data of the local data end and the data of other data ends, which can be regarded as abnormal data. Therefore, in the next round of federated learning, the data end does not need to feedback the updated training model parameters to the central server, that is, it does not participate in subsequent federated learning. If the second alignment of a data end is greater than the first threshold, it is considered that the data of the local data end is similar to the overall distribution of the data of other data ends, and the data situation is similar, so it should continue to participate in the next round of federated learning.

[0162] The following combination Figure 4 , taking the central server connected to two data terminals (data terminal 1 and data terminal 2) as an example, the process of federated learning loop training is explained.

[0163] Step 1: Data end 1 and data end 2 perform consistency calculation locally respectively to obtain the second alignment degree, and compare it with the corresponding threshold (first threshold) to decide whether to return the locally updated training model parameters (second training model parameters); Step 2: Data end 1 and data end 2 send the updated training model parameters to the central server respectively; Step 3: The central server integrates the updated training model parameters sent by data end 1 and data end 2 during the waiting time to obtain the global model parameters (second global model parameters), and calculates the corresponding threshold (second threshold) of this round of federated learning; Step 4: Send the global model parameters (second global model parameters) and the corresponding threshold (second threshold); The central server calculates whether the difference between the global model parameters of this round of federated learning and the global model parameters of the previous round of federated learning (first global model parameters) is less than the preset value. If the difference is less than the preset value (global model parameters converge), the federated learning is terminated. If the difference is greater than or equal to the preset value, the above steps are repeated until the global model parameters converge.

[0164] The following combination Figure 5 , taking the central server connecting to two data terminals (data terminal 1 and data terminal 2) as an example, the process of the federated learning method is explained in detail.

[0165] Data end 1 and data end 2 calculate the training model parameters respectively; the central server calculates the initial threshold; data end 1 and data end 2 send the training model parameters, training model parameter sending timestamp, data end data volume and data end physical machine information to the central server respectively; the central server integrates the training model parameters, calculates the global model parameters, and calculates the waiting time based on the training model parameter arrival timestamp, training model parameter sending timestamp, data end data volume and data end physical machine information; the central server feeds back the global model parameters and the corresponding threshold to data end 1 and data end 2; data end 1 and data end 2 calculate the alignment (second alignment) based on the global model parameters and local training model parameters respectively , and determines whether the second alignment is greater than the corresponding threshold. If so, the updated training model parameters are sent to the central server; the central server only collects the updated training model parameters sent by each data end within the waiting time, calculates the updated training model parameters, obtains the global model parameters, and determines whether the global model parameters of this round of federated learning converge. If so, the federated learning ends, otherwise returns to the step of "the central server integrates the training model parameters, calculates the global model parameters, and calculates the waiting time according to the training model parameter arrival timestamp, the training model parameter sending timestamp, the data end data volume and the data end physical machine information" to perform cyclic training until the global model parameters converge.

[0166] It should be noted that all descriptions about the central server in the above embodiments are applicable to the embodiments of the data end and can achieve the same technical effects.

[0167] like Figure 6 As shown, an embodiment of the present invention further provides a federated learning device, which is applied to a central server, and the device includes:

[0168] A first receiving module 601 is configured to receive a model training result sent by each of a plurality of data terminals; the model training result includes a first training model parameter;

[0169] A first processing module 602 is configured to perform a fusion calculation on the plurality of first training model parameters to obtain global model parameters for a first round of federated learning, and send the first global model parameters and a first threshold for the first round of federated learning to each of the data terminals;

[0170] A second processing module 603 is configured to receive, from each data end participating in the second round of federated learning, second training model parameters recalculated based on the first global model parameters of the first round of federated learning and the amount of data on the data end, and perform a fusion calculation on the second training model parameters to obtain second global model parameters for the second round of federated learning. Whether to participate in the second round of federated learning is determined by the data end based on the first global model parameters, the first training model parameters, and the first threshold of the first round of federated learning.

[0171] The first determining module 604 is configured to determine that the second global model parameter is a converged global model parameter if the difference between the second global model parameter and the first global model parameter is less than a preset value.

[0172] In an embodiment of the present invention, by setting a threshold in each round of federated learning, the data end determines whether to participate in the next round of federated learning based on the global model parameters, training model parameters and corresponding thresholds of the previous round of federated learning. Data end nodes that are abnormal or inconsistent with the overall data situation can be excluded from the data end, thereby ensuring the final model effect of federated learning and reducing communication overhead.

[0173] Optionally, the device further comprises:

[0174] a first sending module, configured to send the second global model parameter and the second threshold of the second round of federated learning to the data terminal participating in the second round of federated learning when the difference between the second global model parameter and the first global model parameter is greater than or equal to the preset value;

[0175] a parameter receiving module, configured to receive, from each data end participating in the third round of federated learning, third training model parameters recalculated based on the second global model parameters of the second round of federated learning and the amount of data on the data end, and to perform a fusion calculation on the third training model parameters to obtain third global model parameters for the third round of federated learning; wherein whether to participate in the third round of federated learning is determined by the data end based on the second global model parameters, the second training model parameters, and the second threshold of the second round of federated learning;

[0176] The second determining module is configured to determine that the third global model parameter is the converged global model parameter when a difference between the third global model parameter and the second global model parameter is less than a preset value.

[0177] Optionally, both the first threshold and the second threshold are determined according to a round of federated learning.

[0178] Optionally, the model training result further includes: a first training model parameter sending timestamp, data end data volume, and physical machine information;

[0179] The second processing module 603 includes:

[0180] A first receiving unit is configured to receive, within a waiting period of the second round of federated learning, second training model parameters recalculated based on the first global model parameters of the first round of federated learning and the amount of data on the data terminals, sent by each data terminal participating in the second round of federated learning;

[0181] Among them, the waiting time of the second round of federated learning is determined according to the first training model parameter sending timestamp corresponding to the data end participating in the first round of federated learning, the first training model parameter reaching timestamp, the data amount of the data end, and the physical machine information of the data end.

[0182] Optionally, the second processing module 603 further includes:

[0183] The first processing unit is configured to, if the second training model parameters sent by the first data end are not received within the waiting period of the second round of federated learning, and the second training model parameters sent by the first data end are received within the waiting period of the Nth round of federated learning, perform consistency calculation on the second training model parameters with the global model parameters obtained in the (N-1)th round of federated learning to obtain a first alignment;

[0184] a second processing unit, configured to, when the first alignment degree is less than a threshold corresponding to the Nth round of federated learning, fuse the second training model parameters sent by the first data end with the second training model parameters sent by each data end participating in the Nth round of federated learning to obtain global model parameters for the Nth round of federated learning;

[0185] A first sending unit, configured to send the global model parameters and corresponding thresholds of the Nth round of federated learning to each data terminal participating in the Nth round of federated learning and the first data terminal;

[0186] The first data end is one of the data ends participating in the second round of federated learning;

[0187] N is a positive integer greater than or equal to 3.

[0188] Optionally, the second processing module 603 further includes:

[0189] A first determining unit is configured to determine a feedback duration corresponding to each data end participating in the first round of federated learning based on a first training model parameter arrival timestamp, a first training model parameter sending timestamp, a data end data volume, and data end physical machine information.

[0190] The second determining unit is configured to determine a maximum duration among the feedback durations corresponding to each of the data terminals as a waiting duration for a second round of federated learning.

[0191] It should be noted that the federated learning device provided in the embodiment of the present invention is a device capable of executing the above-mentioned federated learning method applied to the central server. All embodiments of the above-mentioned federated learning method applied to the central server are applicable to the device and can achieve the same or similar technical effects.

[0192] like Figure 7 As shown, an embodiment of the present invention further provides a federated learning device, which is applied to a data end, and the device includes:

[0193] The third processing module 701 is used to perform initial model training and send the model training results to the central server; the model training results include the first training model parameters;

[0194] A second receiving module 702 is configured to receive the first global model parameter and the first threshold of the first round of federated learning of the central server sent by the central server;

[0195] The fourth processing module 703 is used to recalculate the received first global model parameters and the amount of data on the data end to obtain the second training model parameters and send them to the central server when determining to participate in the second round of federated learning based on the first global model parameters, the first training model parameters and the first threshold of the first round of federated learning, so that the central server can obtain the converged global model parameters.

[0196] In an embodiment of the present invention, a threshold is set in each round of federated learning, so that the data end decides whether to participate in the next round of federated learning based on the corresponding threshold of the previous round of federated learning. This can eliminate data end nodes with abnormalities or data that is inconsistent with the overall data, thereby ensuring the final model effect of federated learning while reducing communication overhead.

[0197] Optionally, the device further comprises:

[0198] a fifth processing module, configured to perform consistency calculation on the first global model parameters and the first training model parameters to obtain a second alignment degree;

[0199] The third determining module is configured to determine whether to participate in a second round of federated learning if the second alignment degree is greater than the first threshold.

[0200] It should be noted that the federated learning device provided in the embodiment of the present invention is a device capable of executing the above-mentioned federated learning method applied to the data end. All embodiments of the above-mentioned federated learning method applied to the data end are applicable to the device and can achieve the same or similar technical effects.

[0201] like Figure 8 As shown, an embodiment of the present invention also provides a central server, including: a processor 800; and a memory 810 connected to the processor 800 through a bus interface, the memory 810 is used to store programs and data used by the processor 800 when performing operations, and the processor 800 calls and executes the programs and data stored in the memory 810.

[0202] The central server further includes a transceiver 820, which is connected to the bus interface and is configured to receive and send data under the control of the processor 800. Specifically, the processor 800 calls and executes the programs and data stored in the memory 810, and the transceiver 820 performs the following processes:

[0203] Receive a model training result sent by each of a plurality of data terminals; the model training result includes a first training model parameter.

[0204] The processor 800 performs the following processes:

[0205] Performing a fusion calculation on the plurality of first training model parameters to obtain global model parameters for a first round of federated learning, and sending the first global model parameters and a first threshold for the first round of federated learning to each of the data terminals;

[0206] Receiving second training model parameters recalculated based on the first global model parameters of the first round of federated learning and the amount of data on the data end, and performing a fusion calculation on the second training model parameters to obtain second global model parameters for the second round of federated learning; wherein whether to participate in the second round of federated learning is determined by the data end based on the first global model parameters, the first training model parameters, and the first threshold of the first round of federated learning;

[0207] When the difference between the second global model parameter and the first global model parameter is smaller than a preset value, the second global model parameter is determined to be a converged global model parameter.

[0208] Optionally, the transceiver 820 is further configured to:

[0209] When the difference between the second global model parameter and the first global model parameter is greater than or equal to the preset value, the second global model parameter and the second threshold of the second round of federated learning are sent to the data end participating in the second round of federated learning.

[0210] The processor 800 is further configured to:

[0211] Receiving third training model parameters sent by each data end participating in the third round of federated learning, which are recalculated based on the second global model parameters of the second round of federated learning and the amount of data on the data end, and performing a fusion calculation on the third training model parameters to obtain third global model parameters for the third round of federated learning; wherein whether to participate in the third round of federated learning is determined by the data end based on the second global model parameters, the second training model parameters, and the second threshold of the second round of federated learning;

[0212] When the difference between the third global model parameter and the second global model parameter is smaller than the preset value, the third global model parameter is determined to be the converged global model parameter.

[0213] Optionally, both the first threshold and the second threshold are determined according to a round of federated learning.

[0214] Optionally, the model training result further includes: a first training model parameter sending timestamp, data end data volume, and physical machine information;

[0215] The transceiver 820 is specifically configured to:

[0216] During the waiting time of the second round of federated learning, receive the second training model parameters recalculated based on the first global model parameters and the data amount of the data terminals in the first round of federated learning, sent by each data terminal participating in the second round of federated learning;

[0217] Among them, the waiting time of the second round of federated learning is determined according to the first training model parameter sending timestamp corresponding to the data end participating in the first round of federated learning, the first training model parameter reaching timestamp, the data amount of the data end, and the physical machine information of the data end.

[0218] Optionally, the processor 800 is specifically configured to:

[0219] If the second training model parameters sent by the first data end are not received within the waiting time of the second round of federated learning, and the second training model parameters sent by the first data end are received within the waiting time of the Nth round of federated learning, consistency calculation is performed on the second training model parameters and the global model parameters obtained in the (N-1)th round of federated learning to obtain a first alignment degree;

[0220] When the first alignment degree is less than a threshold corresponding to the Nth round of federated learning, performing a fusion calculation on the second training model parameters sent by the first data end and the second training model parameters sent by each data end participating in the Nth round of federated learning to obtain global model parameters for the Nth round of federated learning;

[0221] Sending the global model parameters and corresponding thresholds of the Nth round of federated learning to each data end participating in the Nth round of federated learning and the first data end; wherein the first data end is one of the data ends participating in the second round of federated learning;

[0222] N is a positive integer greater than or equal to 3.

[0223] Optionally, the processor 800 is specifically configured to:

[0224] Determine the feedback duration corresponding to each data end according to the first training model parameter arrival timestamp, the first training model parameter sending timestamp, the data end data volume, and the data end physical machine information corresponding to each data end participating in the first round of federated learning;

[0225] Determine the maximum duration of the feedback durations corresponding to each of the data terminals as the waiting duration for the second round of federated learning.

[0226] Among them, Figure 8 In the embodiment, the bus architecture may include any number of interconnected buses and bridges, specifically linking together various circuits of one or more processors represented by processor 800 and memory represented by memory 810. The bus architecture may also link together various other circuits such as peripheral devices, voltage regulators, and power management circuits, which are all well known in the art and, therefore, will not be described further herein. The bus interface provides a user interface 830. The transceiver 820 may be a plurality of components, i.e., a transmitter and a receiver, providing a unit for communicating with various other devices on a transmission medium. The processor 800 is responsible for managing the bus architecture and general processing, and the memory 810 may store data used by the processor 800 when performing operations.

[0227] like Figure 9 As shown, an embodiment of the present invention also provides a data terminal, including: a processor 900; and a memory 910 connected to the processor 900 through a bus interface, the memory 910 is used to store programs and data used by the processor 900 when performing operations, and the processor 900 calls and executes the programs and data stored in the memory 910.

[0228] The data terminal further includes a transceiver 920, which is connected to the bus interface and is configured to receive and send data under the control of the processor 900. Specifically, the processor 900 calls and executes the programs and data stored in the memory 910. The processor 900 performs the following processes:

[0229] Perform initial model training and send the model training results to the central server; the model training results include first training model parameters.

[0230] The transceiver 920 performs the following processes:

[0231] The processor 900 receives the first global model parameter and the first threshold of the first round of federated learning of the central server sent by the central server. The processor 900 performs the following process:

[0232] When participation in the second round of federated learning is determined based on the first global model parameters, the first training model parameters and the first threshold of the first round of federated learning, the received first global model parameters and the amount of data on the data end are recalculated to obtain the second training model parameters and sent to the central server so that the central server can obtain the converged global model parameters.

[0233] Optionally, when the processor 900 determines to participate in the second round of federated learning based on the first global model parameters, the first training model parameters, and the first threshold of the first round of federated learning, before recalculating the received first global model parameters and the amount of data on the data end to obtain the second training model parameters and sending them to the central server, the processor 900 is further configured to:

[0234] Performing consistency calculation on the first global model parameters and the first training model parameters to obtain a second alignment;

[0235] When the second alignment degree is greater than the first threshold, it is determined to participate in a second round of federated learning.

[0236] Among them, Figure 9 In the embodiment, the bus architecture may include any number of interconnected buses and bridges, specifically linking together various circuits of one or more processors represented by processor 900 and memory represented by memory 910. The bus architecture may also link together various other circuits such as peripheral devices, voltage regulators, and power management circuits, which are all well known in the art and, therefore, will not be described further herein. The bus interface provides a user interface 930. The transceiver 920 may be a plurality of components, i.e., a transmitter and a receiver, providing a unit for communicating with various other devices on a transmission medium. The processor 900 is responsible for managing the bus architecture and general processing, and the memory 910 may store data used by the processor 900 when performing operations.

[0237] In addition, a specific embodiment of the present invention further provides a readable storage medium having a computer program stored thereon, wherein when the program is executed by a processor, the steps in any one of the above-described federated learning methods are implemented.

[0238] In the several embodiments provided in this application, it should be understood that the disclosed methods and devices can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection of some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0239] In addition, the functional units in various embodiments of the present invention may be integrated into a single processing unit, each unit may be physically included separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or hardware plus software functional units.

[0240] The above-mentioned integrated unit implemented in the form of a software functional unit can be stored in a computer-readable storage medium. The above-mentioned software functional unit is stored in a storage medium and includes a number of instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) to perform some of the steps of the sending and receiving methods described in various embodiments of the present invention. The aforementioned storage medium includes: a USB flash drive, a mobile hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, an optical disk, and other media that can store program code.

[0241] The above is a preferred embodiment of the present invention. It should be pointed out that for ordinary personnel in this technical field, several improvements and modifications can be made without departing from the principles described in the present invention. These improvements and modifications are also within the scope of protection of the present invention.

Claims

1. A federated learning method, characterized in that: Applied to a central server, the method includes: Receiving a model training result sent by each of the multiple data terminals; the model training result includes a first training model parameter; Performing a fusion calculation on the plurality of first training model parameters to obtain global model parameters for a first round of federated learning, and sending the first global model parameters and a first threshold for the first round of federated learning to each of the data terminals; Receiving second training model parameters recalculated based on the first global model parameters of the first round of federated learning and the amount of data on the data end, and performing a fusion calculation on the second training model parameters to obtain second global model parameters for the second round of federated learning; wherein whether to participate in the second round of federated learning is determined by the data end based on the first global model parameters, the first training model parameters, and the first threshold of the first round of federated learning; When a difference between the second global model parameter and the first global model parameter is less than a preset value, determining that the second global model parameter is a converged global model parameter; The model training result also includes: the first training model parameter sending timestamp, the data volume of the data end, and the physical machine information; Receiving second training model parameters recalculated based on the first global model parameters and the data amount of the data end in the first round of federated learning, sent by each data end participating in the second round of federated learning, including: During the waiting time of the second round of federated learning, receive the second training model parameters recalculated based on the first global model parameters and the data amount of the data terminals in the first round of federated learning, sent by each data terminal participating in the second round of federated learning; Among them, the waiting time of the second round of federated learning is determined according to the first training model parameter sending timestamp corresponding to the data end participating in the first round of federated learning, the first training model parameter reaching timestamp, the data amount of the data end, and the physical machine information of the data end.

2. The federated learning method according to claim 1, characterized in that: The method further comprises: When the difference between the second global model parameter and the first global model parameter is greater than or equal to the preset value, sending the second global model parameter and the second threshold of the second round of federated learning to the data end participating in the second round of federated learning; Receiving third training model parameters sent by each data end participating in the third round of federated learning, which are recalculated based on the second global model parameters of the second round of federated learning and the amount of data on the data end, and performing a fusion calculation on the third training model parameters to obtain third global model parameters for the third round of federated learning; wherein whether to participate in the third round of federated learning is determined by the data end based on the second global model parameters, the second training model parameters, and the second threshold of the second round of federated learning; When the difference between the third global model parameter and the second global model parameter is smaller than the preset value, the third global model parameter is determined to be the converged global model parameter.

3. The federated learning method according to claim 2, characterized in that: The first threshold and the second threshold are both determined according to a round of federated learning.

4. The federated learning method according to claim 1, wherein: The method further comprises: If the second training model parameters sent by the first data end are not received within the waiting time of the second round of federated learning, and the second training model parameters sent by the first data end are received within the waiting time of the Nth round of federated learning, consistency calculation is performed on the second training model parameters and the global model parameters obtained in the (N-1)th round of federated learning to obtain a first alignment degree; When the first alignment degree is less than a threshold corresponding to the Nth round of federated learning, performing a fusion calculation on the second training model parameters sent by the first data end and the second training model parameters sent by each data end participating in the Nth round of federated learning to obtain global model parameters for the Nth round of federated learning; Sending the global model parameters and corresponding thresholds of the Nth round of federated learning to each data end participating in the Nth round of federated learning and the first data end; wherein the first data end is one of the data ends participating in the second round of federated learning; N is a positive integer greater than or equal to 3.

5. The federated learning method according to claim 1, wherein: The method further comprises: Determine the feedback duration corresponding to each data end according to the first training model parameter arrival timestamp, the first training model parameter sending timestamp, the data end data volume, and the data end physical machine information corresponding to each data end participating in the first round of federated learning; Determine the maximum duration of the feedback durations corresponding to each of the data terminals as the waiting duration for the second round of federated learning.

6. A federated learning method, characterized in that: Applied to the data end, the method includes: Performing initial model training and sending model training results to a central server; the model training results include first training model parameters; Receiving a first global model parameter and a first threshold of a first round of federated learning of the central server sent by the central server; When it is determined to participate in the second round of federated learning based on the first global model parameters, the first training model parameters, and the first threshold of the first round of federated learning, recalculate the received first global model parameters and the amount of data on the data end to obtain second training model parameters and send them to the central server, so that the central server obtains converged global model parameters; The model training result also includes: the first training model parameter sending timestamp, the data volume of the data end, and the physical machine information; Recalculating the received first global model parameters and the data amount of the data end to obtain second training model parameters and sending them to the central server, including: Recalculate the second training model parameters based on the first global model parameters and the amount of data on the data end in the first round of federated learning; During the waiting time of the second round of federated learning, sending the second training model parameters to the central server; Among them, the waiting time of the second round of federated learning is determined according to the first training model parameter sending timestamp corresponding to the data end participating in the first round of federated learning, the first training model parameter reaching timestamp, the data amount of the data end, and the physical machine information of the data end.

7. The federated learning method according to claim 6, characterized in that: When determining to participate in the second round of federated learning based on the first global model parameters, the first training model parameters, and the first threshold of the first round of federated learning, before recalculating the received first global model parameters and the amount of data on the data end to obtain the second training model parameters and sending the second training model parameters to the central server, the method further includes: Performing consistency calculation on the first global model parameters and the first training model parameters to obtain a second alignment; When the second alignment degree is greater than the first threshold, it is determined to participate in a second round of federated learning.

8. A federated learning device, characterized in that: Applied to a central server, the device includes: A first receiving module is configured to receive a model training result sent by each of a plurality of data terminals; the model training result includes a first training model parameter; a first processing module, configured to perform a fusion calculation on the plurality of first training model parameters to obtain global model parameters for a first round of federated learning, and send the first global model parameters and a first threshold for the first round of federated learning to each of the data terminals; A second processing module is configured to receive, from each data end participating in the second round of federated learning, second training model parameters recalculated based on the first global model parameters of the first round of federated learning and the amount of data on the data end, and to perform a fusion calculation on the second training model parameters to obtain second global model parameters for the second round of federated learning; wherein whether to participate in the second round of federated learning is determined by the data end based on the first global model parameters, the first training model parameters, and the first threshold of the first round of federated learning; a first determining module, configured to determine, when a difference between the second global model parameter and the first global model parameter is less than a preset value, that the second global model parameter is a converged global model parameter; The model training result also includes: the first training model parameter sending timestamp, the data volume of the data end, and the physical machine information; The second processing module includes: A first receiving unit is configured to receive, within a waiting period of the second round of federated learning, second training model parameters recalculated based on the first global model parameters of the first round of federated learning and the amount of data on the data terminals, sent by each data terminal participating in the second round of federated learning; Among them, the waiting time of the second round of federated learning is determined according to the first training model parameter sending timestamp corresponding to the data end participating in the first round of federated learning, the first training model parameter reaching timestamp, the data amount of the data end, and the physical machine information of the data end.

9. A federated learning device, characterized in that: Applied to a data terminal, the device includes: A third processing module is used to perform initial model training and send the model training results to the central server; the model training results include the first training model parameters; A second receiving module is configured to receive the first global model parameter and the first threshold of the first round of federated learning of the central server sent by the central server; a fourth processing module, configured to, when determining to participate in a second round of federated learning based on the first global model parameters, the first training model parameters, and the first threshold value of the first round of federated learning, recalculate the received first global model parameters and the amount of data on the data end to obtain second training model parameters, and send the second training model parameters to the central server, so that the central server can obtain converged global model parameters; The model training result also includes: the first training model parameter sending timestamp, the data volume of the data end, and the physical machine information; The fourth processing module is configured to: Recalculate the second training model parameters based on the first global model parameters and the amount of data on the data end in the first round of federated learning; During the waiting time of the second round of federated learning, sending the second training model parameters to the central server; Among them, the waiting time of the second round of federated learning is determined according to the first training model parameter sending timestamp corresponding to the data end participating in the first round of federated learning, the first training model parameter reaching timestamp, the data amount of the data end, and the physical machine information of the data end.

10. A central server, characterized in that: include: A processor, a memory, and a program stored in the memory and executable on the processor, wherein when the program is executed by the processor, the steps of the federated learning method according to any one of claims 1 to 5 are implemented.

11. A data terminal, characterized in that: include: A processor, a memory, and a program stored in the memory and executable on the processor, wherein when the program is executed by the processor, the steps of the federated learning method according to claim 6 or 7 are implemented.

12. A readable storage medium, characterized in that: The readable storage medium stores a program, and when the program is executed by the processor, the program implements the steps of the federated learning method according to any one of claims 1 to 5, or the steps of the federated learning method according to claim 6 or 7.

Images