Cross-island federated learning model training method based on decentralized gradient aggregation

Abstract

The present application discloses a cross-island federated learning model training method based on decentralized gradient aggregation, which is applied to local servers and belongs to the field of data model training, including: when the local training round lags behind the global training round, the training gradient data is filtered to obtain honest training gradient data, and the local data model is trained according to the configuration of the local data model based on the honest training gradient data; based on the updated training gradient data, an update application is initiated to an external server so that the external server is updated, and the local training round is updated according to the update result; when the obtained aggregation application reaches a preset number, the global training round and training gradient data are updated. While eliminating the central server, the present application improves the security, reliability and training efficiency of cross-island federated learning model training, and solves the problems of low model training security and high network overhead.

Description

Technical Field

[0001] The present application belongs to the field of data model training, and specifically relates to a cross-island federated learning model training method, device, equipment and storage medium based on decentralized gradient aggregation. Background Art

[0002] Researching the sharing and integration mechanism of medical and health data has become a hot topic in current research. With the rapid development of artificial intelligence and big data technology, Federated Learning (FL) as an emerging distributed machine learning method has gradually attracted attention. It allows participating nodes to jointly train a global model without sharing original data, which greatly protects data privacy.

[0003] Existing decentralized federated learning methods mainly use consensus algorithms to dynamically elect leader nodes for model training. In some methods, blockchain technology is also used to maintain training gradients and achieve global consistency by storing historical gradient sequences on the chain, thereby ensuring data transparency and traceability.

[0004] However, the above method relies on the elected leader node to coordinate the gradient aggregation and model update of each participating node, which is vulnerable to attacks or malicious behavior. At the same time, the method of using blockchain technology to maintain training gradients ensures data privacy and transparency, but also brings problems of storage and network overhead. Summary of the invention

[0005] The present application aims to provide a cross-island federated learning model training method, apparatus, device and storage medium based on decentralized gradient aggregation, at least to solve the problems of insufficient data reliability and high storage and network overhead when training cross-island federated learning models.

[0006] In a first aspect, an embodiment of the present application discloses a cross-island federated learning model training method based on decentralized gradient aggregation, which is applied to a local server in a cross-island federated learning network, where a local data model to be trained is loaded on the local server, including:

[0007] In the case where the local training round of the local data model lags behind the global training round of the cross-island federated learning network, multiple training gradient data stored in the local server are filtered to obtain multiple honest training gradient data for the local data model, and the local data model is configured according to the multiple honest training gradient data, and the local data model is trained to obtain updated training gradient data of the data model of the local server under the local training round; the multiple training gradient data stored in the local server correspond one-to-one to the multiple servers in the cross-island federated learning network;

[0008] According to the updated training gradient data, an update application is initiated to a foreign server in the cross-island federated learning network, so that the foreign server receiving the update application updates the target training gradient data according to the received update application, and updates the local training round according to the update result fed back by the foreign server; the target training gradient data is the training gradient data corresponding to the local server stored in the training gradient data of each foreign server; the training gradient data stored in the foreign server corresponds one-to-one to multiple servers in the cross-island federated learning network; the update result is fed back by the foreign server according to the update application;

[0009] In response to update requests sent from multiple external servers, aggregate requests corresponding to the multiple external servers are obtained, and when the aggregate requests reach a preset number, the global training rounds and the training gradient data stored in the local server are updated; each of the updated training gradient data corresponds one-to-one to the external server that sends each of the update requests.

[0010] In a second aspect, the embodiment of the present application further discloses a cross-island federated learning model training device based on decentralized gradient aggregation, which is applied to a local server in a cross-island federated learning network, and the local data model to be trained is loaded on the local server, including:

[0011] A filtering module, configured to filter the plurality of training gradient data stored in the local server to obtain a plurality of honest training gradient data for the local data model when the local training round of the local data model lags behind the global training round of the cross-island federated learning network, and to train the local data model according to the plurality of honest training gradient data to obtain updated training gradient data of the data model of the local server under the local training round; the plurality of training gradient data stored in the local server corresponds one-to-one to the plurality of servers in the cross-island federated learning network;

[0012] An update module is used to initiate an update application to a foreign server in the cross-island federated learning network according to the updated training gradient data, so that the foreign server receiving the update application updates the target training gradient data according to the received update application, and updates the local training round according to the update result fed back by the foreign server; the target training gradient data is the training gradient data corresponding to the local server stored in the training gradient data of each foreign server; the training gradient data stored in the foreign server corresponds one-to-one to multiple servers in the cross-island federated learning network; the update result is fed back by the foreign server according to the update application;

[0013] An aggregation module is used to respond to update applications sent from multiple external servers, obtain aggregation applications corresponding to the multiple external servers, and when the aggregation applications reach a preset number, update the global training rounds and the training gradient data stored in the local server; each of the updated training gradient data corresponds one-to-one to the external server that sends each update application.

[0014] In a third aspect, an embodiment of the present application further discloses an electronic device, comprising a processor and a memory, wherein the memory stores programs or instructions that can be run on the processor, and when the program or instructions are executed by the processor, the steps of the method described in the first aspect are implemented.

[0015] In a fourth aspect, an embodiment of the present application further discloses a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the steps of the method described in the first aspect are implemented.

[0016] In summary, in the embodiment of the present application, by screening the obtained honest training gradient data, it is ensured that even if there are malicious nodes, the data used by the data model of the local server is still credible, thereby improving the data reliability of model training; and then the local data model can be configured and trained according to the filtered honest training gradient data to obtain updated local training gradient data, so as to ensure that each node performs effective model training locally, avoiding the negative impact of untrusted gradients on the model, and further enhancing the accuracy and robustness of the model; finally, through the synchronization and update of gradient data, the consistency and synchronization of gradient data in the entire network are ensured, the potential model errors and security risks caused by inconsistent data are avoided, and all nodes are ensured to be synchronized in the same training round, maintaining the overall coordination and consistency of the system, and effectively reducing the problems of slow model convergence and low accuracy caused by asynchronous training. Therefore, based on the method of the embodiment of the present application, while eliminating the central server, the decentralized gradient aggregation and adaptive gradient filtering technology are used to improve the security, data reliability and training efficiency of cross-island federated learning model training, and solve the problems of insufficient data reliability and high storage and network overhead in the model training process. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In the attached picture:

[0018] Figure 1 It is a flowchart of the steps of a cross-island federated learning model training method based on decentralized gradient aggregation provided in an embodiment of the present application;

[0019] Figure 2 It is a flowchart of the steps of another cross-island federated learning model training method based on decentralized gradient aggregation provided in an embodiment of the present application;

[0020] Figure 3 It is a schematic diagram of a training gradient update process in a global round according to the method provided in an embodiment of the present application;

[0021] Figure 4 It is a complete training process according to the method provided in the embodiment of the present application;

[0022] Figure 5 It is a block diagram of a cross-island federated learning model training device based on decentralized gradient aggregation provided in an embodiment of the present application;

[0023] Figure 6 is a block diagram of an electronic device according to an embodiment of the present application;

[0024] Figure 7 It is a block diagram of an electronic device of another embodiment provided by the embodiments of the present application. DETAILED DESCRIPTION

[0025] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0026] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0027] Figure 1 A cross-island federated learning model training method based on decentralized gradient aggregation is provided in an embodiment of the present application and is applied to a local server in a cross-island federated learning network, where a local data model to be trained is loaded on the local server.

[0028] The method may include the following steps:

[0029] Step 101, when the local training round of the local data model lags behind the global training round of the cross-island federated learning network, multiple training gradient data stored in the local server are filtered to obtain multiple honest training gradient data for the local data model, and the local data model is configured according to the multiple honest training gradient data, and the local data model is trained to obtain updated training gradient data of the data model of the local server under the local training round.

[0030] Among them, multiple training gradient data stored in the local server correspond one-to-one to multiple servers in the cross-island federated learning network.

[0031] In some embodiments of the present application, in order to ensure the reliability of model training data, when the local training round of the local data model lags behind the global training round of the cross-island federated learning network, it is necessary to filter the multiple training gradient data stored in the local server to retain the honest training gradient data. That is, when the local training round lags behind the global training round, the training gradient data stored in the local server is filtered to remove untrusted gradients and only retain honest training gradient data. Honest training gradient data refers to training data generated by non-malicious nodes. Through this process, the credibility of the data can be effectively improved, making the data used by local nodes more reliable, which helps to improve the accuracy and robustness of model training.

[0032] For example, in the training process of a healthcare data analysis system, suppose that the local training round of server A in a federated learning network lags behind the global training round. Server A stores multiple training gradient data collected from different servers. Users can process these training gradient data through filtering algorithms, such as the AdaKrum algorithm, to filter out the training gradient data generated by honest servers. The filtered data is used to train the local model of server A.

[0033] Step 102, based on the updated training gradient data, initiates an update application to the foreign server in the cross-island federated learning network, so that the foreign server that receives the update application updates the target training gradient data according to the received update application, and updates the local training rounds according to the update result fed back by the foreign server.

[0034] Among them, the target training gradient data is the training gradient data stored in each foreign server, which corresponds to the local server; the training gradient data stored in the foreign server corresponds one-to-one to multiple servers in the cross-island federated learning network; the update result is fed back by the foreign server based on the update application.

[0035] In some embodiments of the present application, in order to ensure data consistency between the data model of the local server and other data models in the cross-island federated learning network, it is necessary to initiate an update application to the foreign data model based on the updated training gradient data. That is, based on the updated training gradient data of the local data model, an update application is initiated to the foreign data model in the cross-island federated learning network, so that the foreign data model that receives the update application updates the training gradient data corresponding to the local server stored in the foreign data model according to the update application. An update application refers to a local server requesting a foreign server to synchronize its training gradient data, and updating the local training rounds based on the update results fed back by the foreign server to the update application. Through this process, data consistency between servers is ensured, thereby avoiding potential model errors caused by data inconsistency and improving the accuracy and stability of the overall model.

[0036] For example, in the training process of a medical health data analysis system, suppose there are three servers A, B, and C in the network, and server A obtains updated training gradient data in local training. Server A initiates an update request to server B and server C based on this data. After receiving the update request, server B and server C update their respective stored training gradient data corresponding to server A. After the update is completed, server B and server C feed back the update results to server A. Based on these feedbacks, server A updates its local training rounds.

[0037] Step 103, in response to update requests sent from multiple external servers, obtain corresponding aggregation requests sent from multiple external servers, and when the aggregation requests reach a preset number, update the global training rounds and the training gradient data stored in the local server.

[0038] Among them, each updated training gradient data corresponds one-to-one to the foreign server that sends each update application.

[0039] In some embodiments of the present application, in order to ensure the synchronization and consistency of global training rounds, it is necessary to update the global training rounds and the training gradient data stored in the local server according to the number of aggregation applications received from external servers. The aggregation application is further obtained based on the update application received from the external server. The global training round refers to the training progress followed by all servers in the entire federated learning network. This process ensures that all servers are synchronously trained in the same training round, thereby maintaining the overall coordination and consistency of the system, and effectively reducing the problems of slow model convergence and low accuracy caused by asynchronous training.

[0040] For example, in the training process of a medical health data analysis system, suppose that in a cross-island federated learning network, there are several servers participating in model training. After server A receives update applications from server B and server C, it reaches the preset number. At this time, server A will update its stored training gradient data according to these applications and adjust the global training round. After the update is completed, server A will feed back the update results to server B and server C. Through this process, server A, server B, and server C are trained synchronously in the same training round, ensuring the consistency and synchronization of training data, and improving the convergence speed and accuracy of the overall model.

[0041] In summary, in the embodiment of the present application, by screening the obtained honest training gradient data, it is ensured that even if there are malicious nodes, the data used by the data model of the local server is still credible, thereby improving the data reliability of model training; and then the local data model can be configured and trained according to the filtered honest training gradient data to obtain updated local training gradient data, so as to ensure that each node performs effective model training locally, avoiding the negative impact of untrusted gradients on the model, and further enhancing the accuracy and robustness of the model; finally, through the synchronization and update of gradient data, the consistency and synchronization of gradient data in the entire network are ensured, the potential model errors and security risks caused by inconsistent data are avoided, and all nodes are ensured to be synchronized in the same training round, maintaining the overall coordination and consistency of the system, and effectively reducing the problems of slow model convergence and low accuracy caused by asynchronous training. Therefore, based on the method of the embodiment of the present application, while eliminating the central server, the decentralized gradient aggregation and adaptive gradient filtering technology are used to improve the security, data reliability and training efficiency of cross-island federated learning model training, and solve the problems of insufficient data reliability and high storage and network overhead in the model training process.

[0042] Figure 2 Another cross-island federated learning model training method based on decentralized gradient aggregation provided in an embodiment of the present application is applied to a local server in a cross-island federated learning network, and a local data model to be trained is loaded on the local server.

[0043] The method may include the following steps:

[0044] Step 201, obtain a set of randomly generated initialization weights, and initialize the local data model of the local server according to the initialization weights.

[0045] In some embodiments of the present application, in order to ensure the consistency of each data model before training begins, a set of randomly generated initialization weights will be obtained, and the data model of the local server will be initialized according to the initialization weights. Initialization weights refer to the initial values ​​set for model parameters at the beginning of deep learning model training. Through this process, it is ensured that all data models have the same initial state before training begins, which helps to unify the training starting point of the model and improve the consistency and comparability of model training.

[0046] For example, in the training process of a medical health data analysis system, a cross-island federated learning network contains servers A, B, and C. Before the training starts, each server will obtain the same set of randomly generated initialization weights. The user initializes the data model of server A and server B based on these initialization weights. After the initialization is completed, the model parameters of server A and server B have the same initial values. Through this process, it is ensured that server A and server B have the same starting point before the training starts, which helps the consistency and efficiency of the subsequent training process.

[0047] Step 202, when the local training round of the local data model lags behind the global training round of the cross-island federated learning network, multiple training gradient data stored in the local server are filtered to obtain multiple honest training gradient data for the local data model, and the local data model is configured according to the multiple honest training gradient data, and the local data model is trained to obtain updated training gradient data of the data model of the local server under the local training round.

[0048] Among them, multiple training gradient data stored in the local server correspond one-to-one to multiple servers in the cross-island federated learning network.

[0049] The method shown in this step has been explained in step 101 and will not be repeated here.

[0050] Optionally, in order to filter multiple training gradient data stored in the local server to obtain multiple honest training gradient data for the local data model, and configure the local data model according to the multiple honest training gradient data, train the local data model to obtain updated training gradient data of the data model of the local server under the local training round, step 202 includes the following sub-steps:

[0051] Sub-step 2021 , filtering the plurality of training gradient data stored in the local server according to the Euclidean distance between every two training gradient data, so as to select the remaining training gradient data after filtering as the honest training gradient data.

[0052] In some embodiments of the present application, in order to filter out credible gradient data from multiple training gradient data, it is necessary to filter multiple training gradient data stored in a local server according to the Euclidean distance between each two training gradient data. Euclidean distance refers to the straight-line distance between two points in Euclidean space. Through this process, those training gradient data with large deviations and unreliable can be effectively filtered out, thereby ensuring that the retained data is more reliable and accurate, which helps to improve the overall effect of subsequent model training.

[0053] For example, during the training process of a medical data analysis system, server A in a federated learning network stores multiple training gradient data from different servers. The user uses the Euclidean distance formula to calculate the distance between each two training gradient data, and then filters out those data with larger distances based on these distance values, and selects the remaining training gradient data as honest training gradient data. Through this process, server A retains more reliable and accurate training data, thus laying a solid foundation for subsequent model training.

[0054] Optionally, sub-step 2021 includes the following sub-steps:

[0055] Sub-step 20211, calculates the Euclidean distance between every two training gradient data among the multiple training gradient data stored in the local server, and performs minimum value filtering on the calculated Euclidean distance according to a preset number of samples to obtain a minimum Euclidean distance group corresponding to each training gradient data.

[0056] The minimum Euclidean distance group includes multiple Euclidean distances, each of which corresponds one-to-one to a training gradient data other than the reference training gradient data; the reference training gradient data is the training gradient data corresponding to the minimum Euclidean distance group.

[0057] In some embodiments of the present application, in order to screen out the minimum distance combination between each training gradient data to improve the credibility of the data, it is necessary to calculate the Euclidean distance between each two training gradient data in the multiple training gradient data stored in the local server, and perform minimum value filtering on the calculated Euclidean distance according to the preset number of samples to obtain the minimum Euclidean distance group corresponding to each training gradient data. Euclidean distance refers to the straight-line distance between two points in Euclidean space. Through this process, other training gradient data closest to the reference training gradient data can be effectively screened out, thereby ensuring that the selected data is more reliable and consistent, which helps to improve the accuracy and stability of subsequent model training.

[0058] For example, during the training process of a medical health data analysis system, server A in a federated learning network stores multiple training gradient data from different servers. After calculating the Euclidean distance between each two data in these training gradient data, these distance values ​​can be filtered for the minimum value according to the preset number of samples to obtain the minimum Euclidean distance group corresponding to each training gradient data. Through this process, other training gradient data closest to the reference training gradient data are screened out, providing a more reliable and consistent data basis for subsequent model training.

[0059] Optionally, the number of samples can be constrained according to the number requirement of nf, where n is the total number of data models in the federated learning network, and f is an agreed number not exceeding n / 3.

[0060] In some embodiments of the present application, considering that under a semi-synchronous communication model, that is, after some unknown global stabilization time (GST), message transmission will be limited by a known boundary Δ, and when the total number of data models is greater than three times the number of untrusted data models, HotStuff is a Byzantine Fault Tolerance (BFT) state machine replication (SMR) protocol that implements linear round changes and optimistic responses by adding a pre-send (PRE-COMMIT) phase in each round of Practical Byzantine Fault Tolerance (PBFT). Each round can only be submitted when the local server collects votes from nf data models in three phases, namely, the preparation phase (PREPARE), the pre-send phase (PRE-COMMIT), and the send phase (COMMIT). Where n represents the total number of data models and f represents the number of untrusted data models. The collected votes are called a quorum certificate (QC). Each QC is associated with a unique round number, and HotStuff uses a threshold signature to The signature vote is converted to a single authenticator.

[0061] like Figure 3 As shown in Figure 1, the specific process of updating the training gradient of the server in a global round is as follows:

[0062] In the PREPARE phase, the local server collects new round messages (NEW-VIEW) from nf data models and generates a prepare certificate (prepareQC) associated with the global round certificate (highQC). Then, the local server broadcasts prepareQC with highQC; after receiving prepareQC from the local server, each foreign server verifies highQC and sends a prepare vote to the local server.

[0063] In the PRE-COMMIT phase, the local server receives nf PREPARE votes for the current proposal and combines them into a prepareQC. Then, the local server broadcasts a PRE-COMMIT message with prepareQC; after receiving the PRE-COMMIT message from the local server, each foreign server sends a PRE-COMMIT vote to the local server.

[0064] The COMMIT phase is similar to the PRE-COMMIT phase: after the local server receives nf PRE-COMMIT votes, it combines them into a pre-send certificate (precommitQC). Then, the local server broadcasts a COMMIT message with precommitQC, and each foreign server sends a COMMIT vote to the local server. It is worth noting that in this phase, each server locks on precommitQC to ensure the security of the consensus decision of the proposal.

[0065] In the DECIDE phase, similar to the PRE-COMMIT phase, the local server receives nf COMMIT votes and combines them into a send certificate (commitQC). The local server then broadcasts a DECIDE message with commitQC; after receiving the DECIDE message from the local server, each foreign server regards the proposal in commitQC as a committed decision and executes it. Finally, each server increments the round period and starts the next round. A correct data model only sends O(n) authenticators to reach consensus, which indicates a linear round change. In addition, a correct data model only waits for the first nf responses to ensure that a promising proposal can be created, which indicates an optimistic response.

[0066] In sub-step 20212, the sum of all minimum Euclidean distances in each minimum Euclidean distance group is determined as the gradient evaluation value of the reference training gradient data, and the reference training gradient data corresponding to the minimum gradient evaluation value is determined as the target reference training gradient data.

[0067] In some embodiments of the present application, in order to screen out the most credible training gradient data, it is necessary to determine the sum of all minimum Euclidean distances in each minimum Euclidean distance group as the gradient evaluation value of the reference training gradient data, and determine the reference training gradient data corresponding to the minimum gradient evaluation value as the target reference training gradient data. The gradient evaluation value is a comprehensive indicator for measuring the credibility of the training gradient data. Through this process, the training gradient closest to the real gradient data can be effectively screened out, thereby improving the credibility of the data and the accuracy of the model.

[0068] For example, during the training process of a medical health data analysis system, server A in a federated learning network calculated the Euclidean distance between the stored training gradient data and obtained the minimum Euclidean distance group corresponding to each training gradient data. The sum of all the minimum Euclidean distances in these minimum Euclidean distance groups is determined as the gradient evaluation value of each reference training gradient data, and the reference training gradient data corresponding to the minimum gradient evaluation value is determined as the target reference training gradient data. Through this process, server A screened out the most credible training gradient data, providing a reliable data foundation for subsequent model training.

[0069] Sub-step 20213, determining the training gradient data corresponding to each Euclidean distance in the minimum Euclidean distance group corresponding to the target reference training gradient data as an honest training gradient data.

[0070] In some embodiments of the present application, in order to further screen out credible training gradient data, it is necessary to determine the training gradient data corresponding to each Euclidean distance in the minimum Euclidean distance group corresponding to the target reference training gradient data as an honest training gradient data. Honest training gradient data refers to training data that has been screened and verified and determined to be reliable and credible. Through this process, it can be ensured that the screened training gradient data has higher credibility and consistency, thereby improving the accuracy and stability of subsequent model training.

[0071] For example, during the training process of a medical health data analysis system, server A in a federated learning network has determined the target reference training gradient data. The training gradient data corresponding to each Euclidean distance in the minimum Euclidean distance group corresponding to the target reference training gradient data are respectively determined as honest training gradient data. Through this process, server A successfully screened out a set of credible training gradient data, providing a reliable data foundation for subsequent model training.

[0072] Sub-step 2022, averaging the selected multiple honest training gradient data to obtain comprehensive training gradient data for the local data model under the local training round.

[0073] In some embodiments of the present application, in order to obtain more stable and accurate gradient data, the selected multiple honest training gradient data can be averaged. Averaging refers to weighted averaging of the values ​​of multiple gradient data to generate a comprehensive gradient data. Through this process, the impact of individual outliers on the training results can be effectively reduced, the stability and accuracy of the comprehensive training gradient data can be ensured, and the effect of model training can be improved.

[0074] For example, during the training process of a medical health data analysis system, server A in a federated learning network has screened out several honest training gradient data. By calculating the average value of these honest training gradient data, the comprehensive training gradient data of server A under the local training round is generated. The comprehensive training gradient data is then used for local model training of server A. Through the averaging process, the comprehensive training gradient data obtained by server A is more stable and reliable, providing a solid data foundation for subsequent model training.

[0075] Optionally, the selected multiple honest training gradient data have corresponding reference training gradient data in common, the reference training gradient data have corresponding reference gradient evaluation values, and the selected multiple honest training gradient data respectively correspond to their own honest gradient evaluation values. Sub-step 2022 includes the following sub-steps:

[0076] Sub-step 20221, determining the confidence interval radius values ​​of multiple comprehensive training gradient data according to all honest gradient evaluation values.

[0077] In some embodiments of the present application, in order to determine the confidence interval radius value of the comprehensive training gradient data, it is necessary to determine the confidence interval radius value of the comprehensive training gradient data based on all honest gradient evaluation values. The honest gradient evaluation value is an indicator to measure the credibility of the training gradient data. Through this process, it can effectively provide a basis for subsequent gradient screening and synthesis, thereby improving the reliability of the data and the accuracy of the model.

[0078] For example, during the training process of a medical health data analysis system, server A in a federated learning network has screened out several honest training gradient data, each of which has its own honest gradient evaluation value. Based on these evaluation values, the confidence interval radius value of the comprehensive training gradient data is calculated. Through this process, server A obtains a reliable confidence interval radius value, which provides an important basis for subsequent gradient data screening and model training.

[0079] Optionally, sub-step 20221 includes the following sub-steps:

[0080] Sub-step 202211, determining the mean of all honest gradient evaluation values ​​as the center value of the confidence interval of the comprehensive training gradient data, and determining the standard deviation of all honest gradient evaluation values ​​as the offset value of the confidence interval of the comprehensive training gradient data.

[0081] In some embodiments of the present application, in order to more accurately define the confidence interval of the comprehensive training gradient data, it is necessary to determine the mean of all honest gradient evaluation values ​​as the center value of the confidence interval of the comprehensive training gradient data, and determine the standard deviation of all honest gradient evaluation values ​​as the offset value of the confidence interval of the comprehensive training gradient data. The mean refers to the average value of all data, and the standard deviation is a statistic that measures the degree of data dispersion. Through this process, the confidence interval of the comprehensive training gradient data can be clarified, thereby improving the reliability of the data and the accuracy of the model.

[0082] For example, during the training of a medical data analysis system, server A in a federated learning network has obtained several honest gradient evaluation values. The mean of these evaluation values ​​is calculated and determined as the center value of the confidence interval of the comprehensive training gradient data; at the same time, the standard deviation of these evaluation values ​​is calculated and determined as the offset value of the confidence interval. Through this process, server A successfully determined the confidence interval of the comprehensive training gradient data, improving the credibility of the data and the accuracy of model training.

[0083] In sub-step 202212, the sum of the offset value and the center value scaled according to the preset scaling parameters is determined as the confidence interval radius.

[0084] In some embodiments of the present application, in order to accurately calculate the confidence interval radius of the comprehensive training gradient data, the offset value needs to be scaled according to a preset scaling parameter and added to the center value. The scaling parameter is a coefficient used to adjust the offset value to ensure that the confidence interval is reasonable and accurate. Through this process, it can be ensured that the confidence interval radius can accurately reflect the credible range of the comprehensive training gradient data, thereby improving the reliability of the data and the accuracy of the model.

[0085] For example, during the training process of a medical health data analysis system, Server A in a federated learning network has calculated the center value and offset value of the comprehensive training gradient data. The offset value is adjusted using the preset scaling parameters, and the adjusted offset value is added to the center value to determine the confidence interval radius. Through this process, Server A successfully calculated the confidence interval radius of the comprehensive training gradient data, ensuring the credible range of the data and improving the reliability and accuracy of model training.

[0086] Sub-step 20222, when the reference gradient evaluation value is less than or equal to the confidence interval radius value, adding the reference training gradient data to the honest training gradient data.

[0087] In some embodiments of the present application, in order to ensure that all selected training gradient data are within an acceptable confidence interval, it is necessary to add the reference training gradient data to the honest training gradient data when the reference gradient evaluation value is less than or equal to the confidence interval radius value. The confidence interval radius value is a range value calculated based on the honest gradient evaluation value, which is used to determine the credibility of the training data. Through this process, it can be ensured that only training gradient data that meets the credibility requirements are included for use, thereby improving the reliability and accuracy of model training.

[0088] For example, during the training of a medical data analysis system, Server A in a federated learning network calculated the confidence interval radius value of the comprehensive training gradient data. When it is found that the gradient evaluation value of a reference training gradient data is less than or equal to the confidence interval radius value, the reference training gradient data is added to the honest training gradient data. Through this process, Server A ensures that the included training gradient data are within the credible range, thereby providing reliable data support for subsequent model training.

[0089] Sub-step 20223, determines the average value of all honest training gradient data as the comprehensive training gradient data.

[0090] In some embodiments of the present application, in order to obtain a representative comprehensive training gradient data, all honest training gradient data can be averaged. The average value refers to adding multiple data and then dividing by the number of data to obtain a comprehensive data value. Through this process, random errors in individual gradient data can be eliminated, ensuring the stability and accuracy of the comprehensive training gradient data, which helps to improve the effect of model training.

[0091] For example, during the training process of a medical health data analysis system, server A in a federated learning network has screened out several honest training gradient data. These honest training gradient data are added together and averaged, and the average is determined as the comprehensive training gradient data of server A. Through this process, server A obtains more stable and accurate gradient data, thereby improving the effect and efficiency of subsequent model training.

[0092] Sub-step 2023, using the comprehensive training gradient data as the configuration of the local data model, training the local data model to obtain updated training gradient data of the local data model in the local training round.

[0093] In some embodiments of the present application, in order to introduce the honest training gradient data screened in the previous step, it is necessary to use the comprehensive training gradient data as the configuration of the local data model to train the local data model. The comprehensive training gradient data refers to the gradient data generated after averaging. Through this process, more stable and reliable gradient data can be effectively used for model training, thereby improving the accuracy and convergence speed of the model.

[0094] For example, during the training process of a medical health data analysis system, server A in a federated learning network has generated comprehensive training gradient data. Based on these comprehensive training gradient data, the local data model of server A is configured and then trained. After training, server A obtains updated training gradient data under the local training round. These data not only improve the accuracy of the server A model, but also speed up the convergence process of the model, laying a more solid foundation for subsequent federated learning training.

[0095] Optionally, sub-step 2023 includes the following sub-steps:

[0096] Sub-step 20231, updates the weight data of the local data model by comprehensive training gradient data.

[0097] In some embodiments of the present application, in order to ensure that the weight data of the local data model reflects the latest comprehensive training gradient data, it is necessary to update the weight data of the local data model through the comprehensive training gradient data. The comprehensive training gradient data refers to the average value of multiple honest training gradient data. Through this process, it can be ensured that the weight data of the local data model is updated in a timely manner, so that more reliable and accurate weight data can be used in subsequent training processes, thereby improving the training effect and performance of the model.

[0098] For example, during the training of a medical health data analysis system, server A in a federated learning network has obtained comprehensive training gradient data. Based on these comprehensive training gradient data, the weight data of the local data model of server A is updated. Through this process, the model weight of server A is adjusted and optimized in a timely manner, thus providing a more reliable data foundation for subsequent model training and improving the training effect and model performance.

[0099] Sub-step 20232, based on the training of the data model of the local server after the weight data is updated, obtain the updated training gradient data of the local data model.

[0100] In some embodiments of the present application, in order to ensure that the local data model can be effectively trained according to the latest weight data, it is necessary to obtain updated training gradient data of the local data model based on the operation of the data model of the local server after the weight data is updated. Weight data refers to the value of the model parameter during the training process. Through this process, it can be ensured that the model uses the latest weights during training, thereby obtaining accurate training gradient data and improving the performance and accuracy of the model.

[0101] For example, during the training process of a healthcare data analysis system, Server A in a federated learning network has updated the weight data of its local data model. The local data model of Server A is run, and updated training gradient data is obtained based on its performance during training. Through this process, Server A obtains accurate updated training gradient data while training with the latest weight data, thereby improving the training effect and performance of the model.

[0102] In some specific embodiments of the present application, the actual running time of the training process will also be constrained according to the global synchronization time (Global Stabilization Time for Local training, GST_LT), that is, the training process completed within the GST_LT will be recorded as a successful training and the server will be put into sleep before the next round of training; otherwise, the training results within the round will not be recorded.

[0103] Step 203, based on the updated training gradient data, initiates an update application to the foreign server in the cross-island federated learning network, so that the foreign server that receives the update application updates the target training gradient data according to the received update application, and updates the local training rounds according to the update results fed back by the foreign server.

[0104] Among them, the target training gradient data is the training gradient data stored in each foreign server, which corresponds to the local server; the training gradient data stored in the foreign server corresponds one-to-one to multiple servers in the cross-island federated learning network; the update result is fed back by the foreign server based on the update application.

[0105] The method shown in this step has been explained in step 102 and will not be repeated here.

[0106] Optionally, step 203 includes the following sub-steps:

[0107] Sub-step 2031, sending the updated training gradient data to the external server, so that when the external training round of the external server is one round ahead of the global training round, the value of the target training gradient data is updated to the value of the updated training gradient data.

[0108] In some embodiments of the present application, in order to ensure that the foreign model can use the latest training gradient data, it is necessary to send the updated training gradient data to the foreign model. When the foreign training round of the foreign server is one round ahead of the global training round, the training gradient data corresponding to the local server stored in the foreign server will be replaced with the updated training gradient data. Updated training gradient data refers to data obtained and verified through local training. Through this process, it can be ensured that the foreign data model updates its stored gradient data in a timely manner, thereby maintaining data consistency and synchronization, which helps to improve the training efficiency and accuracy of the entire federated learning system.

[0109] For example, during the training process of a healthcare data analysis system, server A in a cross-island federated learning network generates updated training gradient data after local training. These updated training gradient data are sent to server B and server C to ensure that the stored training gradient data can be replaced with the latest updated training gradient data in a timely manner when the local training round of the local server is one round ahead of the global training round. Through this process, the gradient data of server B and server C can be updated in a timely manner, improving the training consistency of the entire system and the accuracy of the model.

[0110] Sub-step 2032, when the update result is characterized by updating the value of the target training gradient data to the value of the updated training gradient data, the round value obtained by adding one to the round value of the global training round is used as the round value of the updated local training round.

[0111] In some embodiments of the present application, in order to ensure that the local training round is synchronized with the global training round, it is necessary to replace the training gradient data corresponding to the local server stored in the external data model with the updated training gradient data when the update result of the update application feedback is characterized by adding one to the global training round. The round value is used as the updated local training round. The global training round refers to the unified training progress in the entire federated learning network. Through this process, the consistency of the local training progress and the global progress is ensured, which helps to maintain the coordination and efficiency of the overall training.

[0112] For example, during the training process of a medical health data analysis system, after Server A in a cross-island federated learning network initiates an update request, it receives feedback from Server B and Server C, which indicates that Server B and Server C have replaced their stored training gradient data with updated training gradient data. Based on these feedbacks, the round value after the global training round is increased by one is used as the updated local training round of Server A. Through this process, the training progress of Server A is synchronized with the global progress, thereby improving the training consistency of the entire system and the accuracy of the model.

[0113] Step 204, in response to update requests sent from multiple external servers, obtain corresponding aggregation requests sent from multiple external servers, and when the aggregation requests reach a preset number, update the global training rounds and the training gradient data stored in the local server.

[0114] Among them, each updated training gradient data corresponds one-to-one to the foreign server that sends each update application.

[0115] The method shown in this step has been explained in step 103 and will not be repeated here.

[0116] Optionally, step 204 includes the following sub-steps:

[0117] Sub-step 2041, when the aggregation application reaches a preset aggregation quantity threshold and the foreign training round of the foreign data model is one round ahead of the global training round, the value of the global training round is updated to the value of the foreign training round.

[0118] In some embodiments of the present application, in order to ensure the synchronization and consistency of global training rounds, it is necessary to update the value of the global training round to the value of the foreign training round when the foreign training round of the foreign server is one round ahead of the global training round and the aggregation request sent from the foreign data model reaches the preset aggregation quantity threshold. The preset aggregation quantity threshold refers to the number of data aggregation requests that need to be satisfied. Through this process, it can be ensured that the local training progress is synchronized with the global training progress, which helps to maintain the coordination and consistency of the overall system and improve training efficiency and model accuracy.

[0119] For example, during the training process of a medical health data analysis system, the local training rounds of servers B and C in a cross-island federated learning network are ahead of the global training rounds, and the number of aggregated applications received by server A from servers B and C reaches the preset aggregated number threshold. The global training rounds stored locally on server A are updated to the local training rounds of servers B and C. Through this process, server A is synchronized with the global training rounds, ensuring that server A and other servers are trained in the same training round, improving the training efficiency and accuracy of the overall model.

[0120] Sub-step 2042, updating the value of the target training gradient data in the training gradient data stored in the local server to the value of the updated training gradient data in the update application corresponding to each target training gradient data.

[0121] In some embodiments of the present application, in order to ensure the consistency and accuracy of the training gradient data, it is necessary to determine the foreign updated training gradient data of the foreign data model corresponding to each update application contained in the update application sent from each foreign data model as the updated training gradient data corresponding to each foreign data model. Through this process, the consistency and synchronization of the training gradient data between the local data model and the foreign data model can be ensured, which helps to improve the overall training efficiency and model accuracy of the federated learning system.

[0122] For example, during the training process of a medical health data analysis system, server A in a cross-island federated learning network receives update requests from servers B and C, which contain the out-of-region updated training gradient data of servers B and C. These out-of-region updated training gradient data are determined as updated training gradient data corresponding to servers B and C. Through this process, the consistency of the training gradient data of server A with servers B and C is ensured, which improves the model training effect and accuracy of the entire network.

[0123] Based on the steps disclosed above, Figure 4 As shown, a complete example is given for the method provided in the embodiment of the present application:

[0124] The federated learning network provided in this application includes multiple servers, each of which is loaded with a data model to be trained, and each data model communicates and exchanges data with each other through the server where it is located. Each data model independently processes and trains local data, and at the same time realizes data sharing and aggregation across servers through a communication mechanism. Each data model performs a gradient data aggregation process, including a voting process. In this step, each data model votes on the locally calculated gradient data and selects the most credible gradient data for model updating.

[0125] In the process of mutual interaction between servers, the iteration process is implemented through the following steps:

[0126] S1: Local server sends data to other places: The local data model sends updated training gradient data to other foreign data models. In this way, all participating data models can share the latest training data and use this data to update the model during the training process.

[0127] S2: Filtering in the new round: Each data model will filter the received training gradient data and select the most reliable gradient data for the next round of model training. This filtering process can be achieved by calculating the Euclidean distance and gradient evaluation value between each training gradient data, so as to select the data that best meets the requirements.

[0128] S3: Local data models update local models: Each foreign data model will use the most trusted gradient data selected based on the filtering results to update the local model. Through this process, each data model can synchronously update its model parameters to ensure the consistency and synchronization of model training in the entire distributed system.

[0129] In summary, in the embodiment of the present application, by screening the obtained honest training gradient data, it is ensured that even if there are malicious nodes, the data used by the data model of the local server is still credible, thereby improving the data reliability of model training; and then the local data model can be configured and trained according to the filtered honest training gradient data to obtain updated local training gradient data, so as to ensure that each node performs effective model training locally, avoiding the negative impact of untrusted gradients on the model, and further enhancing the accuracy and robustness of the model; finally, through the synchronization and update of gradient data, the consistency and synchronization of gradient data in the entire network are ensured, the potential model errors and security risks caused by inconsistent data are avoided, and all nodes are ensured to be synchronized in the same training round, maintaining the overall coordination and consistency of the system, and effectively reducing the problems of slow model convergence and low accuracy caused by asynchronous training. Therefore, based on the method of the embodiment of the present application, while eliminating the central server, the decentralized gradient aggregation and adaptive gradient filtering technology are used to improve the security, data reliability and training efficiency of cross-island federated learning model training, and solve the problems of insufficient data reliability and high storage and network overhead in the model training process.

[0130] refer to Figure 5 , which shows a cross-island federated learning model training device 30 based on decentralized gradient aggregation provided by an embodiment of the present application, applied to a data model of a local server in a cross-island federated learning network, including:

[0131] The filtering module 301 is used to filter the multiple training gradient data stored in the local server to obtain multiple honest training gradient data for the local data model when the local training round of the local data model lags behind the global training round of the cross-island federated learning network, and configure the local data model according to the multiple honest training gradient data, and train the local data model to obtain the updated training gradient data of the data model of the local server under the local training round; the multiple training gradient data stored in the local server correspond to the multiple servers in the cross-island federated learning network one by one;

[0132] The updating module 302 is used to initiate an update application to a foreign server in the cross-island federated learning network according to the updated training gradient data, so that the foreign server receiving the update application updates the target training gradient data according to the received update application, and updates the local training round according to the update result fed back by the foreign server; the target training gradient data is the training gradient data corresponding to the local server in the training gradient data stored in each foreign server; the training gradient data stored in the foreign server corresponds one-to-one to multiple servers in the cross-island federated learning network; the update result is fed back by the foreign server according to the update application;

[0133] The aggregation module 303 is used to respond to update applications sent from multiple external servers, obtain the corresponding aggregation applications sent from the multiple external servers, and when the aggregation applications reach a preset number, update the global training rounds and the training gradient data stored in the local server; each updated training gradient data corresponds one-to-one to the external server that sends each update application.

[0134] Optionally, the filtering module 301 includes:

[0135] A filtering submodule, used for filtering the plurality of training gradient data stored in the local server according to the Euclidean distance between every two training gradient data, so as to select the remaining training gradient data after filtering as the honest training gradient data;

[0136] An averaging submodule, used to average the selected multiple honest training gradient data to obtain comprehensive training gradient data for the local data model under the local training round;

[0137] The training submodule is used to train the local data model using the comprehensive training gradient data as the configuration of the local data model to obtain updated training gradient data of the local data model under the local training round.

[0138] Optionally, the filtering submodule includes:

[0139] A distance calculation unit is used to calculate the Euclidean distance between each two training gradient data among the multiple training gradient data stored in the local server, and perform minimum value filtering on the calculated Euclidean distance according to a preset sampling number to obtain a minimum Euclidean distance group corresponding to each training gradient data; the minimum Euclidean distance group includes multiple Euclidean distances, each Euclidean distance corresponds to a training gradient data other than the reference training gradient data; the reference training gradient data is the training gradient data corresponding to the minimum Euclidean distance group;

[0140] An evaluation and selection unit, used to determine the sum of all minimum Euclidean distances in each minimum Euclidean distance group as a gradient evaluation value of the reference training gradient data, and determine the reference training gradient data corresponding to the minimum gradient evaluation value as the target reference training gradient data;

[0141] The filtering unit is used to determine the training gradient data corresponding to each Euclidean distance in the minimum Euclidean distance group corresponding to the target reference training gradient data as an honest training gradient data.

[0142] Optionally, the selected multiple honest training gradient data have corresponding reference training gradient data in common, the reference training gradient data have corresponding reference gradient evaluation values, the selected multiple honest training gradient data respectively correspond to respective honest gradient evaluation values, and the averaging submodule includes:

[0143] A radius calculation unit, used to determine the confidence interval radius values ​​of multiple comprehensive training gradient data according to all honest gradient evaluation values;

[0144] A supplement unit, used for adding the reference training gradient data to the honest training gradient data when the reference gradient evaluation value is less than or equal to the confidence interval radius value;

[0145] The averaging unit is used to determine the average value of all honest training gradient data as comprehensive training gradient data.

[0146] Optionally, the radius calculation unit includes:

[0147] A statistical calculation subunit, used to determine the mean of all honest gradient evaluation values ​​as the center value of the confidence interval of the comprehensive training gradient data, and determine the standard deviation of all honest gradient evaluation values ​​as the offset value of the confidence interval of the comprehensive training gradient data;

[0148] The radius calculation subunit is used to determine the sum of the offset value and the center value scaled according to the preset scaling parameter as the confidence interval radius.

[0149] Optionally, the training submodule includes:

[0150] A weight calculation unit, used to update the weight data of the local data model by comprehensive training gradient data;

[0151] The local update unit is used to obtain updated training gradient data of the local data model based on the training of the data model of the local server after the weight data is updated.

[0152] Optionally, the updating module 302 includes:

[0153] An update initiation submodule, used to send the updated training gradient data to the foreign server, so that when the foreign training round of the foreign server is one round ahead of the global training round, the value of the target training gradient data is updated to the value of the updated training gradient data;

[0154] The local round update submodule is used to, when the update result is characterized by updating the value of the target training gradient data to the value of the updated training gradient data, add one to the round value of the global training round, and use the round value obtained as the round value of the updated local training round.

[0155] Optionally, the aggregation module 303 includes:

[0156] A global round update submodule is used to update the value of the global training round to the value of the foreign training round when the aggregation application reaches a preset aggregation quantity threshold and the foreign training round of the foreign data model is one round ahead of the global training round;

[0157] The update receiving submodule is used to update the value of the target training gradient data in the training gradient data stored in the local server to the value of the updated training gradient data in the update application corresponding to each target training gradient data.

[0158] Optionally, the cross-island federated learning model training device 30 based on decentralized gradient aggregation further includes:

[0159] The initialization module is used to obtain a set of randomly generated initialization weights and initialize the local data model of the local server according to the initialization weights.

[0160] In summary, in the embodiment of the present application, by screening the obtained honest training gradient data, it is ensured that even if there are malicious nodes, the data used by the data model of the local server is still credible, thereby improving the data reliability of model training; and then the local data model can be configured and trained according to the filtered honest training gradient data to obtain updated local training gradient data, so as to ensure that each node performs effective model training locally, avoiding the negative impact of untrusted gradients on the model, and further enhancing the accuracy and robustness of the model; finally, through the synchronization and update of gradient data, the consistency and synchronization of gradient data in the entire network are ensured, the potential model errors and security risks caused by inconsistent data are avoided, and all nodes are ensured to be synchronized in the same training round, maintaining the overall coordination and consistency of the system, and effectively reducing the problems of slow model convergence and low accuracy caused by asynchronous training. Therefore, based on the method of the embodiment of the present application, while eliminating the central server, the decentralized gradient aggregation and adaptive gradient filtering technology are used to improve the security, data reliability and training efficiency of cross-island federated learning model training, and solve the problems of insufficient data reliability and high storage and network overhead in the model training process.

[0161] Reference Figure 6 , the electronic device 500 may include one or more of the following components: a processing component 502 , a memory 504 , a power component 506 , a multimedia component 508 , an audio component 510 , an input / output (I / O) interface 512 , a sensor component 514 , and a communication component 516 .

[0162] The processing component 502 generally controls the overall operation of the electronic device 500, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 502 may include one or more processors 520 to execute instructions to complete all or part of the steps of the above-mentioned method. In addition, the processing component 502 may include one or more modules to facilitate the interaction between the processing component 502 and other components. For example, the processing component 502 may include a multimedia module to facilitate the interaction between the multimedia component 508 and the processing component 502.

[0163] The memory 504 is used to store various types of data to support the operation of the electronic device 500. Examples of such data include instructions for any application or method operating on the electronic device 500, contact data, phone book data, messages, pictures, multimedia, etc. The memory 504 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0164] The power supply component 506 provides power to the various components of the electronic device 500. The power supply component 506 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the electronic device 500.

[0165] The multimedia component 508 includes an interface that provides an output interface between the electronic device 500 and the user. In some embodiments, the interface may include a liquid crystal display (LCD) and a touch panel (TP). If the interface includes a touch panel, the interface may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 508 includes a front camera and / or a rear camera. When the electronic device 500 is in an operating mode, such as a shooting mode or a multimedia mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0166] The audio component 510 is used to output and / or input audio signals. For example, the audio component 510 includes a microphone (MIC), and when the electronic device 500 is in an operation mode, such as a call mode, a recording mode, and a voice recognition mode, the microphone is used to receive an external audio signal. The received audio signal can be further stored in the memory 504 or sent via the communication component 516. In some embodiments, the audio component 510 also includes a speaker for outputting audio signals.

[0167] The input / output I / O interface 512 provides an interface between the processing component 502 and the peripheral interface module, which may be a keyboard, a click wheel, buttons, etc. These buttons may include but are not limited to: a home button, a volume button, a start button, and a lock button.

[0168] The sensor assembly 514 includes one or more sensors for providing various aspects of status assessment for the electronic device 500. For example, the sensor assembly 514 can detect the open / closed state of the electronic device 500, the relative positioning of components, such as the display and keypad of the electronic device 500, and the sensor assembly 514 can also detect the position change of the electronic device 500 or a component of the electronic device 500, the presence or absence of user contact with the electronic device 500, the orientation or acceleration / deceleration of the electronic device 500, and the temperature change of the electronic device 500. The sensor assembly 514 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 514 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 514 may also include an accelerometer, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0169] The communication component 516 is used to facilitate wired or wireless communication between the electronic device 500 and other devices. The electronic device 500 can access a wireless network based on a communication standard, such as WiFi, a carrier network (such as 2G, 3G, 4G or 5G), or a combination thereof. In an exemplary embodiment, the communication component 516 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 516 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0170] In an exemplary embodiment, the electronic device 500 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to implement the methods provided in the embodiments of the present application.

[0171] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 504 including instructions, and the instructions can be executed by a processor 520 of an electronic device 500 to perform the above method. For example, the non-transitory storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0172] Figure 7FIG. 6 is a block diagram of an electronic device 600 according to another embodiment of the present invention. For example, the electronic device 600 may be provided as a server.

[0173] Reference Figure 7 , the electronic device 600 includes a processing component 622, which further includes one or more processors, and a memory resource represented by a memory 632, for storing instructions that can be executed by the processing component 622, such as an application. The application stored in the memory 632 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 622 is configured to execute instructions to perform the method provided in the embodiment of the present application.

[0174] The electronic device 600 may also include a power supply component 626 configured to perform power management of the electronic device 600, a wired or wireless network interface 650 configured to connect the electronic device 600 to a network, and an input / output (I / O) interface 658. The electronic device 600 may operate based on an operating system stored in the memory 632, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™ or the like.

[0175] It should be emphasized that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0176] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the application disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present application are indicated by the following claims.

[0177] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A cross-island federated learning model training method based on decentralized gradient aggregation, characterized in that: Applied to a local server in a cross-island federated learning network, the local server is loaded with a local data model to be trained, including: In the case where the local training round of the local data model lags behind the global training round of the cross-island federated learning network, multiple training gradient data stored in the local server are filtered to obtain multiple honest training gradient data for the local data model, and the local data model is configured according to the multiple honest training gradient data, and the local data model is trained to obtain updated training gradient data of the data model of the local server under the local training round; the multiple training gradient data stored in the local server correspond one-to-one to the multiple servers in the cross-island federated learning network; According to the updated training gradient data, an update application is initiated to a foreign server in the cross-island federated learning network, so that the foreign server receiving the update application updates the target training gradient data according to the received update application, and updates the local training round according to the update result fed back by the foreign server; the target training gradient data is the training gradient data corresponding to the local server stored in the training gradient data of each foreign server; the training gradient data stored in the foreign server corresponds one-to-one to multiple servers in the cross-island federated learning network; the update result is fed back by the foreign server according to the update application; In response to update requests sent from multiple external servers, aggregate requests corresponding to the multiple external servers are obtained, and when the aggregate requests reach a preset number, the global training rounds and the training gradient data stored in the local server are updated; each of the updated training gradient data corresponds one-to-one to the external server that sends each of the update requests.

2. The cross-island federated learning model training method based on decentralized gradient aggregation as claimed in claim 1, characterized in that: The filtering of the plurality of training gradient data stored in the local server to obtain a plurality of honest training gradient data for the local data model, and training the local data model according to the plurality of honest training gradient data to configure the local data model to obtain updated training gradient data of the data model of the local server under the local training round, includes: According to the Euclidean distance between every two of the training gradient data, filtering the plurality of training gradient data stored in the local server, so as to select the training gradient data remaining after filtering as the honest training gradient data; Averaging the selected multiple honest training gradient data to obtain comprehensive training gradient data for the local data model under the local training round; The comprehensive training gradient data is used as the configuration of the local data model to train the local data model to obtain updated training gradient data of the local data model in the local training round.

3. The cross-island federated learning model training method based on decentralized gradient aggregation as claimed in claim 2 is characterized in that: The filtering of the plurality of training gradient data stored in the local server according to the Euclidean distance between every two of the training gradient data, so as to select the training gradient data remaining after filtering as the honest training gradient data, comprises: Calculating the Euclidean distance between every two training gradient data stored in the local server, and performing minimum value filtering on the calculated Euclidean distance according to a preset sampling number to obtain a minimum Euclidean distance group corresponding to each training gradient data; the minimum Euclidean distance group includes a plurality of Euclidean distances, each of which corresponds one to one with a training gradient data other than the reference training gradient data; the reference training gradient data is the training gradient data corresponding to the minimum Euclidean distance group; Determining the sum of all minimum Euclidean distances in each minimum Euclidean distance group as the gradient evaluation value of the reference training gradient data, and determining the reference training gradient data corresponding to the minimum gradient evaluation value as the target reference training gradient data; The training gradient data corresponding to each Euclidean distance in the minimum Euclidean distance group corresponding to the target reference training gradient data is respectively determined as one of the honest training gradient data.

4. The cross-island federated learning model training method based on decentralized gradient aggregation as claimed in claim 2, characterized in that: The selected multiple honest training gradient data have commonly corresponding reference training gradient data, the reference training gradient data have corresponding reference gradient evaluation values, the selected multiple honest training gradient data respectively correspond to respective honest gradient evaluation values, and the averaging of the selected multiple honest training gradient data to obtain comprehensive training gradient data for the local data model under the local training round includes: Determining confidence interval radius values ​​of a plurality of the comprehensive training gradient data according to all the honest gradient evaluation values; When the reference gradient evaluation value is less than or equal to the confidence interval radius value, adding the reference training gradient data to the honest training gradient data; The average value of all the honest training gradient data is determined as the comprehensive training gradient data.

5. The cross-island federated learning model training method based on decentralized gradient aggregation as claimed in claim 2, characterized in that: The step of using the comprehensive training gradient data as the configuration of the local data model to train the local data model to obtain updated training gradient data of the local data model in the local training round includes: Updating the weight data of the local data model through the comprehensive training gradient data; According to the training of the data model of the local server after the weight data is updated, the updated training gradient data of the local data model is obtained.

6. The cross-island federated learning model training method based on decentralized gradient aggregation as claimed in claim 1, characterized in that: The initiating an update application to a foreign server in the cross-island federated learning network according to the updated training gradient data, so that the foreign server receiving the update application updates the target training gradient data according to the received update application, and updates the local training round according to the update result fed back by the foreign server, including: Sending the updated training gradient data to the foreign server, so that when the foreign training round of the foreign server is ahead of the global training round by one round, the value of the target training gradient data is updated to the value of the updated training gradient data; In a case where the update result is characterized by updating the value of the target training gradient data to the value of the updated training gradient data, a round value obtained by adding one to the round value of the global training round is used as the round value of the updated local training round.

7. The cross-island federated learning model training method based on decentralized gradient aggregation as claimed in claim 1, characterized in that: When the aggregated application reaches a preset number, updating the global training round and the training gradient data stored in the local server includes: When the aggregation application reaches a preset aggregation quantity threshold, and when the foreign training round of the foreign data model is one round ahead of the global training round, updating the value of the global training round to the value of the foreign training round; The value of the target training gradient data in the training gradient data stored in the local server is updated to the value of the updated training gradient data in the update application corresponding to each target training gradient data.

8. A cross-island federated learning model training device based on decentralized gradient aggregation, characterized in that: Applied to a local server in a cross-island federated learning network, the local server is loaded with a local data model to be trained, including: A filtering module, configured to filter the plurality of training gradient data stored in the local server to obtain a plurality of honest training gradient data for the local data model when the local training round of the local data model lags behind the global training round of the cross-island federated learning network, and to train the local data model according to the plurality of honest training gradient data to obtain updated training gradient data of the data model of the local server under the local training round; the plurality of training gradient data stored in the local server corresponds one-to-one to the plurality of servers in the cross-island federated learning network; An update module is used to initiate an update application to a foreign server in the cross-island federated learning network according to the updated training gradient data, so that the foreign server receiving the update application updates the target training gradient data according to the received update application, and updates the local training round according to the update result fed back by the foreign server; the target training gradient data is the training gradient data corresponding to the local server stored in the training gradient data of each foreign server; the training gradient data stored in the foreign server corresponds one-to-one to multiple servers in the cross-island federated learning network; the update result is fed back by the foreign server according to the update application; An aggregation module is used to respond to update applications sent from multiple external servers, obtain aggregation applications corresponding to the multiple external servers, and when the aggregation applications reach a preset number, update the global training rounds and the training gradient data stored in the local server; each of the updated training gradient data corresponds one-to-one to the external server that sends each update application.

9. An electronic device, characterized in that: include: A processor, a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: When the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to perform the method as claimed in any one of claims 1 to 7.

Images