Federal learning method and related equipment

Abstract

The invention provides a federated learning method and related equipment, and relates to the technical field of federated learning. The method comprises the following steps: a client obtains a first global model sent by a central server; training the first global model according to the client data, and determining a first model parameter; uploading the first model parameter and the current client resource state information to a central server, so that the central server trains a first global model through a dynamic selection algorithm according to the first model parameter uploaded by each client and the current client resource state information to obtain a second global model; and receiving a second global model issued by the central server, and entering a next round of iteration until a predefined stop condition is met. According to the method, the resource state and the model parameters are combined, the dynamic change of the training environment can be sensed and adapted through the dynamic selection algorithm, and the problem that a fixed fusion algorithm adopted in federated learning lacks sensing and adaption to the dynamic change of the training environment is solved.

Description

Technical Field

[0001] This disclosure relates to the field of federated learning technology, and more particularly to a federated learning method and related equipment. Background Technology

[0002] Federated learning, a distributed machine learning approach, allows participants to collaboratively train models without sharing data. It has gained widespread attention in recent years, especially given the increasing importance of data privacy and security. With the rapid development of large-scale models, publicly available training data is becoming increasingly scarce. Meanwhile, while abundant high-quality private data exists, privacy concerns make it difficult to share and directly use for model training. In this context, federated learning has emerged as a key technological solution to this dilemma. It cleverly balances data privacy with model training needs, enabling efficient collaborative training among participants while ensuring data remains locally, thus providing strong support for the continued development of large-scale models.

[0003] Existing federated learning methods typically employ fixed fusion algorithms, lacking mechanisms for perceiving and adapting to dynamic changes in the training environment. In federated learning scenarios, the computing resources, data distribution, and communication conditions of the participants often change over time, and fixed fusion strategies cannot effectively cope with these changes.

[0004] It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0005] This disclosure provides a federated learning method and related apparatus, which at least to some extent overcomes the problem that federated learning in related technologies lacks perception and adaptation to dynamic changes in the training environment due to the use of fixed fusion algorithms.

[0006] Other features and advantages of this disclosure will become apparent from the following detailed description, or may be learned in part from practice of this disclosure.

[0007] According to one aspect of this disclosure, a federated learning method is provided, applied to a client, comprising: obtaining a first global model sent by a central server; training the first global model based on client data to determine first model parameters; uploading the first model parameters and current client resource status information to the central server, so that the central server trains the first global model to obtain a second global model based on the first model parameters uploaded by each client and the current client resource status information through a dynamic selection algorithm; and receiving the second global model sent by the central server to enter the next iteration until a predefined stopping condition is met.

[0008] In some embodiments, client resource status information includes: hardware load information and network load information; the dynamic selection algorithm includes: when the model loss of the second global model increases beyond a first threshold during training, or the difference in accuracy among clients exceeds a second threshold, or when the hardware load information exceeds a third threshold, or the network load information exceeds a fourth threshold, triggering the dynamic selection algorithm: if the current algorithm is federated weighted average, switch to federated near-end algorithm; if the current algorithm is federated near-end algorithm, switch to federated weighted average algorithm; if the current algorithm is federated normalization algorithm, switch to federated buffer algorithm; if the current algorithm is federated buffer algorithm, switch to federated normalization algorithm.

[0009] In some embodiments, receiving the second global model issued by the central server to enter the next iteration until a predefined stopping condition is met includes: training the second global model based on client data to determine the second model parameters; uploading the second model parameters and the current client resource status information to the central server; and stopping the iteration when a preset number of iterations is reached or the global model achieves a preset performance on the verification data.

[0010] According to another aspect of this disclosure, a federated learning method is also provided, applied to a central server, comprising: sending a first global model to each client; obtaining first model parameters and current client resource status information sent by each client, wherein the first model parameters are determined by the client through training the first global model based on client data; training the first global model to obtain a second global model through a dynamic selection algorithm based on the first model parameters uploaded by each client and the current client resource status information; and distributing the second global model to each client to enter the next iteration, until a predefined stopping condition is met.

[0011] In some embodiments, the client resource status information includes: hardware load information and network load information; the dynamic selection algorithm includes: when the model loss of the second global model increases beyond a first threshold during training, or the difference in accuracy among clients exceeds a second threshold, or when the hardware load information exceeds a third threshold, or the network load information exceeds a fourth threshold, triggering the dynamic selection algorithm: if the current algorithm is a federated weighted average algorithm, switch to a federated near-end algorithm; if the current algorithm is a federated near-end algorithm, switch to a federated weighted average algorithm; if the current algorithm is a federated normalization algorithm, switch to a federated buffer algorithm; if the current algorithm is a federated buffer algorithm, switch to a federated normalization algorithm.

[0012] In some embodiments, the step of distributing the second global model to each client to enter the next iteration until a predefined stopping condition is met includes: obtaining the second model parameters sent by each client and the current client resource status information; training the second global model through a dynamic selection algorithm based on the first model parameters uploaded by each client and the current client resource status information; and stopping the iteration when a preset number of iterations is reached or the global model achieves a preset performance on the validation data.

[0013] According to another aspect of this disclosure, a federated learning device is also provided, applied to a client, comprising: a model acquisition module for acquiring a first global model sent by a central server; a parameter training module for training the first global model based on client data to determine first model parameters; an information uploading module for uploading the first model parameters and current client resource status information to the central server, so that the central server trains the first global model to obtain a second global model based on the first model parameters uploaded by each client and the current client resource status information through a dynamic selection algorithm; and a model acquisition iteration module for receiving the second global model sent by the central server to enter the next iteration until a predefined stopping condition is met.

[0014] According to another aspect of this disclosure, a federated learning device is also provided, applied to a central server, comprising: a model sending module for sending a first global model to each client; an information acquisition module for acquiring first model parameters and current client resource status information sent by each client, wherein the first model parameters are determined by the client through training the first global model based on client data; a model training module for training the first global model to obtain a second global model through a dynamic selection algorithm based on the first model parameters uploaded by each client and the current client resource status information; and a model sending iteration module for distributing the second global model to each client to enter the next iteration until a predefined stopping condition is met.

[0015] According to another aspect of this disclosure, an electronic device is also provided, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to perform the federated learning method described in any of the preceding claims by executing the executable instructions.

[0016] According to another aspect of this disclosure, a computer-readable storage medium is also provided, on which a computer program is stored, which, when executed by a processor, implements the federated learning method described in any of the preceding claims.

[0017] According to another aspect of this disclosure, a computer program product is also provided, including a computer program that, when executed by a processor, implements the federated learning method of any of the above.

[0018] The federated learning method provided in the embodiments of this disclosure involves the client acquiring a first global model sent by a central server; training the first global model based on client data to determine first model parameters; uploading the first model parameters and current client resource status information to the central server, so that the central server can train the first global model to obtain a second global model through a dynamic selection algorithm based on the first model parameters uploaded by each client and the current client resource status information; and receiving the second global model from the central server to enter the next iteration until a predefined stopping condition is met. This disclosure combines resource status with model parameters and, through a dynamic selection algorithm, can perceive and adapt to dynamic changes in the training environment, solving the problem that federated learning using fixed fusion algorithms lacks perception and adaptation to dynamic changes in the training environment, thereby improving the accuracy and efficiency of model training in federated learning.

[0019] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0020] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure. It is obvious that the drawings described below are merely some embodiments of this disclosure, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0021] Figure 1 This diagram illustrates a federated learning system architecture according to an embodiment of the present disclosure; Figure 2 This illustration shows a flowchart of a federated learning method applied to a client in an embodiment of this disclosure; Figure 3 A flowchart illustrating another specific example of a federated learning method applied to a client in this disclosure embodiment is shown; Figure 4 A flowchart illustrating yet another specific example of a federated learning method applied to a client in accordance with the embodiments of this disclosure is shown. Figure 5 This illustration shows a flowchart of a federated learning method applied to a central server in an embodiment of this disclosure; Figure 6 A flowchart illustrating another specific example of a federated learning method applied to a central server according to an embodiment of this disclosure is shown. Figure 7A schematic diagram of a federated learning device applied to a client is shown in an embodiment of this disclosure; Figure 8 This illustration shows a schematic diagram of a federated learning device applied to a central server in an embodiment of the present disclosure; Figure 9 A structural block diagram of an electronic device according to an embodiment of the present disclosure is shown. Detailed Implementation

[0022] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, they are provided so that this disclosure will be more comprehensive and complete, and will fully convey the concept of the exemplary embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0023] Furthermore, the accompanying drawings are merely illustrative of this disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0024] The specific implementation methods of the embodiments of this disclosure will now be described in detail with reference to the accompanying drawings.

[0025] Figure 1 A schematic diagram of an exemplary application system architecture to which the federated learning method of the embodiments of this disclosure can be applied is shown. For example... Figure 1 As shown, the system architecture may include a client 101, a network 102, and a central server 103.

[0026] Network 102 is a medium used to provide a communication link between terminal device 101 and server 103, and can be a wired network or a wireless network.

[0027] Optionally, the aforementioned wireless or wired networks use standard communication technologies and / or protocols. The network is typically the Internet, but can also be any network, including but not limited to Local Area Networks (LANs), Metropolitan Area Networks (MANs), Wide Area Networks (WANs), mobile, wired or wireless networks, private networks, or any combination of virtual private networks. In some embodiments, technologies and / or formats, including Hyper Text Markup Language (HTML), Extensible Markup Language (XML), etc., are used to represent data exchanged over the network. Furthermore, conventional encryption technologies such as Secure Socket Layer (SSL), Transport Layer Security (TLS), Virtual Private Networks (VPNs), and Internet Protocol Security (IPSec) can be used to encrypt all or some links. In other embodiments, custom and / or dedicated data communication technologies can be used to replace or supplement the aforementioned data communication technologies.

[0028] Optionally, the aforementioned clients are participants in the federated model training process. These are entities that possess local data and participate in the training. They can take various forms, including: terminal devices (including but not limited to smartphones, tablets, laptops, desktop computers, smart speakers, smartwatches, wearable devices, augmented reality devices, virtual reality devices, etc., for example, these devices locally store user data, calculate model updates, and upload them to the central server); and institutions or organizations (including but not limited to banks, enterprises, or medical institutions, for example, multiple banks collaborating to train a risk control model while protecting data privacy, each acting as an independent participant). It should be noted that all participants store the raw data locally and do not share it externally; they only transmit encrypted model parameters or gradient updates (such as gradient descent results) to the central server.

[0029] In one example of this disclosure, the client obtains a first global model sent by the central server; the client trains the first global model based on client data to determine the first model parameters; the client uploads the first model parameters and current client resource status information to the central server, so that the central server trains the first global model to obtain a second global model through a dynamic selection algorithm based on the first model parameters uploaded by each client and the current client resource status information; the client receives the second global model sent by the central server and enters the next iteration until a predefined stopping condition is met.

[0030] Central server 103 is the core coordinator, not merely a storage server. It serves as the central coordination hub in the federated learning architecture, its primary responsibility being to aggregate model updates from participants rather than store raw data. The central server does not access the raw data; it only processes encrypted intermediate results (such as gradients or parameters), ensuring data privacy. It is a dedicated server responsible for coordination tasks.

[0031] In one example of this disclosure, the central server sends a first global model to each client; the central server obtains the first model parameters and current client resource status information sent by each client, wherein the first model parameters are determined by the client through training the first global model based on client data; the central server trains the first global model using a dynamic selection algorithm based on the first model parameters uploaded by each client and the current client resource status information to obtain a second global model; the central server distributes the second global model to each client to enter the next iteration, until a predefined stopping condition is met.

[0032] Those skilled in the art will know that Figure 1 The number of clients, networks, and central servers shown is merely illustrative; any number of terminal devices, networks, and servers can be included as needed. This disclosure does not limit the scope of the embodiments.

[0033] Figure 2 This illustration shows a flowchart of a federated learning method applied to a client in an embodiment of this disclosure, such as... Figure 2 As shown, the federated learning method applied to the client provided in this embodiment includes the following steps: S202, Obtain the first global model sent by the central server.

[0034] It should be noted that the first global model mentioned above can be the initial global model, which is uniformly defined and distributed by the server during the initialization phase.

[0035] For example, the central server initializes global model parameters. Set the initial fusion algorithm to federated weighted average algorithm and configure the parameters. (Learning rate) (Number of local training rounds) (Sliding window size).

[0036] S204, Train the first global model based on client data and determine the parameters of the first model.

[0037] It should be noted that the client-side data mentioned above can be data stored locally on the client side. The training described above can be local training. The first model parameters mentioned above include the loss value and accuracy. For example, after receiving the global model from the server, the client trains the model using its local data and calculates the local loss value. and accuracy .

[0038] S206, the first model parameters and the current client resource status information are uploaded to the central server, so that the central server can train the first global model and obtain the second global model through a dynamic selection algorithm based on the first model parameters uploaded by each client and the current client resource status information.

[0039] It should be noted that the aforementioned dynamic selection algorithm can dynamically adopt different federated model algorithms based on different situations. These federated model algorithms include: FedAvg (Federated Weighted Average); FedProx (Federated Proxest Term); FedNova (Federated Normalization); and FedBuff (Federated Buffer). FedProx adds a proxest term to the client's local objective function, restricting local updates to the vicinity of the global model. This constrains local updates from deviating too far from the global model, thereby improving model stability and convergence, reducing client drift issues caused by data heterogeneity, and is suitable for scenarios with high levels of data heterogeneity (Non-IID). FedNova eliminates bias caused by different local update steps by normalizing the update weights of clients, making aggregation more equitable. Normalized aggregation refers to normalizing the update amount on the server side based on the actual number of update steps executed by each client, and then performing a weighted average. This can solve the aggregation bias problem caused by directly weighting the average when clients execute different numbers of local update steps due to different computing capabilities (or different amounts of local data). It is suitable for scenarios with heterogeneous client computing capabilities (e.g., some devices compute quickly and have more local iterations; some devices compute slowly and have fewer local iterations). FedBuff allows clients to asynchronously upload updates. The server stores updates in a buffer, and aggregates them when the buffer reaches a certain quantity or a certain condition is met. Clients upload updates immediately after completing local training, without waiting for other clients. Through the buffer mechanism: the server sets up a buffer, and when the buffer is full (reaching a predetermined quantity) or a trigger condition is met (such as timeout), the server aggregates the updates in the buffer and updates the global model. This can solve the latency problem caused by waiting for all clients in synchronous federated learning (such as FedAvg). It is suitable for scenarios with large differences in device response time (such as edge computing), limited network bandwidth, and scenarios that require fast updates.

[0040] Client resource status information includes hardware load information and network load information. Specifically, hardware load information includes GPU utilization; network load information includes bandwidth availability. For example, the client uploads trained model parameters. and resource status information (GPU utilization) Bandwidth availability ).

[0041] S208: Receive the second global model sent by the central server and enter the next iteration until the predefined stopping condition is met.

[0042] It should be noted that entering the next iteration can be a repetition of steps S202-S206. For example, the central server aggregates the client model parameters according to the selected algorithm to obtain a new global model (equivalent to the second global model mentioned above), and distributes the new global model to the client to start the next round of training.

[0043] This disclosure combines resource status with model parameters and uses a dynamic selection algorithm to perceive and adapt to dynamic changes in the training environment. This solves the problem that federated learning, which uses fixed fusion algorithms, lacks the ability to perceive and adapt to dynamic changes in the training environment, thereby improving the accuracy and efficiency of model training in federated learning.

[0044] In one embodiment of this disclosure, such as Figure 3 As shown in the embodiments of this disclosure, the client-side federated learning method can achieve dynamic algorithm selection through the following steps: by using a dynamic fusion strategy selection mechanism, the optimal fusion algorithm is adaptively selected based on multi-dimensional evaluation results, thereby improving the stability and efficiency of federated learning. S302, when the model loss of the second global model increases beyond the first threshold during training, or when the difference in accuracy among clients exceeds the second threshold, or when hardware load information exceeds the third threshold or network load information exceeds the fourth threshold, the dynamic selection algorithm is triggered: S304, if the current algorithm is the federated weighted average algorithm, switch to the federated near-end algorithm; S306, If the current algorithm is the Federated Near End Algorithm, switch to the Federated Weighted Average Algorithm; S308, if the current algorithm is federated normalization, switch to federated buffer algorithm; S310, if the current algorithm is the federated buffer algorithm, switch to the federated normalization algorithm.

[0045] This disclosure achieves a reasonable and effective timing and method for algorithm switching by using triggering conditions and a dynamic selection algorithm switching strategy.

[0046] For example, in a specific instance, specifically a dynamic algorithm selection based on gradient diversity and moving average loss, the federated learning system first initializes, with the central server initializing the global model parameters (equivalent to the first global model mentioned above). Subsequently, the central server initiates monitoring, collecting client resource status information and setting trigger condition thresholds as follows: (Loss rise threshold, equivalent to the first threshold mentioned above) (Accuracy variance threshold, equivalent to the second threshold mentioned above) (Bandwidth threshold, equivalent to the fourth threshold mentioned above) (GPU load threshold, equivalent to the third threshold mentioned above). Each training round includes: 1. Central server distributes the model: The central server distributes the current global model... 1. Distribute to all participants. 2. Client-side local training: Each client trains the model using local data and calculates the local loss. and accuracy The client uploads the trained model parameters. and resource status information. 3. Central server monitoring and evaluation: Monitoring phase: Collect local loss information from all clients. Accuracy GPU utilization Bandwidth availability Evaluation phase: Calculating gradient diversity ,in Let $\frac{i}{i}$ be the sample size proportion for client $i$. Calculate the moving average $Loss$. Calculate the variance of accuracy. Computing resource status metrics: Average GPU utilization Average bandwidth availability 4. Dynamic selection algorithm: Scoring function: ,in , , Triggering condition: When and When, the algorithm is triggered to select. Or when or This triggers algorithm selection. The scoring function comprehensively considers multiple factors, enabling automatic selection of the optimal algorithm under different training stages and resource conditions, improving model accuracy and training efficiency. Furthermore, based on multi-dimensional evaluation results, the optimal fusion algorithm is dynamically selected through the scoring function and triggering conditions. Algorithm selection logic: If... If the current algorithm is FedAvg federated weighted average, then switch to FedProx and set... ;like If the current algorithm is FedProx Federated Near-End, then switch back to FedAvg. 5. Model Aggregation and Distribution: The central server aggregates client model parameters according to the selected algorithm (FedAvg or FedProx) to obtain a new global model. The new global model is deployed to the client to begin the next round of training. Finally, the algorithm switching implementation includes: FedProx local training formula: ,in These are the coefficients of the proximal terms. The algorithm switching logic is a central server decision-making logic. For example, when model performance deteriorates and client performance varies too much, switch to the FedProx algorithm and set the regularization strength; when resources are limited (insufficient bandwidth or GPU overload), switch to the FedProx algorithm and set the regularization strength; by default, when resources are sufficient and model performance is stable, use the FedAvg algorithm (without regularization constraints).

[0047] This disclosure presents a dynamic algorithm selection method based on gradient diversity and moving average loss. By comprehensively considering the diversity of gradients and moving average loss, it can more accurately switch between different algorithms, thereby improving the learning efficiency and performance of the model in complex data environments. It can effectively utilize the differences in gradient information and the long-term trend of loss, providing a more comprehensive basis for algorithm selection.

[0048] In another specific example, the scenario involves dynamic algorithm selection for resource-constrained environments. First, the system initializes, with the central server initializing global model parameters. Set the initial fusion algorithm to FedNova and configure the parameters. (Learning rate) (Number of local training rounds) (Sliding window size). The central server starts the monitoring module and begins collecting client resource status. Set trigger condition thresholds: (Bandwidth threshold) (GPU load threshold). Each training round includes: 1. Central server distributes the model: The central server distributes the current global model. 1. Distribute to all participants. 2. Client-side local training: Each client trains the model using local data and calculates the local loss value. and accuracy The client uploads the trained model parameters. and resource status information. 3. Central server monitoring and evaluation: Monitoring phase: Collect local loss information from all clients. Accuracy GPU utilization Bandwidth availability Evaluation phase: Calculate bandwidth availability. and GPU utilization Calculate communication time 4. Dynamic Selection Algorithm: Triggering Condition: When And when the current algorithm is FedNova federated normalization, switch to FedBuff and set the edge computing parameters. When Furthermore, if the current algorithm is FedBuff, switch back to FedNova. Algorithm selection logic: If bandwidth availability is below a threshold and the current algorithm is FedNova, switch to FedBuff, employing an edge computing architecture. If GPU utilization is above a threshold and the current algorithm is FedBuff, switch back to FedNova to reduce computational overhead. 5. Model aggregation and distribution: The central server aggregates client model parameters based on the selected algorithm (FedNova or FedBuff) to obtain a new global model. The new global model is deployed to the client to begin the next round of training. Finally, the algorithm switching implementation includes: FedBuff aggregation formula: ,in Weights are based on communication latency. The algorithm switching logic is a central server decision-making logic. For example, when network bandwidth is insufficient and the FedNova algorithm is currently being used, switch to the FedBuff algorithm and enable edge computing mode; when GPU resources are overloaded and the FedBuff algorithm is currently being used, switch back to the FedNova algorithm and disable edge computing; the default situation is to maintain the current algorithm or the initial state and use the FedNova algorithm (edge ​​computing is disabled).

[0049] This disclosure focuses on dynamic algorithm selection for resource-constrained scenarios. It aims to dynamically select the most suitable algorithm based on real-time conditions when resources are limited, ensuring high operating efficiency and performance even when hardware resources are scarce. It fully considers the key factor of resource constraints and enables the system to operate efficiently in constrained environments through flexible algorithm switching strategies.

[0050] In one embodiment of this disclosure, such as Figure 4 As shown in the embodiments of this disclosure, the federated learning method applied to the client, which receives the second global model issued by the central server and proceeds to the next iteration until a predefined stopping condition is met, includes the following steps: S402, Train the second global model based on client data and determine the parameters of the second model; S404: Upload the second model parameters and the current client resource status information to the central server. Stop iterating when the preset number of iterations is reached or the global model achieves the preset performance on the verification data.

[0051] Figure 5 This invention discloses a flowchart of a federated learning method applied to a central server, as shown in an embodiment of the present disclosure. Figure 5 As shown, the federated learning method applied to a central server provided in this embodiment includes the following steps: S502 sends the first global model to each client; S504, obtain the first model parameters and current client resource status information sent by each client, wherein the first model parameters are determined by the client by training the first global model based on the client data; S506, based on the first model parameters uploaded by each client and the current client resource status information, the first global model is trained through a dynamic selection algorithm to obtain the second global model; S508 distributes the second global model to each client to enter the next iteration until the predefined stopping condition is met.

[0052] In one embodiment of this disclosure, such as Figure 6 As shown, the federated learning method applied to the client provided in this embodiment of the disclosure, which distributes the second global model to each client to enter the next iteration until a predefined stopping condition is met, includes the following steps: S602, obtain the second model parameters sent by each client and the current client resource status information; S604: Based on the first model parameters uploaded by each client and the current client resource status information, a second global model is trained using a dynamic selection algorithm. The iteration stops when the preset number of iterations is reached or the global model achieves the preset performance on the validation data.

[0053] Based on the same inventive concept, this disclosure also provides two federated learning devices, as described in the following embodiments. Since the principle by which these devices solve the problem is similar to that of the method embodiments described above, the implementation of these devices can refer to the implementation of the method embodiments described above, and repeated details will not be repeated.

[0054] Figure 7 This illustration shows a schematic diagram of a federated learning device applied to a client in an embodiment of this disclosure, such as... Figure 7 As shown, the device includes: a model acquisition module 71, a parameter training module 72, an information uploading module 73, and a model acquisition iteration module 74.

[0055] Among them, the model acquisition module 71 is used to acquire the first global model sent by the central server.

[0056] The parameter training module 72 is used to train the first global model based on client data and determine the parameters of the first model.

[0057] The information upload module 73 is used to upload the first model parameters and the current client resource status information to the central server, so that the central server can train the first global model and obtain the second global model through a dynamic selection algorithm based on the first model parameters and the current client resource status information uploaded by each client.

[0058] The model acquisition iteration module 74 is used to receive the second global model issued by the central server and enter the next round of iteration until the predefined stopping condition is met.

[0059] In one example of this disclosure, the aforementioned client resource status information applied to the client-side federated learning device includes: hardware load information and network load information.

[0060] In one example of this disclosure, the dynamic selection algorithm in the federated learning device applied to the client includes: when the model loss of the second global model increases beyond a first threshold, the difference in accuracy among clients exceeds a second threshold, or when hardware load information exceeds a third threshold or network load information exceeds a fourth threshold, the dynamic selection algorithm is triggered: if the current algorithm is a federated weighted average algorithm, switch to a federated near-end algorithm; if the current algorithm is a federated near-end algorithm, switch to a federated weighted average algorithm; if the current algorithm is a federated normalization algorithm, switch to a federated buffer algorithm; if the current algorithm is a federated buffer algorithm, switch to a federated normalization algorithm.

[0061] In one example of this disclosure, the model acquisition iteration module is further configured to: train the second global model based on client data to determine the parameters of the second model; upload the second model parameters and the current client resource status information to the central server; and stop iterating when the preset number of iterations is reached or the global model achieves the preset performance on the validation data.

[0062] It should be noted that the model acquisition module 71, parameter training module 72, information upload module 73, and model acquisition iteration module 74 mentioned above correspond to S202 to S208 in the method embodiment. The examples and application scenarios implemented by the above modules and corresponding steps are the same, but are not limited to the content disclosed in the above method embodiment. It should be noted that the above modules, as part of the device, can be executed in a computer system such as a set of computer-executable instructions.

[0063] Figure 8 This illustration shows a schematic diagram of a federated learning device applied to a central server according to an embodiment of this disclosure, such as... Figure 8 As shown, the device includes: a model sending module 81, an information acquisition module 82, a model training module 83, and a model sending iteration module 84.

[0064] The model sending module 81 is used to send the first global model to each client.

[0065] The information acquisition module 82 is used to acquire the first model parameters and current client resource status information sent by each client. The first model parameters are determined by the client through training the first global model based on the client data.

[0066] The model training module 83 is used to train the first global model and obtain the second global model by using a dynamic selection algorithm based on the first model parameters uploaded by each client and the current client resource status information.

[0067] The model sending iteration module 84 is used to send the second global model to each client to enter the next round of iteration until the predefined stopping condition is met.

[0068] In one example of this disclosure, the aforementioned model sending iteration module is further configured to: obtain the second model parameters sent by each client and the current client resource status information; train the second global model using a dynamic selection algorithm based on the first model parameters uploaded by each client and the current client resource status information; and stop iterating when the preset number of iterations is reached or the global model achieves a preset performance on the validation data.

[0069] It should be noted that the model sending module 81, information acquisition module 82, model training module 83, and model sending iteration module 84 mentioned above correspond to S502 to S508 in the method embodiment. The examples and application scenarios implemented by the above modules and corresponding steps are the same, but are not limited to the content disclosed in the above method embodiment. It should be noted that the above modules, as part of the apparatus, can be executed in a computer system such as a set of computer-executable instructions.

[0070] Those skilled in the art will understand that various aspects of this disclosure can be implemented as a system, method, or program product. Therefore, various aspects of this disclosure can be specifically implemented in the following forms: a completely hardware implementation, a completely software implementation (including firmware, microcode, etc.), or a combination of hardware and software aspects, collectively referred to herein as a "circuit," "module," or "system."

[0071] The following reference Figure 9 To describe an electronic device 900 according to such an embodiment of the present disclosure. Figure 9 The electronic device 900 shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments disclosed herein.

[0072] like Figure 9 As shown, the electronic device 900 is manifested in the form of a general-purpose computing device. The components of the electronic device 900 may include, but are not limited to: at least one processing unit 910, at least one storage unit 920, and a bus 930 connecting different system components (including storage unit 920 and processing unit 910).

[0073] The storage unit stores program code that can be executed by the processing unit 910, causing the processing unit 910 to perform the steps described in the "Exemplary Methods" section above according to various exemplary embodiments of this disclosure.

[0074] For example, the processing unit 910 can execute the following steps of the above method embodiment: obtain the first global model sent by the central server; train the first global model according to the client data to determine the first model parameters; upload the first model parameters and the current client resource status information to the central server, so that the central server can train the first global model to obtain the second global model through a dynamic selection algorithm based on the first model parameters uploaded by each client and the current client resource status information; receive the second global model sent by the central server to enter the next iteration until the predefined stopping condition is met.

[0075] For example, the processing unit 910 can execute the following steps of the above method embodiment: when the model loss of the second global model increases beyond a first threshold, the difference in accuracy among clients exceeds a second threshold, or when the hardware load information exceeds a third threshold or the network load information exceeds a fourth threshold, a dynamic selection algorithm is triggered: if the current algorithm is federated weighted average, switch to federated near-end algorithm; if the current algorithm is federated near-end algorithm, switch to federated weighted average algorithm; if the current algorithm is federated normalization algorithm, switch to federated buffer algorithm; if the current algorithm is federated buffer algorithm, switch to federated normalization algorithm.

[0076] For example, the processing unit 910 can perform the following steps in the above method embodiment: train the second global model based on client data to determine the second model parameters; upload the second model parameters and the current client resource status information to the central server; and stop iterating when the preset number of iterations is reached or the global model achieves the preset performance on the verification data.

[0077] For example, the processing unit 910 can execute the following steps of the above method embodiment: sending the first global model to each client; obtaining the first model parameters and current client resource status information sent by each client, wherein the first model parameters are determined by the client through training the first global model based on client data; training the first global model through a dynamic selection algorithm to obtain the second global model based on the first model parameters uploaded by each client and the current client resource status information; and distributing the second global model to each client to enter the next iteration until a predefined stopping condition is met.

[0078] For example, the processing unit 910 can perform the following steps in the above method embodiment: obtain the second model parameters sent by each client and the current client resource status information; train the second global model through a dynamic selection algorithm based on the first model parameters uploaded by each client and the current client resource status information, and stop iterating when the preset number of iterations is reached or the global model achieves the preset performance on the verification data.

[0079] Storage unit 920 may include readable media in the form of volatile storage units, such as random access memory (RAM) 9201 and / or cache 9202, and may further include read-only memory (ROM) 9203.

[0080] The storage unit 920 may also include a program / utility 9204 having a set (at least one) of program modules 9205, including but not limited to: an operating system, one or more application programs, other program modules, and program data, each or some combination of these examples may include an implementation of a network environment.

[0081] Bus 930 can represent one or more of several types of bus structures, including a memory cell bus or memory cell controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any of the various bus structures.

[0082] Electronic device 900 can also communicate with one or more external devices 940 (e.g., keyboard, pointing device, Bluetooth device, etc.), one or more devices that enable a user to interact with electronic device 900, and / or any device that enables electronic device 900 to communicate with one or more other computing devices (e.g., router, modem, etc.). This communication can be performed via input / output (I / O) interface 950. Furthermore, electronic device 900 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 960. As shown, network adapter 960 communicates with other modules of electronic device 900 via bus 930. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with electronic device 900, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0083] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, terminal device, or network device, etc.) to execute the methods according to the embodiments of this disclosure.

[0084] In particular, according to embodiments of this disclosure, the process described above with reference to the flowchart can be implemented as a computer program product, which includes a computer program that, when executed by a processor, implements the above-described federated learning method.

[0085] In exemplary embodiments of this disclosure, a computer-readable storage medium is also provided, which may be a readable signal medium or a readable storage medium. A program product capable of implementing the methods described above is stored thereon. In some possible implementations, various aspects of this disclosure may also be implemented as a program product including program code, which, when run on a terminal device, causes the terminal device to perform the steps described in the "Exemplary Methods" section of this specification according to various exemplary embodiments of this disclosure.

[0086] More specific examples of computer-readable storage media in this disclosure may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0087] In this disclosure, a computer-readable storage medium may include a data signal propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable signal medium may also be any readable medium other than a readable storage medium, capable of transmitting, propagating, or transmitting a program for use by or in connection with an instruction execution system, apparatus, or device.

[0088] Optionally, the program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0089] In practical implementation, program code for performing the operations of this disclosure can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the user's computing device, partially on the user's device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0090] It should be noted that although several modules or units for the device used to perform actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to embodiments of this disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by individual modules or units.

[0091] Furthermore, although the steps of the method in this disclosure are described in a specific order in the accompanying drawings, this does not require or imply that the steps must be performed in that specific order, or that all the steps shown must be performed to achieve the desired result. Additional or alternative steps may be omitted, steps may be combined into a single step, and / or a single step may be broken down into separate steps.

[0092] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, mobile terminal, or network device, etc.) to execute the methods according to the embodiments of this disclosure.

[0093] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the appended claims.

Claims

1. A federated learning method, characterized in that, The application is applied to a client, comprising: obtaining a first global model sent by a central server; training the first global model according to client data to determine first model parameters; uploading the first model parameters and current client resource state information to the central server, so that the central server trains the first global model according to the first model parameters and current client resource state information uploaded by each client through a dynamic selection algorithm to obtain a second global model; receiving the second global model issued by the central server to enter the next round of iteration until a predefined stop condition is met.

2. The federated learning method of claim 1, wherein, The client resource state information comprises hardware load information and network load information; the dynamic selection algorithm comprises: when the model loss of the second global model rises by more than a first threshold value in the training process, the accuracy difference of each client exceeds a second threshold value, or when the hardware load information exceeds a third threshold value and the network load information exceeds a fourth threshold value, the dynamic selection algorithm is triggered: if the current algorithm is a federated weighted average algorithm, the federated proximal algorithm is switched to; if the current algorithm is a federated proximal algorithm, the federated weighted average algorithm is switched to; if the current algorithm is a federated normalization algorithm, the federated buffer algorithm is switched to; if the current algorithm is a federated buffer algorithm, the federated normalization algorithm is switched to.

3. The federated learning method of claim 1, wherein, The receiving of the second global model issued by the central server to enter the next round of iteration until the predefined stop condition is met comprises: training the second global model according to client data to determine second model parameters; uploading the second model parameters and current client resource state information to the central server, and stopping iteration when a preset selection frequency is reached or the global model reaches a preset performance on validation data.

4. A federated learning method, characterized in that, The application is applied to a central server, comprising: sending a first global model to each client; obtaining first model parameters and current client resource state information sent by each client, wherein the first model parameters are determined by training the first global model according to client data by the client; training the first global model according to the first model parameters and current client resource state information uploaded by each client through a dynamic selection algorithm to obtain a second global model; issuing the second global model to each client to enter the next round of iteration until a predefined stop condition is met.

5. The federated learning method of claim 4, wherein, The client resource state information comprises hardware load information and network load information; the dynamic selection algorithm comprises: when the model loss of the second global model rises by more than a first threshold value in the training process, the accuracy difference of each client exceeds a second threshold value, or when the hardware load information exceeds a third threshold value and the network load information exceeds a fourth threshold value, the dynamic selection algorithm is triggered: if the current algorithm is a federated weighted average algorithm, the federated proximal algorithm is switched to; if the current algorithm is a federated proximal algorithm, the federated weighted average algorithm is switched to; if the current algorithm is a federated normalization algorithm, the federated buffer algorithm is switched to; if the current algorithm is a federated buffer algorithm, the federated normalization algorithm is switched to.

6. The federated learning method of claim 4, wherein, The issuing of the second global model to each client to enter the next round of iteration until the predefined stop condition is met comprises: obtain the second model parameters and the current client resource state information sent by each client; train the second global model according to the first model parameters and the current client resource state information uploaded by each client through a dynamic selection algorithm, and stop iteration when a preset number of iterations is reached or the global model reaches a preset performance on validation data.

7. A federated learning apparatus, comprising: Applied to a client, comprising: a model obtaining module configured to obtain a first global model sent by a central server; a parameter training module configured to train the first global model according to client data to determine first model parameters; an information uploading module configured to upload the first model parameters and current client resource state information to the central server, so that the central server trains the first global model to obtain a second global model according to the first model parameters and the current client resource state information uploaded by each client through a dynamic selection algorithm; a model obtaining iteration module configured to receive the second global model sent by the central server to enter the next round of iteration until a predefined stop condition is met.

8. A federated learning apparatus, comprising: Applied to a central server, comprising: a model sending module configured to send a first global model to each client; an information obtaining module configured to obtain first model parameters and current client resource state information sent by each client, wherein the first model parameters are determined by the client according to training of the first global model based on client data; a model training module configured to train the first global model to obtain a second global model according to the first model parameters and the current client resource state information uploaded by each client through a dynamic selection algorithm; a model sending iteration module configured to send the second global model to each client to enter the next round of iteration until a predefined stop condition is met.

9. An electronic device, comprising: comprising: a processor; and a memory configured to store executable instructions of the processor; wherein the processor is configured to execute the federated learning method of any one of claims 1 to 6 by executing the executable instructions.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the federated learning method of any one of claims 1 to 6.

11. A computer program product, comprising: Computer program or instructions, characterized in that the computer program or instructions are executed by the processor to implement the federated learning method of any one of claims 1 to 6.

Images