Node coordination method and apparatus, network, node device, and storage medium

Abstract

The application discloses a node cooperation method and device, a network, a node device and a storage medium. The method comprises the following steps: obtaining model parameters sent by follower nodes in a federated learning network; in response to the newly accumulated model parameters in the current iteration being not less than a preset share, performing fusion based on the newly accumulated model parameters to obtain fusion parameters, and sending the fusion parameters to each follower node; re-executing the step of obtaining the model parameters sent by the follower nodes in the federated learning network and subsequent steps until the countdown of the current iteration ends; performing fusion based on the fusion parameters in the current iteration process to obtain final parameters of the current iteration, and sending the final parameters to the follower nodes; and starting a new round of iteration based on the final parameters. Through the above method, the information of each participant node can be synchronized, and the efficiency of model iteration can be improved.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a node collaboration method, apparatus, network, node device, and storage medium. Background Technology

[0002] Federated learning, as a newly emerging foundational artificial intelligence technology in recent years, has been widely applied in various industries and fields. Currently, centralized federated learning models for improving communication efficiency already exist, and there are already methods to alleviate the communication bottleneck of the central node and to select parameters for the fusion model. However, the problem of how to achieve information consistency among all parties when each node does not know each other and is uncertain about the state of each other still exists. Summary of the Invention

[0003] The main technical problem addressed by this application is to provide a node collaboration method, apparatus, network, node device, and storage medium that can ensure information synchronization among participating nodes and improve the efficiency of model iteration.

[0004] To address the aforementioned technical problems, the first aspect of this application provides a node collaboration method, comprising: acquiring model parameters sent by follower nodes in a federated learning network; wherein follower nodes send model parameters to the leader node in the federated learning network based on completing a model training task, and the leader node also performs a model training task; in response to the newly accumulated model parameters in this iteration being no less than a preset number, fusing the newly accumulated model parameters to obtain fused parameters, and sending the fused parameters to each follower node; wherein each participating node, in response to the fused parameters, stops its current training and re-executes the model training task based on the fused parameters; re-executes the step of acquiring model parameters sent by follower nodes in the federated learning network and subsequent steps until the countdown of this iteration ends; fusing the fused parameters in this iteration to obtain the final parameters of this iteration, and sending the final parameters to the follower nodes; and starting a new iteration based on the final parameters.

[0005] To address the aforementioned technical issues, a second aspect of this application provides a node collaboration method, comprising: sending model parameters after completing a model training task to a leader node in a federated learning network; wherein the leader node also performs a model training task, and in response to the fact that the newly accumulated model parameters in this iteration are not less than a preset number, the leader node fuses the newly accumulated model parameters to obtain fused parameters; receiving the fused model sent by the leader node, stopping the current training, and re-executing the model training task based on the fused parameters; re-executing the step of sending the model parameters after completing the model training task to the leader node in the federated learning network and subsequent steps until the countdown of this iteration ends; wherein, at the end of the countdown of this iteration, the leader node obtains the final parameters of this iteration based on the fused parameters during this iteration; receiving the final parameters, and starting a new round of iteration based on the final parameters.

[0006] To address the aforementioned technical issues, a third aspect of this application provides a node collaboration method, comprising: follower nodes sending model parameters after completing a model training task to a leader node in a federated learning network; wherein the leader node also performs a model training task; in response to the leader node having accumulated model parameters of not less than a preset number in the current iteration, the leader node fuses the newly accumulated model parameters to obtain fused parameters and sends the fused parameters to each follower node; follower nodes receive the fused model parameters sent by the leader node, stop the current training, and re-execute the model training task based on the fused parameters; repeating the steps of follower nodes sending model parameters after completing a model training task to the leader node in the federated learning network and subsequent steps until the countdown of the current iteration ends; the leader node fuses the fused parameters in the current iteration to obtain the final parameters of the current iteration and sends the final parameters to the follower nodes; the leader node and follower nodes start a new round of iteration based on the final parameters.

[0007] To address the aforementioned technical problems, a fourth aspect of this application provides a node collaboration device, comprising an acquisition module, a fusion module, a looping module, and a termination module. The acquisition module acquires model parameters sent by follower nodes in a federated learning network. Follower nodes send model parameters to the leader node in the federated learning network based on the completion of a model training task, and the leader node also performs a model training task. The fusion module, in response to the newly accumulated model parameters in the current iteration being no less than a preset number, fuses the newly accumulated model parameters to obtain fused parameters and sends the fused parameters to each follower node. Each participating node, in response to the fused parameters, stops its current training and re-executes the model training task based on the fused parameters. The looping module re-executes the step of acquiring model parameters sent by follower nodes in the federated learning network and subsequent steps until the countdown of the current iteration ends. The termination module fuses the fused parameters during the current iteration to obtain the final parameters of the current iteration and sends the final parameters to the follower nodes, and starts a new iteration based on the final parameters.

[0008] To address the aforementioned technical problems, a fifth aspect of this application provides a node coordination device, comprising a sending module, a stop module, a looping module, and a restart module. The sending module sends model parameters after completing the model training task to the leader node in the federated learning network. The leader node also performs a model training task, and in response to the newly accumulated model parameters in this iteration being no less than a preset number, it fuses the newly accumulated model parameters to obtain fused parameters. The stop module receives the fused model from the leader node, stops the current training, and re-executes the model training task based on the fused parameters. The looping module re-executes the step of sending the model parameters after completing the model training task to the leader node in the federated learning network, as well as subsequent steps, until the countdown of the current iteration ends. At the end of the countdown for the current iteration, the leader node obtains the final parameters for the current iteration based on the fused parameters accumulated during the current iteration. The restart module receives the final parameters and starts a new iteration based on them.

[0009] To address the aforementioned technical problems, a sixth aspect of this application provides a node device comprising a memory and a processor. The memory stores program instructions, and the processor executes the program instructions to implement the aforementioned node collaboration method.

[0010] To address the aforementioned technical problems, a seventh aspect of this application provides a node collaboration network, which includes several participating nodes connected by communication, the participating nodes being the aforementioned node devices, wherein the participating nodes in the node collaboration network elect a leader node and at least one follower node.

[0011] To address the aforementioned technical problems, an eighth aspect of this application provides a computer-readable storage medium storing program instructions executable by a processor, the program instructions being used to implement the aforementioned node collaboration method.

[0012] The above technical solution, when the newly accumulated model parameters in the current iteration are not less than a preset number, fuses the newly accumulated model parameters to obtain fused parameters; then, it re-executes the step of obtaining the model parameters sent by the follower nodes in the federated learning network, as well as subsequent steps, until the countdown of the current iteration ends; based on the fused parameters obtained during the current iteration, it fuses the parameters to obtain the final parameters of the current iteration, sends the final parameters to the follower nodes, and starts a new round of iteration based on the final parameters. Therefore, by using both countdown timer and preset score mechanisms to simultaneously control the model update, the iteration efficiency of the model is guaranteed; in addition, the leader node sends the final parameters to each follower node, thereby synchronizing information between the nodes. Attached Figure Description

[0013] Figure 1 This is a flowchart illustrating an embodiment of the node collaboration method provided in this application;

[0014] Figure 2 This is the logic diagram of role switching provided in this application;

[0015] Figure 3 This is a schematic diagram of the log replication process provided in this application;

[0016] Figure 4 This is a logic diagram of an embodiment of node collaboration provided in this application;

[0017] Figure 5 This is a flowchart illustrating another embodiment of the node collaboration method provided in this application;

[0018] Figure 6 This is a flowchart illustrating another embodiment of the node collaboration method provided in this application;

[0019] Figure 7 This is a schematic diagram of the structure of an embodiment of the node collaboration device provided in this application;

[0020] Figure 8 This is a schematic diagram of another embodiment of the node collaboration device provided in this application;

[0021] Figure 9 This is a schematic diagram of the structure of an embodiment of the node device provided in this application;

[0022] Figure 10 This is a schematic diagram of an embodiment of the computer-readable storage medium provided in this application. Detailed Implementation

[0023] The embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0024] In the following description, specific details such as particular system architectures, interfaces, and technologies are presented for illustrative purposes rather than for limiting purposes, in order to provide a thorough understanding of this application.

[0025] In this document, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " generally indicates that the preceding and following related objects have an "or" relationship. Furthermore, "many" in this document means two or more. Moreover, the term "at least one" in this document means any combination of at least two of any one or more of a plurality of objects. For example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C.

[0026] Please see Figure 1 , Figure 1 This is a flowchart illustrating an embodiment of the node collaboration method provided in this application. It should be noted that if substantially the same result is achieved, this embodiment does not necessarily reflect that outcome. Figure 1 The illustrated process sequence is limited. For example... Figure 1 As shown, this embodiment includes:

[0027] Step S11: Obtain the model parameters sent by the follower nodes in the federated learning network.

[0028] Existing federated learning networks do not consider how to achieve consistency of information among participants in a decentralized architecture, where each node (participant) is unaware of each other's identities and states. This embodiment addresses this issue by using a consensus mechanism for federated learning. The consensus mechanism is the process by which multiple participating nodes in a federated learning network achieve a consistent result during information exchange, ensuring consensus synchronization among all participating nodes in the network.

[0029] It's important to note that in a federated learning network, participating nodes are categorized into three roles: leader nodes, follower nodes, and candidate nodes. The leader node controls the fusion and development of the model and synchronizes the fused model parameters to other follower nodes. Candidate nodes represent intermediate states between follower and leader nodes. In a federated learning network, all participating nodes initially function as followers. After the leader election, the remaining nodes that do not become leaders become followers, and these followers must send their trained model parameters to the leader node. Furthermore, regardless of their role, all participating nodes are required to perform model training tasks.

[0030] In this embodiment, the leader node obtains model parameters sent by follower nodes in the federated learning network. Follower nodes send model parameters to the leader node after completing their model training tasks, and the leader node also performs its own model training task. In other words, all participating nodes in the federated learning network, i.e., nodes of each role, need to perform model training tasks locally. Furthermore, after completing their model training tasks, follower nodes in the federated learning network send their trained model parameters to the leader node. Therefore, the leader node in the federated learning network can obtain the model parameters sent by follower nodes after completing their model training tasks. It should also be noted that when each participating node in the federated learning network performs its first model training task, it needs to initialize the model locally (e.g., establish the same model state, training termination conditions, fusion methods, etc.) before executing the model training task.

[0031] Since all participating nodes in a federated learning network are initially follower nodes, a leader node needs to be selected from among them. Therefore, in one implementation, before obtaining the model parameters sent by the follower nodes in the federated learning network, in response to the first completion of model training after model initialization, a participating node sends a voting request to participating nodes other than itself in the federated learning network. Here, a participating node only votes for the node to which the first voting request is received. Then, based on receiving a preset percentage or more of the votes, the node switches its role to become the leader node. In other words, after completing the model training task for the first time after model initialization, a participating node in the federated learning network sends a voting request to other participating nodes in the network; if it receives a preset percentage or more of the votes (it can also vote for itself), it switches its role to become the leader node, i.e., the participating node that receives a preset percentage or more of the votes becomes the leader node of the federated learning network.

[0032] There is no fixed limit to the preset ratio; it can be set according to actual needs. For example, the preset ratio is 50%. It should be noted that there can be a maximum of one leader node in the same federated learning network.

[0033] In one specific implementation, after sending a voting request to participating nodes outside itself in the federated learning network, and before switching its role as a leader node based on receiving a preset proportion of votes, it also switches its role as a candidate node; then, in response to the leader node being elected in the federated learning network, it switches its role as a follower node. Specifically, as... Figure 2 As shown, Figure 2 This is the logic diagram of role switching provided in this application. After the participating node in the federated learning network completes the model training task for the first time after model initialization, it will send a voting request to other participating nodes in the federated learning network. At this time, the participating node switches from a follower node to a candidate node. If a leader node has been elected in the federated learning network, the participating node switches from a candidate node to a follower node. If it receives more than a preset percentage of votes, the participating node switches from a candidate node to a leader node.

[0034] In one implementation, after switching roles to become the leader node, a notification is sent to other participating nodes or other candidate nodes in the federated learning network to inform them that they have been elected as the leader node. This allows candidate nodes among the other participating nodes to know as soon as possible that a leader node has been elected, thereby improving the efficiency of node collaboration.

[0035] In one implementation, in response to receiving model parameters from a new leader node after a communication disconnection and reconnection, the node switches its role to a follower node based on the fact that the iteration number of its latest local model parameters lags behind the iteration number of the model parameters sent by the new leader node. That is, after a leader node disconnects and reconnects, since it has not received other synchronization information during the interruption, it will retrieve the latest updated model parameters from its own maintained model log according to the original process and execute the model training task; however, if it receives model parameters from a new leader node during this period, and determines that the latest updated model parameters retrieved from the model log are lagging behind the model parameters sent by the new leader node, it switches its role to a follower node.

[0036] It should be noted that when the leader node network is interrupted, the follower nodes will not receive any information and will re-elect a leader node. Any follower node may become the new leader node after the leader node network is interrupted. By designing a leader election process adapted to federated learning, a new leader node is elected in a timely manner when the leader node fails, avoiding the problem of global paralysis due to leader node failure, enhancing the robustness of the system, and improving the fault tolerance of the federated learning network.

[0037] Furthermore, after a new participating node joins the federated learning network, it performs model training using the initial model parameters until it receives the fusion parameters from the leader node, at which point it switches to a follower node role. In other words, the node collaboration method provided in this application exhibits better fault tolerance and disaster recovery capabilities when new nodes are added.

[0038] Step S12: In response to the fact that the newly accumulated model parameters in this iteration are not less than the preset number, the newly accumulated model parameters are fused to obtain fused parameters, and the fused parameters are sent to each follower node.

[0039] Because each follower node has a different running speed or network speed, their efficiency in completing the model training task varies. A follower node with a very slow running speed or network speed will drag down the overall training efficiency of the model. It should be noted that, for example... Figure 3 As shown, Figure 3 This is a schematic diagram of the log replication process provided in this application. Follower nodes send model parameters to the leader node after completing the model training task. However, since the model computing power of each follower node may be different, the completion time of the model training task of each follower node may be different. Therefore, the leader node will obtain the model parameters sent by the follower nodes in the federated learning network one after another.

[0040] Therefore, to ensure iteration efficiency and avoid poor generalization ability caused by updating step by step, this implementation method, in response to the fact that the newly accumulated model parameters in this iteration are not less than a preset number of parts, fuses are performed based on the newly accumulated model parameters to obtain fused parameters. The preset number of parts refers to the minimum number of model parameters that can be fused. When the newly accumulated model parameters in this iteration are not less than the minimum number of model parameters that can be fused, the newly accumulated model parameters are used to fuse to obtain fused parameters. That is, a semi-asynchronous federated learning scheme is adopted, and the preset number of parts mechanism is used to control the update of model parameters, effectively alleviating the problem of training blocking caused by serial training in ordinary decentralized scenarios.

[0041] It should be noted that if the leader node has not yet completed its own model training task, the newly accumulated model parameters do not include the model parameters trained by the leader node; conversely, the newly accumulated model parameters include the model parameters trained by the leader node.

[0042] In addition, such as Figure 3 As shown, the leader node will also send the fused parameters, obtained by fusing the newly accumulated model parameters, to each follower node. Each follower node, in response to the fused parameters, will stop its current training and re-execute the model training task based on the fused parameters. In other words, after the leader node completes the fusion of the model parameters, it will transmit the fused parameters to all other follower nodes. At this point, each follower node will stop its current training and re-execute the model training task based on the fused parameters.

[0043] In one implementation, the preset number of copies is greater than 1 and less than the total number of participating nodes in the federated learning network.

[0044] Step S13: Re-execute the steps of obtaining the model parameters sent by the follower nodes in the federated learning network and subsequent steps until the countdown of this iteration ends.

[0045] In this implementation, the steps of obtaining the model parameters sent by the follower nodes in the federated learning network and subsequent steps are re-executed until the countdown of the current iteration ends. In other words, a countdown is set to control the duration of one iteration; when the countdown ends, the current iteration terminates. This semi-asynchronous federated learning scheme, utilizing a countdown mechanism to control model parameter updates, effectively alleviates the training blocking problem caused by sequential training in typical decentralized scenarios. It ensures iteration efficiency while avoiding issues such as poor generalization ability caused by step-by-step updates.

[0046] Step S14: Perform fusion based on the fusion parameters obtained in this iteration to obtain the final parameters of this iteration, send the final parameters to the follower nodes, and start a new round of iteration based on the final parameters.

[0047] In this implementation, the fusion parameters from the local iteration process are fused to obtain the final parameters for this iteration. These final parameters are then sent to the follower nodes, and a new round of iteration is initiated based on these final parameters. In other words, the fusion parameters from the current iteration process are fused to obtain the final parameters for this iteration; at this point, the final parameters for this iteration are synchronously distributed to each follower node, and a new round of iteration is initiated based on these final parameters.

[0048] In one implementation, after the leader node synchronously distributes the final parameters of this iteration to each follower node, the leader node restarts its countdown to begin a new round of iteration. Additionally, each follower node restarts its own countdown after receiving the final parameters or after completing its model training task. The initial value of the leader node's countdown and the initial values ​​of the follower nodes' countdowns can be the same or different values.

[0049] In one embodiment, participating nodes maintain model logs locally for each participating node in the federated learning network. The model logs maintained for participating nodes contain model parameters at each update time. Each iteration includes at least one update time, and the model parameters at the update time are the fusion parameters after the fusion operation is performed. Before the fusion operation is performed, the model parameters at the update time in the model logs maintained for participating nodes are temporarily replaced by model parameters obtained from the participating nodes. If a participating node does not participate in the fusion operation at the update time, the model parameters at the update time in the model logs maintained for participating nodes are the model parameters of the previous update time. The model parameters of the last update time in the same iteration are all final parameters.

[0050] In one implementation, the leader node performs the fusion operation based on the model parameters from the last update in the model log maintained for each participating node.

[0051] In one specific implementation, after obtaining the model parameters sent by the follower nodes in the federated learning network, and before fusing the newly accumulated model parameters to obtain the fused parameters, the model parameters are filled with the latest update time in the model log maintained by the follower node that received the model parameters; and the model parameters of the previous update time in the model log maintained by the follower node that did not receive the model parameters are filled with the latest update time in the model log. At this point, fusing the newly accumulated model parameters to obtain the fused parameters specifically includes fusing the model parameters based on the last update time in the model log maintained by each participating node.

[0052] In one specific implementation, after fusing the newly accumulated model parameters to obtain the fusion parameters, and before fusing the fusion parameters from the current iteration to obtain the final parameters for this iteration, the latest update time from the model logs maintained by the follower nodes participating in the fusion operation is used to fill the fusion parameters; and the latest update time from the model logs maintained by the follower nodes not participating in the fusion operation is used to fill the model parameters from the previous update time in the model logs. At this point, fusing based on the fusion parameters from the current iteration to obtain the final parameters for this iteration specifically involves fusing the model parameters from the last update time in the model logs maintained by each participating node to obtain the final parameters.

[0053] For example, such as Figure 4 As shown, Figure 4 This is a logic diagram of an embodiment of node collaboration provided in this application. Taking the current iteration as the τth iteration and the federated learning network including participant node A (leader node), participant node B (follower node 1), and participant node C (follower node 2) as an example: Participant node A, participant node B, and participant node C each maintain a model log in the local federated learning network. The model log maintained by the participant node contains the model parameters at each update time. Each iteration includes at least one update time, and the model parameters at the update time are the fusion parameters after the fusion operation is performed.

[0054] First, at the very beginning of the τth iteration, ω merge,j The final parameters for the (τ-1)th iteration are shown below, and the model logs maintained by each party are as follows:

[0055] The model log maintained by the leader node (participant A node):

[0056] A from A:……,(ω merge,j ,τ-1)

[0057] A from B:……,(ω merge,j ,τ-1)

[0058] A fromC:……,(ω merge,j ,τ-1)

[0059] It should be noted that A from A represents the model log maintained by the leader node (participant A node) for itself; A from B represents the model log maintained by the leader node (participant A node) for follower node 1 (participant B node); and A from C represents the model log maintained by the leader node (participant A node) for follower node 2 (participant C node).

[0060] The queue maintained by follower node 1 (participant node B):

[0061] B from A:……,(ω merge,j ,τ-1)

[0062] B fromB:……,(ω merge,j ,τ-1)

[0063] B from C:……,(ω merge,j ,τ-1)

[0064] The model log maintained by follower node 2 (participant node C):

[0065] C from A:……,(ω merge,j ,τ-1)

[0066] C fromB:……,(ω merge,j ,τ-1)

[0067] C fromC:……,(ω merge,j ,τ-1)

[0068] Then, as Figure 4 As shown, the leader node (participant A node) completes the model training task first, at which point the leader node (participant A node) receives its own model parameters ω. A,1 At this point, follower node 1 (participant B node) and follower node 2 (participant C node) have not yet completed their model training tasks, so the leader node (participant A node) has not yet received the corresponding model parameters sent by follower node 1 (participant B node) and follower node 2 (participant C node). Therefore, for the current update timing, for the leader node (participant A node), since its own model log has been updated, the leader node (participant A node) will fill in the corresponding model parameter ω with the latest update timing in its own model log. A,1 ; and for the follower nodes that have not received the model parameters, populate the model parameters in the model log for the last update time, resulting in the queue maintained by the leader node (participant A node) as shown below:

[0069] A from A:……,(ω merge,j ,τ-1),(ω A,1 ,τ)

[0070] A from B:……,(ω merge,j ,τ-1),(ω merge,j ,τ)

[0071] A fromC:……,(ω merge,j ,τ-1),(ω merge,j ,τ)

[0072] The queue maintained by follower node 1 (participant node B) is:

[0073] B from A:……,(ω merge,j ,τ-1),(ω merge,j ,τ)

[0074] B fromB:……,(ω merge,j ,τ-1),(ω merge,j ,τ)

[0075] B from C:……,(ω merge,j ,τ-1),(ω merge,j ,τ)

[0076] The queue maintained by follower node 2 (participant node C) is as follows:

[0077] C from A:……,(ω merge,j ,τ-1),(ω merge,j ,τ)

[0078] C fromB:……,(ω merge,j ,τ-1),(ω merge,j ,τ)

[0079] C fromC:……,(ω merge,j ,τ-1),(ω merge,j ,τ)

[0080] It should be noted that follower node 1 (participant B node) and follower node 2 (participant C node) did not receive any model parameters and did not complete the model training task themselves. Therefore, they filled the model parameters with the previous update time in the model log maintained by other participating nodes with the latest update time.

[0081] Then, as time goes on, follower node 1 (participant node B) completes the model training task and obtains the corresponding model parameters ω. B,1The model parameters are sent to the leader node (participant A node). However, at this time, follower node 2 (participant C node) has not yet completed its model training task, so the leader node (participant A node) has not yet received the corresponding model parameters sent by follower node 2 (participant C node). Therefore, for the current update timing, for the leader node (participant A node), the model log maintained by the leader node (participant A node) for follower node 1 (participant B node) has been updated. The latest update timing in the model log maintained by the leader node (participant A node) for follower node 1 (participant B node) is then used to fill in the corresponding model parameters ω. B,1 ; and for the follower nodes that have not received the model parameters, populate the model parameters in the model log for the last update time, resulting in the queue maintained by the leader node (participant A node) as shown below:

[0082] A from A:……,(ω merge,j ,τ-1),(ω A,1 ,τ)

[0083] A from B:……,(ω merge,j ,τ-1),(ω B,1 ,τ)

[0084] A fromC:……,(ω merge,j ,τ-1),(ω merge,j ,τ)

[0085] Follower node 1 (participant node B) completes its own model training task and has not received any other model parameters, so it is able to obtain its own corresponding model parameters ω. B,1 Therefore, if follower node 1 (participant B node) has an updated model log, the model parameter ω will be filled with the latest update time from the model log maintained by follower node 1 (participant B node). B,1 ; and for the latest update timing in the model log maintained by other participating nodes that have not received the model parameters, populate the model parameters in the model log for the previous update timing, resulting in the queue maintained by follower node 1 (participant node B) as shown below:

[0086] B from A:……,(ω merge,j ,τ-1),(ω merge,j ,τ)

[0087] B fromB:……,(ω merge,j ,τ-1),(ω B,1 ,τ)

[0088] B from C:……,(ω merge,j,τ-1),(ω merge,j ,τ)

[0089] Follower node 2 (participant node C) has not received any model parameters and has not completed its own model training task. Therefore, it fills the model parameters with the previous update time into the model log maintained by other participating nodes, resulting in the queue maintained by follower node 2 (participant node C) as shown below:

[0090] C from A:……,(ω merge,j ,τ-1),(ω merge,j ,τ)

[0091] C fromB:……,(ω merge,j ,τ-1),(ω merge,j ,τ)

[0092] C fromC:……,(ω merge,j ,τ-1),(ω merge,j ,τ)

[0093] Then, since the newly accumulated model parameters in this iteration are 2 after receiving the model parameters sent by follower node 1 (participant B node), which is not less than the preset number of parts - 1, the leader node (participant A node) begins the first fusion. Figure 4 The aggregation operation in the middle is used to obtain the fusion parameter ω. merge,1 and the fusion parameter ω merge,1 Synchronize to follower node 1 (participant B node) and follower node 2 (participant C node). Therefore, set the leader node (participant A node) to the latest update time in its own maintained model log, and fill in the fusion parameter ω. merge,1 The fusion parameter ω is populated based on the latest update time in the model log maintained by follower node 1 (participant node B) participating in the fusion operation. merge,1 Since follower node 2 (participant node C) did not participate in this fusion operation, the model parameters from the previous update time are populated into the model log maintained by follower node 2 (participant node C) based on the latest update time. The updated model logs for each party are shown below:

[0094] The model log maintained by the leader node (participant A node):

[0095] A from A:……,(ω merge,j ,τ-1),(ω merge,1 ,τ)

[0096] A from B:……,(ω merge,j ,τ-1),(ω merge,1 ,τ)

[0097] A fromC:……,(ω merge,j ,τ-1),(ω merge,j ,τ)

[0098] The model log maintained by follower node 1 (participant node B):

[0099] B from A:……,(ω merge,j ,τ-1),(ω merge,1 ,τ)

[0100] B fromB:……,(ω merge,j ,τ-1),(ω merge,1 ,τ)

[0101] B from C:……,(ω merge,j ,τ-1),(ω merge,1 ,τ)

[0102] The model log maintained by follower node 2 (participant node C):

[0103] C from A:……,(ω merge,j ,τ-1),(ω merge,1 ,τ)

[0104] C fromB:……,(ω merge,j ,τ-1),(ω merge,1 ,τ)

[0105] C fromC:……,(ω merge,j ,τ-1),(ω merge,1 ,τ)

[0106] It should be noted that the fusion parameter ω is obtained because the first fusion operation is completed at the leader node (participant A node). merge,1 Then, the fusion parameter ω will be... merge,1 The message is sent to follower node 1 (participant B node) and follower node 2 (participant C node). Therefore, from the perspective of follower nodes, they will believe that all other participating nodes have received the fusion parameter ω sent by the leader node. merge,1 Therefore, the latest update timing of the model logs of each participating node it maintains is used to fill the fusion parameter ω. merge,1 .

[0107] like Figure 4 As shown, in the second fusion operation ( Figure 4In the aggregation step (II), the model parameters sent by follower node 1 (participant B node) and follower node 2 (participant C node) are actually fused together. Therefore, following the same model log update and saving method, after the second fusion operation, the model logs maintained by each party are as follows:

[0108] The model log maintained by the leader node (participant A node):

[0109] A from A:……,(ω merge,j ,τ-1),(ω merge,1 ,τ),(ω merge,1 ,τ)

[0110] A from B:……,(ω merge,j ,τ-1),(ω merge,1 ,τ),(ω merge,2 ,τ)

[0111] A fromC:……,(ω merge,j ,τ-1),(ω merge,j ,τ),(ω merge,2 ,τ)

[0112] The model log maintained by follower node 1 (participant node B):

[0113] B from A:……,(ω merge,j ,τ-1),(ω merge,1 ,τ),(ω merge,2 ,τ)

[0114] B fromB:……,(ω merge,j ,τ-1),(ω merge,1 ,τ),(ω merge,2 ,τ)

[0115] B from C:……,(ω merge,j ,τ-1),(ω merge,1 ,τ),(ω merge,2 ,τ)

[0116] The model log maintained by follower node 2 (participant node C):

[0117] C from A:……,(ω merge,j ,τ-1),(ω merge,1 ,τ),(ω merge,2 ,τ)

[0118] C fromB:……,(ω merge,j ,τ-1),(ω merge,1 ,τ),(ωmerge,2 ,τ)

[0119] C fromC:……,(ω merge,j ,τ-1),(ω merge,1 ,τ),(ω merge,2 ,τ)

[0120] like Figure 4 As shown, in the third fusion operation ( Figure 4 In the aggregation (iii), although only the model parameters ω obtained by the leader node (participant A node) from completing the model training task were received, A,2 However, since the countdown for the τth iteration has ended, the last model parameter from each model log in the τth iteration is retrieved and fused. Therefore, following the same model log update and saving method, after the third fusion operation, the model logs maintained by each party are as follows:

[0121] The model log maintained by the leader node (participant A node):

[0122] A from A:……,(ω merge,j ,τ-1),(ω merge,1 ,τ),(ω merge,1 ,τ),(ω merge,3 ,τ)

[0123] A from B:……,(ω merge,j ,τ-1),(ω merge,1 ,τ),(ω merge,2 ,τ),(ω merge,3 ,τ)

[0124] A fromC:……,(ω merge,j ,τ-1),(ω merge,j ,τ),(ω merge,2 ,τ),(ω merge,3 ,τ)

[0125] The model log maintained by follower node 1 (participant node B):

[0126] B from A:……,(ω merge,j ,τ-1),(ω merge,1 ,τ),(ω merge,2 ,τ),(ω merge,3 ,τ)

[0127] B fromB:……,(ω merge,j ,τ-1),(ω merge,1 ,τ),(ω merge,2 ,τ),(ω merge,3 ,τ)

[0128] B from C:……,(ω merge,j ,τ-1),(ω merge,1 ,τ),(ω merge,2 ,τ),(ω merge,3 ,τ)

[0129] The model log maintained by follower node 2 (participant node C):

[0130] C from A:……,(ω merge,j ,τ-1),(ω merge,1 ,τ),(ω merge,2 ,τ),(ω merge,3 ,τ)

[0131] C fromB:……,(ω merge,j ,τ-1),(ω merge,1 ,τ),(ω merge,2 ,τ),(ω merge,3 ,τ)

[0132] C fromC:……,(ω merge,j ,τ-1),(ω merge,1 ,τ),(ω merge,2 ,τ),(ω merge,3 ,τ)

[0133] In the above implementation, when the newly accumulated model parameters in this iteration are not less than a preset number, the newly accumulated model parameters are fused to obtain fused parameters. The steps of obtaining the model parameters sent by the follower nodes in the federated learning network and subsequent steps are then re-executed until the countdown of this iteration ends. The fused parameters from this iteration are then fused to obtain the final parameters for this iteration, which are sent to the follower nodes, and a new iteration is initiated based on the final parameters. Therefore, by simultaneously controlling the model update using both countdown time and preset scores, the iteration efficiency of the model is ensured. Furthermore, the leader node sends the final parameters to each follower node, thereby synchronizing information between the nodes.

[0134] Please see Figure 5 , Figure 5 This is a flowchart illustrating another embodiment of the node collaboration method provided in this application. It should be noted that if substantially the same result is achieved, this embodiment does not necessarily reflect that outcome. Figure 5 The illustrated process sequence is limited. For example... Figure 5 As shown, this embodiment includes:

[0135] Step S51: Send the model parameters after completing the model training task to the leader node in the federated learning network.

[0136] Steps S51, S11, and S12 are similar and will not be described again here.

[0137] Step S52: Receive the fused model sent by the leader node, stop the current training, and re-execute the model training task based on the fused parameters.

[0138] Steps S52 and S12 are similar and will not be repeated here.

[0139] Step S53: Re-execute the step of sending the model parameters after the model training task is completed to the leader node in the federated learning network, as well as subsequent steps, until the countdown of this iteration ends.

[0140] Steps S53 and S13 are similar and will not be repeated here.

[0141] Step S54: Receive the final parameters and start a new round of iteration based on the final parameters.

[0142] Steps S54 and S14 are similar and will not be repeated here.

[0143] In one implementation, a newly joined participant node switches its role to a follower node in response to receiving model parameters from the leader node. In other words, upon receiving a message from the leader node, the newly joined participant node will change its role to that of a follower node.

[0144] In one implementation, in response to the leader node's communication being interrupted, and based on the end of a countdown, a voting request is sent to participating nodes other than itself in the federated learning network. Each participating node only votes for the node that first receives the voting request. If a preset percentage of votes is received, the node switches its role to become the leader node. In other words, after the leader node's network is interrupted in the federated learning network, a new leader node election will take place when its corresponding countdown ends. If a preset percentage of votes is received, the node switches its role to become the leader node.

[0145] There is no fixed preset ratio; it can be set according to actual usage needs. For example, the preset ratio is 50%.

[0146] In one specific implementation, the countdown duration of both the leader node and the follower node is set based on computing power, and the countdown duration is negatively correlated with computing power.

[0147] Please see Figure 6 , Figure 6 This is a flowchart illustrating another embodiment of the node collaboration method provided in this application. It should be noted that if substantially the same result is achieved, this embodiment does not necessarily reflect that outcome. Figure 6 The illustrated process sequence is limited. For example... Figure 6 As shown, this embodiment includes:

[0148] Step S61: Follower nodes send the model parameters after completing the model training task to the leader node in the federated learning network.

[0149] Step S61 is similar to step S51, and will not be described again here.

[0150] Step S62: In response to the fact that the newly accumulated model parameters in this iteration are not less than a preset number, the leader node performs fusion based on the newly accumulated model parameters to obtain fusion parameters, and sends the fusion parameters to each of the follower nodes.

[0151] Step S62 is similar to step S12, and will not be described again here.

[0152] Step S63: The follower node receives the fused model sent by the leader node, stops the current training, and re-executes the model training task based on the fused parameters.

[0153] Step S63 is similar to step S52, and will not be described again here.

[0154] Step S64: Repeat the steps of the follower node sending the model parameters after completing the model training task to the leader node in the federated learning network, and subsequent steps, until the countdown of this iteration ends.

[0155] Step S64 is similar to step S13, and will not be described again here.

[0156] Step S65: The leader node performs fusion based on the fusion parameters obtained in this iteration to obtain the final parameters for this iteration, and sends the final parameters to the follower nodes.

[0157] Step S65 is similar to step S53, and will not be described again here.

[0158] Step S66: The leader node and follower nodes start a new round of iteration based on the final parameters.

[0159] Step S66 is similar to steps S54 and S14, and will not be described again here.

[0160] Please see Figure 7 , Figure 7This is a schematic diagram of an embodiment of the node collaboration device provided in this application. The node collaboration device 70 includes an acquisition module 71, a fusion module 72, a loop module 73, and a termination module 74. The acquisition module 71 is used to acquire model parameters sent by follower nodes in the federated learning network. The follower nodes send model parameters to the leader node in the federated learning network based on the completion of the model training task, and the leader node also performs the model training task. The fusion module 72 is used to fuse the newly accumulated model parameters in the current iteration to obtain fused parameters and send the fused parameters to each follower node in response to the fused parameters. Each participating node stops the current training in response to the fused parameters and re-executes the model training task based on the fused parameters. The loop module 73 is used to re-execute the step of acquiring the model parameters sent by the follower nodes in the federated learning network and subsequent steps until the countdown of the current iteration ends. The termination module 74 is used to fuse the fused parameters in the current iteration to obtain the final parameters of the current iteration and send the final parameters to the follower nodes, and start a new round of iteration based on the final parameters.

[0161] The node collaboration device 70 also includes a switching module 75. The switching module 75 is used to, before obtaining the model parameters sent by the follower nodes in the federated learning network, specifically include: in response to the first completion of the model training task after model initialization, sending a voting request to the participating nodes other than itself in the federated learning network; wherein, the participating node only votes for the node to which the first voting request is received; and based on receiving more than a preset proportion of votes, switching the role to leader node.

[0162] The switching module 75 is used to switch roles as a leader node after sending a voting request to participating nodes other than itself in the federated learning network, and before switching roles based on receiving more than a preset proportion of votes. Specifically, it includes: switching roles as a candidate node; and switching roles as a follower node in response to a leader node that has been elected in the federated learning network.

[0163] The switching module 75 is used to switch its role to a follower node in response to receiving model parameters from the new leader node after communication is disconnected and reconnected, based on the fact that the iteration round to which the latest local model parameters belong is lagging behind the iteration round to which the model parameters sent by the new leader node belong.

[0164] The aforementioned participating nodes maintain model logs locally for each participating node in the federated learning network. The model logs maintained for the participating nodes contain model parameters at each update time. Each iteration includes at least one update time, and the model parameters at the update time are the fusion parameters after the fusion operation is performed. Before the fusion operation is performed, the model parameters at the update time in the model logs maintained for the participating nodes are temporarily replaced by model parameters newly obtained from the participating nodes. If a participating node does not participate in the fusion operation at the update time, the model parameters at the update time in the model logs maintained for the participating nodes are the model parameters of the previous update time. The model parameters of the last update time in the same iteration are all final parameters.

[0165] The leader node performs the fusion operation based on the model parameters from the last update in the model log maintained for each participating node.

[0166] The node collaboration device 70 further includes a filling module 76, which is used to fill the model parameters after obtaining the model parameters sent by the follower nodes in the federated learning network and before fusing the newly accumulated model parameters to obtain the fusion parameters. Specifically, the filling module 76 fills the model parameters corresponding to the latest update time in the model log maintained by the follower node to which the received model parameters belong; and fills the model parameters of the previous update time in the model log maintained by the follower node that did not receive the latest update time. The fusion module 72 is used to fuse the newly accumulated model parameters to obtain the fusion parameters, including fusing the model parameters based on the last update time in the model log maintained by each participating node.

[0167] The filling module 76 is used to obtain fusion parameters after fusing based on newly accumulated model parameters and before fusing based on the fusion parameters in the current iteration to obtain the final parameters of the current iteration. Specifically, it includes: filling the fusion parameters with the latest update timing in the model log maintained by the follower nodes participating in the fusion operation; and filling the model parameters of the previous update timing in the model log maintained by the follower nodes not participating in the fusion operation with the latest update timing in the model log. The abort module 74 is used to obtain the final parameters of the current iteration based on the fusion parameters in the current iteration. Specifically, it includes: fusing based on the model parameters of the last update timing in the model log maintained by each participating node to obtain the final parameters.

[0168] The preset number of copies is greater than 1 and less than the total number of participating nodes in the federated learning network.

[0169] Please see Figure 8 , Figure 8This is a schematic diagram of another embodiment of the node collaboration device provided in this application. The node collaboration device 80 includes a sending module 81, a stop module 82, a loop module 83, and a restart module 84. The sending module 81 is used to send the model parameters after the model training task is completed to the leader node in the federated learning network. The leader node also performs the model training task, and in response to the fact that the newly accumulated model parameters in this iteration are not less than a preset number, the leader node fuses the newly accumulated model parameters to obtain fused parameters. The stop module 82 is used to receive the fused model sent by the leader node, stop the current training, and re-execute the model training task based on the fused parameters. The loop module 83 is used to re-execute the step of sending the model parameters after the model training task is completed to the leader node in the federated learning network and subsequent steps until the countdown of this iteration ends. When the countdown of this iteration ends, the leader node obtains the final parameters of this iteration based on the fused parameters in this iteration process. The restart module 84 is used to receive the final parameters and start a new round of iteration based on the final parameters.

[0170] The node collaboration device 80 also includes a switching module 85, which is used to switch the role of a newly joined participating node to a follower node in response to receiving model parameters sent by the leader node.

[0171] The switching module 85 is used to respond to the disconnection of the leader node communication and send a voting request to the participating nodes other than itself in the federated learning network based on the end of the countdown; the participating nodes only vote for the node to which the first voting request is received; and the role is switched to the leader node based on receiving more than a preset proportion of votes.

[0172] The countdown durations for both the leader and follower nodes are set based on computing power, and the countdown durations are negatively correlated with computing power.

[0173] Please see Figure 9 , Figure 9 This is a schematic diagram of a node device according to an embodiment of the present application. The node device 90 includes a memory 91 and a processor 92. The memory 91 stores program instructions, and the processor 92 executes the program instructions to implement the steps of any of the above-described node collaboration method embodiments. In a specific implementation scenario, the node device 90 may include, but is not limited to, a microcomputer or a server. In addition, the node device 90 may also include mobile devices such as laptops and tablets, which are not limited here.

[0174] Specifically, processor 92 controls itself and memory 91 to implement the steps of any of the above-described node collaboration method embodiments. Processor 92 can also be referred to as a CPU (Central Processing Unit). Processor 92 may be an integrated circuit chip with signal processing capabilities. Processor 92 can also be a general-purpose processor, digital signal processor (DSP), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. A general-purpose processor can be a microprocessor or any conventional processor. Furthermore, processor 92 can be implemented using integrated circuit chips.

[0175] This application also provides a node collaboration network, which includes several participating nodes with communication connections. The participating nodes are the node devices described above. In the node collaboration network, the participating nodes elect a leader node and at least one follower node.

[0176] Please see Figure 10 , Figure 10 This is a schematic diagram of an embodiment of the computer-readable storage medium provided in this application. The computer-readable storage medium 100 of this application embodiment stores program instructions 101. When executed, these program instructions 101 implement the methods provided by any embodiment of the node coordination method and any non-conflicting combination thereof. The program instructions 101 can form a program file and be stored in the aforementioned computer-readable storage medium 100 in the form of a software product, so that a computer device (which may be a personal computer, server, or network device, etc.) can execute all or part of the steps of the methods of various embodiments of this application. The aforementioned computer-readable storage medium 100 includes various media capable of storing program code, such as a USB flash drive, mobile hard drive, read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk, or terminal devices such as computers, servers, mobile phones, and tablets.

[0177] If the technical solution of this application involves personal information, the product using this technical solution has clearly informed the user of the personal information processing rules and obtained the user's voluntary consent before processing the personal information. If the technical solution of this application involves sensitive personal information, the product using this technical solution has obtained the user's separate consent before processing the sensitive personal information, and also meets the requirement of "express consent". For example, at personal information collection devices such as cameras, clear and prominent signs are set up to inform users that they have entered the scope of personal information collection and that personal information will be collected. If an individual voluntarily enters the collection scope, it is deemed that they have agreed to the collection of their personal information; or on the personal information processing device, with clear signs / information informing users of the personal information processing rules, authorization is obtained from the individual through pop-up information or by asking the individual to upload their personal information; wherein, the personal information processing rules may include information such as the personal information processor, the purpose of personal information processing, the processing method, and the types of personal information processed.

Claims

1. A method of node collaboration, the method comprising: include: Obtain model parameters sent by follower nodes in a federated learning network; wherein the follower nodes send the model parameters to the leader node in the federated learning network based on completing the model training task, and the leader node also performs the model training task; In response to the fact that the newly accumulated model parameters in this iteration are not less than a preset number, the newly accumulated model parameters are fused to obtain fused parameters, and the fused parameters are sent to each of the follower nodes; wherein, each participating node responds to the fused parameters, stops the current training, and re-runs the model training task based on the fused parameters; Repeat the steps of obtaining the model parameters sent by the follower nodes in the federated learning network and subsequent steps until the countdown of this iteration ends; The final parameters for this iteration are obtained by fusing the parameters obtained during the current iteration and sent to the follower nodes. A new iteration is then initiated based on these final parameters. Each participating node maintains a model log locally for each participating node in the federated learning network. The model log contains model parameters at various update times. Each iteration includes at least one update time, and the model parameters at each update time are the fusion parameters after the fusion operation. The model log maintained by the participating nodes before the fusion operation is as follows: The model parameters for the update timing are temporarily replaced by the model parameters newly obtained from the participating nodes. If the participating node does not participate in the fusion operation at the update timing, the model parameters for the update timing in the model log maintained for the participating node are the model parameters of the previous update timing. When the countdown of this iteration ends, the leader node performs the fusion operation based on the model parameters of the last update timing in the model log maintained for each participating node to obtain the final parameters of this iteration. The model parameters of the last update timing in the same iteration round are all the final parameters.

2. The method of claim 1, wherein, Before obtaining the model parameters sent by the follower nodes in the federated learning network, the method further includes: In response to the first completion of the model training task after model initialization, a voting request is sent to participating nodes other than itself in the federated learning network; wherein, the participating node only votes for the node to which the first voting request is received. If a node receives more than a predetermined percentage of votes, it switches to the role of the leader node.

3. The method of claim 2, wherein, After sending a voting request to participating nodes outside itself in the federated learning network, and before switching roles as the leader node based on receiving a preset proportion of votes, the method includes: Switch the role to a candidate node; In response to the leader node being elected in the federated learning network, the node switches its role to that of a follower node.

4. The method of claim 1, wherein, The method further includes: In response to receiving model parameters from a new leader node after communication is disconnected and reconnected, the system switches its role to the follower node based on the fact that the iteration round to which the latest local model parameters belong lags behind the iteration round to which the model parameters sent by the new leader node belong.

5. The method of claim 1, wherein, After obtaining the model parameters sent by the follower nodes in the federated learning network, and before fusing the newly accumulated model parameters to obtain the fused parameters, the process includes: The model parameters will be populated with the latest update timing from the model log maintained by the follower node to which the received model parameters belong; and... The model parameters of the previous update time will be filled into the model log for the follower node to which the model parameters belong, for the latest update time in the model log that has not been received; The fusion based on the newly accumulated model parameters yields fused parameters, including: The fusion parameters are obtained by fusing the model parameters of the last update time in the model log maintained for each of the participating nodes.

6. The method of claim 1, wherein, After fusing the newly accumulated model parameters to obtain fused parameters, and before fusing the fused parameters from the current iteration to obtain the final parameters for the current iteration, the method further includes: The fusion parameters will be populated with the latest update timing from the model logs maintained by the follower nodes participating in the fusion operation; and... The model parameters of the previous update time in the model log will be filled into the latest update time in the model log maintained for follower nodes that did not participate in the fusion operation. The process of fusing parameters based on the current iteration to obtain the final parameters for this iteration includes: The final parameters are obtained by fusing the model parameters from the last update time in the model log maintained for each of the participating nodes.

7. The method according to any one of claims 1 to 6, characterized in that, The preset number of copies is greater than 1 and less than the total number of participating nodes in the federated learning network.

8. A method of node collaboration, the method comprising: include: The model parameters after the model training task is completed are sent to the leader node in the federated learning network; wherein, the leader node also performs the model training task, and in response to the fact that the newly accumulated model parameters in this iteration are not less than a preset number, the leader node fuses the newly accumulated model parameters to obtain fused parameters. Receive the fused model sent by the leader node, stop the current training, and re-execute the model training task based on the fused parameters; The steps of sending the model parameters after the model training task is completed to the leader node in the federated learning network and subsequent steps are repeated until the countdown of the current iteration ends; wherein, when the countdown of the current iteration ends, the leader node obtains the final parameters of the current iteration based on the fusion parameters in the current iteration process; The final parameters are received, and a new round of iteration is initiated based on the final parameters. Each participating node maintains a model log locally for each participating node in the federated learning network. The model logs maintained for each participating node contain model parameters at each update timing. Each iteration includes at least one update timing. The model parameters at each update timing are the fusion parameters after the fusion operation. Before the fusion operation is performed, the model parameters at each update timing in the model logs maintained for each participating node are temporarily replaced by newly acquired model parameters from the participating node. If a participating node does not participate in the fusion operation at an update timing, the model parameters at that update timing in the model logs maintained for each participating node are the model parameters from the previous update timing. When the countdown for this iteration ends, the leader node performs the fusion operation based on the model parameters of the last update timing in the model logs maintained for each participating node, obtaining the final parameters for this iteration. The model parameters of the last update timing in the same iteration are all the final parameters.

9. The method of claim 8, wherein, The method further includes: In response to a newly joined participant node in the federated learning network receiving model parameters sent by the leader node, it switches its role to that of a follower node.

10. The method of claim 8, wherein, The method further includes: In response to the leader node's communication being disconnected, and based on the end of the countdown, a voting request is sent to participating nodes other than itself in the federated learning network; wherein, the participating node only votes for the node to which the first voting request is received. If a node receives more than a predetermined percentage of votes, it switches to the role of the leader node.

11. The method according to claim 8 or 10, characterized in that, The countdown durations for both the leader node and the follower nodes are set based on computing power, and the countdown durations are negatively correlated with the computing power.

12. A method of node collaboration, the method comprising: include: Follower nodes send the model parameters after completing the model training task to the leader node in the federated learning network; wherein, the leader node also performs the model training task. When the leader node responds to the fact that the newly accumulated model parameters in this iteration are not less than a preset number, it performs fusion based on the newly accumulated model parameters to obtain fused parameters, and sends the fused parameters to each of the follower nodes; The follower node receives the fused model sent by the leader node, stops the current training, and re-executes the model training task based on the fused parameters; Repeat the steps of sending the model parameters after the completion of the model training task from the follower node to the leader node in the federated learning network, and subsequent steps, until the countdown of this iteration ends; The leader node performs fusion based on the fusion parameters obtained in the current iteration to obtain the final parameters for the current iteration, and sends the final parameters to the follower node; The leader node and follower nodes initiate a new iteration based on the final parameters. Each participating node maintains a model log locally for each participating node in the federated learning network. This model log contains model parameters at each update timing. Each iteration includes at least one update timing. The model parameters for each update timing are the fusion parameters after a fusion operation. Before the fusion operation, the model parameters for each update timing in the model log maintained by the participating node are temporarily replaced by newly acquired model parameters. If a participating node does not participate in the fusion operation at an update timing, the model parameters for that update timing in the model log maintained by the participating node are the model parameters from the previous update timing. At the end of the countdown for this iteration, the leader node performs the fusion operation based on the model parameters of the last update timing in the model log maintained by each participating node, obtaining the final parameters for this iteration. The model parameters of the last update timing in the same iteration are all the final parameters.

13. A node coordination apparatus, comprising: include: An acquisition module is used to acquire model parameters sent by follower nodes in a federated learning network; wherein the follower nodes send the model parameters to the leader node in the federated learning network based on completing the model training task, and the leader node also performs the model training task; The fusion module is used to respond to the fact that the newly accumulated model parameters in this iteration are not less than a preset number, to fuse the newly accumulated model parameters to obtain fusion parameters, and to send the fusion parameters to each of the follower nodes; wherein, each participating node responds to the fusion parameters, stops the current training, and re-executes the model training task based on the fusion parameters; The loop module is used to re-execute the step of obtaining the model parameters sent by the follower nodes in the federated learning network and subsequent steps until the countdown of the current iteration ends; The termination module is used to perform fusion based on the fusion parameters in the current iteration process to obtain the final parameters of the current iteration, and send the final parameters to the follower nodes, as well as to start a new round of iteration based on the final parameters; wherein, each participating node maintains a model log locally for each participating node in the federated learning network, and the model log maintained for the participating node contains the model parameters at each update time, each iteration round includes at least one update time, and the model parameters at the update time are the fusion parameters after the fusion operation is performed, and the model log maintained for the participating node before the fusion operation is performed. The model parameters for the update timing in the model log are temporarily replaced by the model parameters newly obtained from the participating node. If the participating node does not participate in the fusion operation at the update timing, the model parameters for the update timing in the model log maintained for the participating node are the model parameters of the previous update timing. When the countdown of this iteration ends, the leader node performs the fusion operation based on the model parameters of the last update timing in the model log maintained for each participating node to obtain the final parameters of this iteration. The model parameters of the last update timing in the same iteration round are all the final parameters.

14. A node coordination apparatus, comprising: include: The sending module is used to send the model parameters after the model training task is completed to the leader node in the federated learning network; wherein, the leader node also performs the model training task, and the leader node responds to the fact that the newly accumulated model parameters in this iteration are not less than a preset number, and performs fusion based on the newly accumulated model parameters to obtain fused parameters; The abort module is used to receive the fused model sent by the leader node, stop the current training, and re-execute the model training task based on the fused parameters; The loop module is used to re-execute the step of sending the model parameters after the model training task is completed to the leader node in the federated learning network and subsequent steps until the countdown of the current iteration ends; wherein, when the countdown of the current iteration ends, the leader node obtains the final parameters of the current iteration based on the fusion parameters in the current iteration process; A restart module is used to receive the final parameters and start a new round of iteration based on the final parameters. Each participating node maintains a model log locally for each participating node in the federated learning network. The model logs maintained for each participating node contain model parameters at each update time. Each iteration includes at least one update time. The model parameters at each update time are the fusion parameters after the fusion operation. Before the fusion operation is performed, the model parameters at each update time in the model logs maintained for each participating node are temporarily replaced by newly obtained model parameters from the participating node. If a participating node does not participate in the fusion operation at an update time, the model parameters at that update time in the model logs maintained for each participating node are the model parameters from the previous update time. When the countdown for this iteration ends, the leader node performs the fusion operation based on the model parameters of the last update time in the model logs maintained for each participating node to obtain the final parameters for this iteration. The model parameters of the last update time in the same iteration are all the final parameters.

15. A node device, characterized by It includes a memory and a processor, wherein the memory stores program instructions and the processor executes the program instructions to implement the node collaboration method according to any one of claims 1 to 11.

16. A network of nodes, characterized in that The network includes several participating nodes with communication connections, wherein the participating nodes are the node devices as described in claim 15, and the participating nodes in the node collaboration network elect a leader node and at least one follower node.

17. A computer-readable storage medium, characterized in that, The system stores program instructions that can be executed by a processor, the program instructions being used to implement the node collaboration method according to any one of claims 1 to 11.

Images