Semi-centralized energy-saving federated learning method and system

Abstract

The invention provides a semi-centralized energy-saving federated learning method and system, which are applied to a federated learning system and system composed of a plurality of nodes, each node is configured with a machine learning model, and the method comprises the following steps: after the nodes are grouped, training the model of each node through a self-supervised online transfer learning algorithm, generating a local training model and a clustering center, extracting data features by using a pre-trained convolutional neural network, and generating a pseudo tag through clustering after dimension reduction; after the main node and the standby node are selected from the group, the main node updates an intra-group model and a clustering center, and the standby node takes over a task of the main node according to a preset condition; the main node exchanges the aggregation model and the clustering center with other groups through bidirectional annular communication, and carries out global updating; finally, the global aggregation model and the clustering center are synchronized to each node, the communication overhead and the use of computing resources are optimized, and the method adapts to an unmanned cluster environment.

Description

Technical Field

[0001] The present application relates to the field of machine learning technology, and in particular to a semi-centralized energy-saving federated learning method and system. Background Art

[0002] There are abundant but scattered data and computing resources distributed in unmanned clusters. In order to make full use of the resources of unmanned clusters and improve the performance of distributed artificial intelligence applications, researchers have studied various distributed machine learning schemes deployed on these clusters, among which Federated Learning (FL) has attracted much attention as a potential paradigm. Existing FL methods usually involve sending local model updates of participating nodes to a central server, which aggregates and updates the global model, or participates in peer-to-peer communication to update the global model.

[0003] However, individual nodes in unmanned swarms have limited resources, are sensitive to communication overhead, and are vulnerable to it. Existing FL methods can lead to a surge in communication overhead and unacceptable network latency. In addition, unmanned swarms collect a large amount of unlabeled data during flight. To address these issues, researchers have proposed a variety of methods, including improving existing FL frameworks and improving the performance of self-supervised learning algorithms.

[0004] To address the challenges of deploying FL in large-scale dynamic edge node networks, existing work mainly focuses on improving the FL architecture, reducing the communication process overhead in FL, or proposing more effective worker selection strategies. Multi-parameter server FL effectively utilizes more nodes to perform FL tasks in parallel, thereby improving efficiency. Research on minimizing communication overhead tends to be integrated with existing FL frameworks and applied to scaling edge nodes for large databases. However, these methods often have difficulty handling single node failures, slow convergence, or large communication overhead when deployed on highly dynamic large-scale unmanned clusters.

[0005] In terms of improving the performance and reducing the overhead of self-supervised learning, researchers have proposed techniques such as adversarial training, generative models, and domain adaptation. Faced with the challenge of balancing computational overhead and performance, researchers have continuously worked to design innovative methods, including the optimization of model structure and the integration of distributed training methods. However, existing methods either generate too much computational overhead, making it difficult to complete the local training process on a single node, or fail to achieve practical performance. Summary of the invention

[0006] The purpose of this application is to solve at least one of the above-mentioned technical deficiencies, especially the technical deficiencies in the prior art that either excessive computing overhead is generated, making it difficult to complete the local training process on a single node, or the performance cannot reach a practical level.

[0007] In a first aspect, the present application provides a semi-centralized energy-saving federated learning method, which is applied to a semi-centralized energy-saving federated learning system. The system includes multiple nodes, each node is configured with a machine learning model for federated learning, and the method includes:

[0008] After dividing each node into multiple groups, the machine learning model of each node is trained for at least one round using a self-supervised online transfer learning algorithm to obtain a local training model and local cluster center for each node. The self-supervised online transfer learning algorithm is used to extract data features through a pre-trained convolutional neural network model and generate pseudo labels through clustering using the reduced dimensionality data features;

[0009] In each group, after the node selection algorithm is applied to select the master node and the backup node, the master node of the group performs intra-group update aggregation on each local training model and each local clustering center in the group to obtain the intra-group aggregation model and intra-group clustering center of the group. The backup node is used to take over the master node under the preset takeover conditions.

[0010] The master nodes of each two groups exchange their own intra-group aggregation models and intra-group clustering centers through bidirectional ring communication;

[0011] In each group, the master node of the group performs global update aggregation on each received intra-group aggregation model and each intra-group clustering center to obtain a global aggregation model and a global clustering center, and synchronizes the global aggregation model and the global aggregation center to each node in the group.

[0012] In one embodiment, the step of dividing each node into a plurality of groups includes:

[0013] Each node updates its own routing table regularly;

[0014] When it is detected that the position of at least one node or the number of each node has changed, for each node, if the number of neighbor nodes in the routing table of the node is greater than the preset number, the node is used as a temporary group center, and a group invitation is broadcast to each neighbor node in the routing table of the node;

[0015] For a node that receives multiple group invitations, an invitation confirmation message is sent to the node that received the group invitation earliest;

[0016] For each node of the temporary group center, after receiving the invitation confirmation message, the group list is broadcast to each node to form multiple groups.

[0017] In one embodiment, the step of training the machine learning model of each node using a self-supervised online transfer learning algorithm for at least one round to obtain a local training model and a local cluster center of each node includes:

[0018] In each training round, for each node, the unlabeled data of the node is input into the pre-trained convolutional neural network model to obtain data features. After the data features are reduced in dimension, the clustering results obtained by clustering are used as pseudo labels, and the pseudo labels are used to update the machine learning model parameters of the node. If the current training round meets the preset training end conditions, the updated machine learning model of the current training round is used as the local training model, and the clustering results of the current training round are used as the local clustering center.

[0019] In one embodiment, the step of using the clustering result obtained by clustering as a pseudo label and using the pseudo label to update the machine learning model parameter of the node includes:

[0020] The distributed k-means clustering algorithm is used to cluster the data features after dimensionality reduction, and the clustering results are used as pseudo labels. The pseudo labels are used to classify the unlabeled data through the machine learning model of the node to obtain the classification results and the model loss function value. The machine learning model parameters of the node are updated according to the classification results and the model loss function value.

[0021] In one embodiment, in each group, the step of applying a node selection algorithm to select a master node and a backup node includes:

[0022] In each group, each node calculates its participation index and then broadcasts it. Each node calculates the average participation index of the current round based on the received participation indexes. After determining the nodes participating in the competition based on the average participation index of the current round and the pre-adjusted difficulty of the puzzle, the nodes participating in the competition broadcast a completion notification to each node after successfully completing the proof-of-work algorithm. Each node determines the master node and backup node among the nodes participating in the competition based on the received completion notification.

[0023] In one embodiment, the step of each node calculating its participation index includes:

[0024] Each node calculates its participation index according to the following formula:

[0025]

[0026]

[0027] in, is the node participation indicator, and is the percentage of remaining energy and idle computing power; and It indicates the safety threshold of energy and computing power usage set by the administrator in advance. Lower than When the unmanned node stops the mission and automatically returns or lands on the spot; Indicates the number of nodes in the local routing table, represents the number of all nodes in the unmanned network, It represents the ratio of the number of nodes in the local routing table to the number of all nodes in the unmanned network.

[0028] In one embodiment, the step of each node determining a master node and a backup node among the nodes participating in the competition according to the received completion notification includes:

[0029] After receiving the first completion notification and the second completion notification, each node takes the node corresponding to the first completion notification as the first candidate node and the node corresponding to the second completion notification as the second candidate node, and broadcasts them;

[0030] The number of each first candidate node and the number of each second candidate node are counted, and the first candidate node with the largest number is used as the main node, and the second candidate node with the largest number is used as the backup node.

[0031] In one embodiment, the process of pre-adjusting the difficulty of the current round of puzzles includes:

[0032] Adjust the difficulty of the current round's puzzles based on the number of completion notifications received by the master node in the previous round and the average participation index.

[0033] In one embodiment, the step of adjusting the difficulty of the current round of puzzles according to the number of completion notifications received by the master node in the previous round and the average participation index includes:

[0034] The difficulty of the current round of puzzles is calculated using the following formula:

[0035]

[0036] in, and Respectively represent the difficulty of the current round of problems and the difficulty of the previous round of problems, and Indicates the number of nodes participating in the last round and the number of nodes in the last two rounds, that is, the number of completion notifications in the last round and the number of completion notifications in the last two rounds. and represents the average engagement index of the last round and the average engagement index of the last two rounds, and is a preset adjustment factor.

[0037] In a second aspect, the present application provides a semi-centralized energy-saving federated learning system, the system comprising multiple nodes, each node being configured with a machine learning model for federated learning, and the system being used to execute a semi-centralized energy-saving federated learning method such as any of the above embodiments.

[0038] It can be seen from the above technical solutions that the embodiments of the present application have the following advantages:

[0039] The semi-centralized energy-saving federated learning method provided by the present application can effectively cope with the challenges of communication overhead and computing resource limitations faced by existing methods in unmanned clusters. First, by dividing the nodes into multiple groups and applying the self-supervised online transfer learning algorithm for training in each group, the amount of calculation required for global model update is reduced, and by generating pseudo labels and updating the local cluster center, each node can make full use of local unlabeled data for online training, thereby reducing communication overhead. At the same time, the mechanism of master node and backup node is adopted, and the master node is responsible for parameter aggregation within the group, ensuring efficient collaboration between nodes, and the backup node takes over the function of the master node under the preset takeover conditions, further improving stability. In addition, the bidirectional ring communication mechanism is adopted to avoid nodes not on the critical forwarding path from processing additional information, further optimizing the communication efficiency between nodes. Finally, through the group aggregation and global aggregation strategy of the master node, the global update of the model can be quickly realized, while ensuring the collaborative work between each node, effectively improving the performance of federated learning in the entire unmanned cluster. This method not only improves the utilization of resources, but also ensures fast convergence and low latency in a dynamically changing network environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0041] Figure 1 A flow chart of a semi-centralized energy-saving federated learning method provided in an embodiment of the present application;

[0042] Figure 2 An example diagram of the overall structure of a semi-centralized energy-saving federated learning system provided in an embodiment of the present application;

[0043] Figure 3 An example diagram of the Fed-deepcluster machine learning algorithm provided in the embodiment of the present application;

[0044] Figure 4An example diagram of the master node competition process provided in an embodiment of the present application. DETAILED DESCRIPTION

[0045] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0046] The present application provides a semi-centralized energy-saving federated learning method, which is applied to a semi-centralized energy-saving federated learning system. The system includes multiple nodes, and each node is configured with a machine learning model for federated learning. Specifically, the semi-centralized energy-saving federated learning system is suitable for large-scale dynamic unmanned clusters, which are composed of multiple nodes, and the nodes are unmanned equipment or drones. Each node has independent motion capabilities, communication equipment, sensor units, and airborne computing units. The nodes collaborate through wireless communication, and the transmitted content includes model updates and some necessary auxiliary data. When performing a task, the node collects environmental data along the way through its sensor unit, and uses the airborne computing unit to process and train the machine learning model. The machine learning model can be any multi-classification task model suitable for a federated learning training environment. For example Figure 1 As shown, the method may include the following steps:

[0047] S101: After dividing each node into multiple groups, the machine learning model of each node is trained for at least one round using a self-supervised online transfer learning algorithm to obtain a local training model and a local clustering center for each node. The self-supervised online transfer learning algorithm is used to extract data features through a pre-trained convolutional neural network model, and generate pseudo labels through clustering using the reduced dimensionality data features.

[0048] Among them, the self-supervised online transfer learning algorithm is an algorithm based on transfer learning. It uses a large amount of unlabeled data for training in a self-supervised manner to adapt to new tasks or data distributions. The algorithm extracts features through a pre-trained model and uses unlabeled data for online learning. Convolutional neural network is a deep learning model. Data features refer to the available information contained in the data, which can help machine learning models better predict or classify. Dimensionality reduction is to reduce the dimension of data through technical means, retain the most important part of the data, reduce computational complexity and improve processing efficiency. Clustering is to divide the data points in the data set into different groups or clusters so that the data points in the same cluster are similar. Pseudo-labels refer to labels generated by unsupervised learning methods or pre-trained models, which are used to label unlabeled data and improve the effect of self-supervised learning. The local training model is a machine learning model obtained by each node trained locally based on the collected data. The local cluster center is the data center point obtained after each node is clustered based on its local data, representing the main features of the data.

[0049] In this step, in a large-scale dynamic unmanned cluster, the process of node grouping can be carried out through a preset allocation strategy, such as allocation based on spatial location or allocation based on node performance. In practical applications, for example, when multiple drones participate in environmental monitoring tasks, they can be divided into different groups according to the mission area, sensor data type, and processing capability of each drone. Each group will have common goals and tasks, and have stronger communication and computing collaboration, which reduces cross-group communication overhead and improves the overall efficiency of the system.

[0050] The nodes collect data about the surrounding environment through sensors and use local computing units for preliminary processing. The machine learning model of each node is trained using a self-supervised online transfer learning algorithm based on the collected unlabeled data. First, the node uses a pre-trained convolutional neural network model to extract features from the data. These features can be environmental information collected from sensors, such as meteorological data, terrain information, or patterns of other sensor data. Next, the node performs dimensionality reduction on the extracted features to reduce the dimensionality of the data, making subsequent processing more efficient while retaining information useful for the task. After feature dimensionality reduction, the node uses a clustering algorithm to group the reduced data and generate pseudo labels. These pseudo labels assign a category to each data point, allowing the node to complete training without manually annotated data. In this way, the node can make full use of local data for training without relying on external annotated data, thereby improving the model's adaptability and computational efficiency.

[0051] In addition, during this process, the nodes will calculate local cluster centers based on the clustering results. These cluster centers represent the feature distribution learned by each node on a specific data set. Through these local training models and cluster centers, the nodes can dynamically adjust their models to adapt to changes in the surrounding environment, further improving the adaptability of the nodes to different tasks. Finally, the nodes use the training results of the self-supervised online transfer learning algorithm to obtain not only the local training model but also the local cluster centers, thus laying a solid foundation for the subsequent federated learning process and ensuring the efficient operation of the system in an unmanned cluster environment.

[0052] In one example, suppose a drone node is performing an environmental monitoring mission. The sensor data it collects may include a variety of information such as temperature, humidity, wind speed, etc., and these data usually lack clear labels. Through the self-supervised online transfer learning algorithm, the drone can extract data features through pre-trained CNN, generate pseudo labels using clustering algorithms, and then train locally to optimize its environmental perception capabilities. This method enables the unmanned cluster to autonomously learn and improve its model without external labels, which is particularly suitable for application in large-scale dynamic environments.

[0053] It can be understood that dividing nodes into multiple groups and using self-supervised online transfer learning algorithms for training can effectively reduce the dependence of each node on the central server and improve efficiency. By extracting data features locally and generating pseudo labels, nodes can complete training without a large amount of labeled data. This method can achieve adaptive learning in large-scale unmanned clusters, reduce the need for manually labeled data, and effectively address the resource constraints of each node in the unmanned cluster. The application of dimensionality reduction and clustering can reduce computing overhead and increase data processing speed, especially for nodes with limited resources.

[0054] S102: In each group, after applying the node selection algorithm to select the master node and the backup node, the master node of the group performs intra-group update aggregation on each local training model and each local clustering center in the group to obtain the intra-group aggregation model and intra-group clustering center of the group. The backup node is used to take over the master node under the preset takeover conditions.

[0055] Among them, the node selection algorithm is used to select the master node and the backup node among each node. The master node is the node selected in a group, responsible for coordinating and executing the update and aggregation of the model and cluster center within the group. The backup node, as the backup node of the master node, takes over the responsibilities of the master node under the preset takeover conditions. The update aggregation within the group is that the master node merges the local models and cluster centers of all nodes in the group to form a new model and cluster center within the group. The aggregated model within the group is the model after aggregation, which is a comprehensive reflection of the local training results of all nodes in the group. The cluster center within the group is the merged result of the local cluster centers of all nodes, representing the main features of the data within the group.

[0056] In this step, in each group, the node selection algorithm is first used to select the master node and the backup node. The selection of the master node can be based on multiple factors, such as the node's computing resources, historical performance in training, data volume, etc. The backup node serves as a backup for the master node. It becomes the master node under the preset takeover conditions, and the backup node maintains the same training tasks and aggregation tasks as the master node. When the master node fails, it can take over seamlessly to avoid task interruption. For example, when the master node fails or cannot continue to work, or the task execution time times out, it can quickly take over its responsibilities to ensure the stability and fault tolerance of the system.

[0057] Once the master node is determined, it will be responsible for updating and aggregating the local training models and local cluster centers of all nodes in the group. The master node will merge the local model and cluster center of each node through certain aggregation rules, such as average value, weighted average, etc., to form the group's intra-group aggregation model and intra-group cluster center. For example, if there are 5 nodes in the group, the master node will collect the local models and cluster centers of these 5 nodes and aggregate them according to the preset algorithm. The aggregated intra-group aggregation model and intra-group cluster center will represent the overall learning results of the group.

[0058] It can be understood that by adopting the selection mechanism of the master node and the backup node, stability and fault tolerance can be effectively improved. When the master node fails to work due to failure or other reasons, the backup node can take over the task in time to ensure the continuity of the system. In addition, the master node's intra-group aggregation of local training models and cluster centers helps to integrate the learning results of nodes in the group, avoiding resource waste and model inconsistency that may be caused by independent training of each node. The model after intra-group aggregation can not only improve the training effect of each node, but also enhance the globality and consistency of the system. In an unmanned cluster environment with multiple nodes in parallel, the cooperation of the master node and the backup node and the efficient intra-group aggregation strategy greatly improve the robustness, training efficiency and computing resource utilization of the system, ensuring that federated learning can proceed smoothly and converge quickly in a dynamic and complex network environment.

[0059] S103: The master nodes of each two groups exchange their own intra-group aggregation models and intra-group clustering centers through bidirectional ring communication.

[0060] Bidirectional ring communication refers to two or more nodes communicating with each other through a ring network structure, and each node exchanges information with its adjacent node in a bidirectional manner.

[0061] In this step, in the bidirectional ring communication between every two groups, first, the master node of each group will send the group's intra-group aggregation model and intra-group cluster center to the master nodes of other groups. Through the bidirectional ring method, information can flow along the ring structure. Each master node not only sends its own aggregation model and cluster center, but also receives similar information from other groups. This process can be carried out through wireless communication or wired network, depending on the specific application scenario. The time consumption during the bidirectional ring communication process can be expressed by the following formula:

[0062] ,in

[0063] in the formula is the time consumption of the communication process, r is the number of communications, is the maximum value of r, is the delay caused by node s communication, and N is the total number of current master nodes.

[0064] In an example, assuming that there is a two-way ring communication between Group A and Group B, the master node of Group A will send its intra-group aggregation model and intra-group clustering center to the master node of Group B, and the master node of Group B will also send its intra-group aggregation model and intra-group clustering center to the master node of Group A.

[0065] It can be understood that the execution of two-way ring communication can ensure the flow of information between multiple groups, avoid the phenomenon of information islands, and improve the coordination and consistency of the system. In addition, this communication mechanism reduces the communication load. Compared with the traditional full-network broadcast method, two-way ring communication avoids the transmission of redundant information and reduces the consumption of network bandwidth through local and directional information exchange. In a large-scale dynamic unmanned cluster environment, this efficient information exchange method significantly improves data sharing efficiency, reduces latency, speeds up model convergence, and ensures the smooth progress of federated learning in complex environments.

[0066] S104: In each group, the master node of the group performs global update aggregation on each received intra-group aggregation model and each received intra-group cluster center to obtain a global aggregation model and a global cluster center, and synchronizes the global aggregation model and the global aggregation center to each node in the group.

[0067] Among them, the global aggregation model is the final model obtained by aggregating the aggregation models of multiple groups and performing global updates, which represents the learning results of nodes in the global scope. The global cluster center is the final cluster center obtained by global updating and fusion of the cluster centers of multiple groups, which represents the aggregation point of global data features.

[0068] In this step, in each group, the master node will process the information after receiving the intra-group aggregation model and intra-group cluster center from other groups or exceeding the predetermined time. First, the master node will globally update and aggregate each intra-group aggregation model received, which can be weighted averaging, selective fusion or other aggregation strategies for the received multiple models to ensure that the aggregated global model can reflect the learning results of multiple groups. Similarly, the master node will also globally update the received intra-group cluster center, usually by merging the cluster centers of different groups in some way to obtain a new global cluster center. After the aggregation is completed, the master node will synchronize the obtained global aggregation model and global cluster center to all nodes in the group. At this time, all nodes in the group will update their local models and cluster centers to ensure that they are consistent with the global model and the global cluster center, thereby promoting the synchronization and consistency of the overall learning process.

[0069] In one example, after the master node of group A receives the intra-group aggregation models of groups B and C, it first fuses these models to obtain a global aggregation model. Then, in the same way, the master node updates the cluster center and fuses the cluster results from groups B and C. Finally, the master node synchronizes the updated global model and global cluster center to each node in group A to ensure that each node can obtain the latest global information.

[0070] It can be understood that by globally updating and aggregating the received intra-group aggregation models and cluster centers through the master node, it can ensure that the learning outcomes of the groups are effectively integrated to form a more comprehensive and accurate global model. This global update can alleviate the deviation problem caused by local optimization to a certain extent, and ensure that the global model better reflects the learning trends and characteristics of the entire system. In addition, synchronizing the global model and the global cluster center to each node in the group can ensure that the learning progress and goals of all nodes in the group are consistent, promote the synchronization of the overall learning process, and avoid the risk of the learning outcomes of each node deviating from the global goal. This strategy not only improves the collaborative efficiency of the learning process, but also helps to quickly update and converge the model in a large-scale distributed environment, thereby enhancing the learning ability and adaptability of the entire system.

[0071] In the above embodiments, the challenges of communication overhead and computing resource limitation faced by existing methods in unmanned clusters can be effectively addressed. First, by dividing the nodes into multiple groups and applying the self-supervised online transfer learning algorithm for training in each group, the amount of calculation required for global model update is reduced, and by generating pseudo labels and updating the local cluster center, each node can make full use of local unlabeled data for online training, thereby reducing communication overhead. At the same time, the mechanism of master node and backup node is adopted, and the master node is responsible for parameter aggregation within the group, ensuring efficient collaboration between nodes, and the backup node takes over the function of the master node under the preset takeover conditions, further improving stability. In addition, the bidirectional ring communication mechanism is adopted to avoid nodes not on the critical forwarding path from processing additional information, further optimizing the communication efficiency between nodes. Finally, through the group aggregation and global aggregation strategy of the master node, the global update of the model can be quickly realized, while ensuring the collaborative work between each node, effectively improving the performance of federated learning in the entire unmanned cluster. This method not only improves the utilization of resources, but also ensures fast convergence and low latency in a dynamically changing network environment.

[0072] In one embodiment, the step of dividing each node into a plurality of groups includes:

[0073] Each node updates its own routing table regularly;

[0074] When it is detected that the position of at least one node or the number of each node has changed, for each node, if the number of neighbor nodes in the routing table of the node is greater than the preset number, the node is used as a temporary group center, and a group invitation is broadcast to each neighbor node in the routing table of the node;

[0075] For a node that receives multiple group invitations, an invitation confirmation message is sent to the node that received the group invitation earliest;

[0076] For each node of the temporary group center, after receiving the invitation confirmation message, the group list is broadcast to each node to form multiple groups.

[0077] Among them, the routing table is a data structure used by each node to record the information of neighboring nodes in the network, including the connection status and related parameters of the node with other nodes. When the number of neighbor nodes in the routing table of a node exceeds the preset threshold, the temporary group center is temporarily selected as the group center, responsible for initiating group invitations and coordinating group operations with neighbor nodes. Group invitation is a broadcast message, a request sent by the temporary group center to its neighbor nodes, in order to invite neighbor nodes to join its group. The invitation confirmation message is a node that receives the group invitation, chooses to respond to the earliest received invitation, and sends a confirmation message to the temporary group center, indicating that it agrees to join the group. The group list is a list of nodes that join the group after the temporary group center receives the invitation confirmation message. The list will then be broadcast to all relevant nodes to ensure the consistency of group members.

[0078] Specifically, the node first broadcasts and updates its own routing table regularly to ensure that it keeps the latest communication information with the surrounding neighbor nodes. This can be done by periodically exchanging neighbor information to ensure that each node has a clear understanding of the surrounding environment, especially in a dynamic environment such as an unmanned cluster, where the location and status of the node may change at any time. Node communication in a large-scale cluster can be completed through wireless broadcasting and node forwarding. Each node broadcasts its latest location information, status, and information about neighbor nodes to the network. After receiving this information, other nodes update their respective routing tables, ensuring that the network topology between nodes is always kept up to date, which is helpful for subsequent grouping and collaborative operations.

[0079] When the location of an unmanned node changes, or the number of nodes in the cluster changes, the node grouping operation is triggered. When the number of network neighbors in the cluster exceeds the preset number, such as more than 4, the node will automatically become a temporary group center. This node will broadcast a group invitation to its neighbor nodes through its routing table, inviting its neighbor nodes to join a new group, ensuring that grouping can only be initiated when the node has enough network neighbors, avoiding unstable grouping caused by too few neighbors of a single node.

[0080] When a node receives multiple group invitations, it will select the one that responds earliest based on the order in which the invitations are received. The node will send an invitation confirmation message to the temporary group center to confirm joining the group, avoiding group conflicts between nodes and ensuring that the grouping operation is efficient and orderly throughout the process. The node only accepts the earliest invitation received, avoiding multiple invitation competition and simplifying the complexity of grouping decisions.

[0081] Once the temporary group center receives enough invitation confirmation messages, it will broadcast the group list to all relevant nodes through wireless broadcast and node forwarding, and forward the information to other temporary group centers to complete the entire grouping process. This broadcast and forwarding process ensures that all nodes can get consistent grouping information and that the newly formed groups can run smoothly in the entire cluster. In this way, nodes in the cluster can be quickly and efficiently organized into multiple groups, ensuring more efficient task collaboration and communication in unmanned clusters.

[0082] In one example, when node A detects that the number of its neighbor nodes is greater than a preset number, it will become a temporary group center and start sending group invitations to its neighbor nodes, such as node B and node C. After receiving the invitation, node B and node C will send an invitation confirmation message to node A if the invitation from node A is the first one received. After node A receives all the invitation confirmations, it will eventually broadcast the confirmed node list, i.e., node B and node C, to all group members to ensure that each node updates its member information and forms a complete group.

[0083] Furthermore, broadcast communication is used between nodes to automatically divide large-scale clusters into neighboring groups of no more than 10 nodes. The upper limit of the number of nodes in a single group can be flexibly adjusted according to the actual task situation, such as the relationship between the node communication coverage and the task scope, the range, the task type, etc.

[0084] In this embodiment, by regularly updating the routing table and triggering the group invitation according to the change in the number of neighbor nodes, the grouping of nodes can be flexibly adjusted according to the actual environmental changes, ensuring that the number of nodes in each group meets expectations and maximizing the collaboration efficiency between nodes. When the node becomes a temporary group center and broadcasts a group invitation, it can quickly and effectively organize a new group, thereby avoiding the situation of over-reliance on a single node and improving the robustness and flexibility of the system. The invitation confirmation mechanism ensures that the selection of group members is reasonable, and the earliest received invitation is responded to first, avoiding multiple competition conflicts and simplifying the grouping operation. In addition, by broadcasting the group list, the consistency of each node's current group member information is ensured, thereby ensuring the efficiency of task collaboration within the group. This grouping mechanism can maintain efficient resource management and coordination in a dynamically changing environment, which helps to improve the response speed and stability of the entire system.

[0085] In one embodiment, the step of training the machine learning model of each node using a self-supervised online transfer learning algorithm for at least one round to obtain a local training model and a local cluster center of each node includes:

[0086] In each training round, for each node, the unlabeled data of the node is input into the pre-trained convolutional neural network model to obtain data features. After the data features are reduced in dimension, the clustering results obtained by clustering are used as pseudo labels, and the pseudo labels are used to update the machine learning model parameters of the node. If the current training round meets the preset training end conditions, the updated machine learning model of the current training round is used as the local training model, and the clustering results of the current training round are used as the local clustering center.

[0087] Specifically, in each training round, the node will first input the unlabeled data it has collected into the pre-trained convolutional neural network model. The convolutional neural network extracts high-level features of the input data through its multiple convolutional layers and pooling layers, such as extracting useful patterns or regularities from information such as temperature and humidity collected by sensors. After feature extraction, the obtained data features will be reduced in dimension to reduce the dimension of the data, reduce the computational burden, and retain the most valuable information. Next, the reduced feature data is grouped by a clustering algorithm to generate pseudo labels. This pseudo label assigns data according to their similarity to form different categories. The node uses the pseudo label to train the machine learning model and update its parameters. The updated model reflects the learning results of the node in the current environment while maintaining the adaptive characteristics of unlabeled data. When the current training round reaches the preset training end condition, such as the number of rounds or performance reaches a certain threshold, the node saves the current trained machine learning model as a local training model and saves the current clustering result as a local clustering center.

[0088] In this embodiment, by inputting unlabeled data into a pre-trained convolutional neural network model and extracting features, the node can use the knowledge and model that have been learned to perform feature mining, avoiding the computational overhead of training from scratch. The dimensionality reduction operation makes the node more efficient when processing high-dimensional data, reduces the consumption of computing resources, and retains important information, which helps to improve the effect of subsequent model training. Using a clustering algorithm to generate pseudo-labels can provide a kind of unsupervised training signal for the node, so that the node can still learn effectively in the absence of manual annotation. By updating the local model parameters and local clustering centers, the node can gradually optimize its machine learning model and provide a valuable initial state for future training rounds. Ultimately, through this process, the node can not only effectively learn the characteristics of local data, but also improve its adaptive ability in a dynamic environment. This method can not only reduce the dependence on labeled data, but also improve the computational efficiency and robustness of the federated learning system in large-scale unmanned clusters, thereby reducing communication and computing costs while maintaining model accuracy.

[0089] In one embodiment, the step of using the clustering result obtained by clustering as a pseudo label and using the pseudo label to update the machine learning model parameter of the node includes:

[0090] The distributed k-means clustering algorithm is used to cluster the data features after dimensionality reduction, and the clustering results are used as pseudo labels. The pseudo labels are used to classify the unlabeled data through the machine learning model of the node to obtain the classification results and the model loss function value. The machine learning model parameters of the node are updated according to the classification results and the model loss function value.

[0091] Among them, the distributed k-means clustering algorithm is used to divide the data into multiple clusters. In a distributed environment, the k-means algorithm can be executed in parallel on multiple computing nodes to improve the efficiency of large-scale data processing. The goal of the algorithm is to iteratively adjust the center point of each cluster until all data points are reasonably classified into different clusters.

[0092] Specifically, the node uses the reduced data features to perform distributed k-means clustering. Due to the high dimensionality of the data, the dimensionality reduction operation first simplifies the data to improve the efficiency of the clustering process. In the distributed k-means algorithm, multiple nodes work together to cluster the data. Each node is responsible for part of the data calculation, and the clustering results are shared among multiple nodes. Each node calculates the local clustering results and assigns pseudo labels to the data based on the clustering results. Next, the node inputs the pseudo labels obtained by clustering into its machine learning model and classifies the unlabeled data. Specifically, the machine learning model uses these pseudo labels as supervision signals to perform classification tasks. The model's prediction results for the data are compared with the actual labels to obtain the classification loss. By calculating the loss function value, the node can evaluate the performance of the model and use the loss function to update the model parameters. In this iterative way, the node gradually optimizes its model.

[0093] In this embodiment, by using the distributed k-means clustering algorithm, the node can effectively cluster high-dimensional data and generate pseudo labels for each data point, avoiding dependence on manual annotation, so that the node can self-learn without labeled data. Dimensionality reduction ensures the efficiency of the feature extraction process while maintaining the core information of the data. Using the pseudo labels generated by clustering for classification and combining the model loss function to update the model parameters can continuously optimize the machine learning model of the node and improve its classification ability for unlabeled data. This process not only effectively utilizes unlabeled data, but also improves the processing speed through distributed computing and adapts to large-scale data environments.

[0094] In one embodiment, in each group, the step of applying a node selection algorithm to select a master node and a backup node includes:

[0095] In each group, each node calculates its participation index and then broadcasts it. Each node calculates the average participation index of the current round based on the received participation indexes. After determining the nodes participating in the competition based on the average participation index of the current round and the pre-adjusted difficulty of the puzzle, the nodes participating in the competition broadcast a completion notification to each node after successfully completing the proof-of-work algorithm. Each node determines the master node and backup node among the nodes participating in the competition based on the received completion notification.

[0096] Among them, the participation index refers to the degree to which each node participates in the task in the current round. This indicator reflects the node's activity and resources invested in this round of competition. A node with high participation means that it contributes more computing power or resources to the current task. The proof-of-work algorithm is used to verify the computing power of a node to complete a certain work task. The node proves that it participated in the calculation and successfully completed a specific workload by calculating the result of a complex task.

[0097] Specifically, each node first calculates its participation index, which can be determined by measuring the computing resources or task completion provided by the node in this round. For example, a node can evaluate its participation based on the amount of computing it performs, the response speed, or the amount of data processed. Each node broadcasts this participation index to other nodes so that other nodes can understand their respective input. Next, all nodes that receive the broadcast information will calculate the average participation index of the current round based on these participation indicators. The calculation method can be to add up the participation of all nodes and then take the average. Then, based on this average participation and the difficulty of the preset problem, such as the difficulty of the proof of work algorithm, the node decides which nodes will participate in the competition in this round. After determining the nodes participating in the competition, these nodes need to complete a proof of work algorithm to prove that they have invested enough computing resources and successfully solved the preset problem. After completion, these nodes will broadcast completion notifications to all other nodes, indicating that they have successfully completed the task. Finally, after receiving these completion notifications, other nodes will select the master node and the backup node based on the content of the completion notification. The master node is the most authoritative or most suitable node among the nodes that successfully completed the task, and it is responsible for further processing of the task. The backup node is a node in standby state. Once the main node fails or cannot continue to work, it can immediately take over the tasks of the main node to ensure the stable operation of the system.

[0098] In this embodiment, by calculating and broadcasting the participation index, each node can fairly evaluate its own and other nodes' contributions in the current round. This process ensures the reasonable distribution of tasks, allows more nodes with greater contributions to participate in the competition, and improves resource utilization. By calculating the average participation index, the competition rules can be adjusted in each round according to the overall situation of participation and the preset difficulty, ensuring that the system is stable and can cope with tasks of different difficulties. After the node completes the task through the proof-of-work algorithm, it broadcasts the completion notification to other nodes, ensuring the transparency and traceability of the task. The election of the master node and the backup node ensures the efficient execution of the task and provides a reliable backup mechanism when the node fails. This method not only improves the system's adaptability, but also reduces the risk of single point failures in the system and enhances the reliability and stability of task processing.

[0099] In one embodiment, the step of each node calculating its participation index includes:

[0100] Each node calculates its participation index according to the following formula:

[0101]

[0102]

[0103] in, is the node participation indicator, and is the percentage of remaining energy and idle computing power; and It indicates the safety threshold of energy and computing power usage set by the administrator in advance. Lower than When the unmanned node stops the mission and automatically returns or lands on the spot; Indicates the number of nodes in the local routing table, represents the number of all nodes in the unmanned network, It represents the ratio of the number of nodes in the local routing table to the number of all nodes in the unmanned network.

[0104] It can be understood that the remaining energy and idle computing power directly affect the ability of a node to perform tasks. If a node has more remaining energy and stronger computing power, it means that it can better complete the task. The normalization of the number of neighbor nodes reflects the position and relative importance of the node in the network. If a node has many neighbors, it may be a key node in the network and can better influence or coordinate task execution.

[0105] In this embodiment, when a node's participation is high, it indicates that the node has sufficient resources and plays an important role in the network, so it can participate more actively in task processing. Conversely, when a node's participation is low, it indicates that its resources are limited or it has few connections with other nodes, and it may not be suitable to continue to undertake tasks. In this way, the network can flexibly adjust which nodes participate in tasks, ensure the smooth execution of tasks, and avoid over-utilization of nodes with limited resources.

[0106] In one embodiment, the step of each node determining a master node and a backup node among the nodes participating in the competition according to the received completion notification includes:

[0107] After receiving the first completion notification and the second completion notification, each node takes the node corresponding to the first completion notification as the first candidate node and the node corresponding to the second completion notification as the second candidate node, and broadcasts them;

[0108] The number of each first candidate node and the number of each second candidate node are counted, and the first candidate node with the largest number is used as the main node, and the second candidate node with the largest number is used as the backup node.

[0109] Specifically, after each node receives two completion notifications, it determines the nodes associated with the first and second completion notifications as the first candidate node and the second candidate node, respectively, and broadcasts the information of these two nodes to the network. All nodes count the number of each first candidate node and the second candidate node based on the received broadcast, and eventually select the first candidate node with the most appearances as the primary node, and the second candidate node with the most appearances as the backup node. This execution method ensures the broad participation and fairness of the election process, avoids the judgment bias of a single node, and ensures that the selected nodes can represent the needs of the entire network through broadcasting and statistics, thereby improving the robustness and fault tolerance of the system, and effectively ensuring that when a node fails, the task can be smoothly switched and continued to be executed, enhancing the stability and sustainability of the network.

[0110] In one embodiment, the process of pre-adjusting the difficulty of the current round of puzzles includes:

[0111] Adjust the difficulty of the current round's puzzles based on the number of completion notifications received by the master node in the previous round and the average participation index.

[0112] Specifically, the master node will first analyze the completion notifications received, count the number of nodes that have completed the task, and combine the average participation index of the nodes in the network - that is, the degree of participation of the nodes in computing and energy resources. If more nodes successfully completed the task in the previous round and the node participation is high, the master node may choose to increase the difficulty of the puzzle to increase the network load and challenge. On the contrary, if the participation is low or there are fewer completion notifications, the master node will appropriately reduce the difficulty of the puzzle to reduce network pressure and ensure the stability of the network.

[0113] In this embodiment, by adjusting the difficulty of the problem, the master node can dynamically adapt to the current state of the network and the capabilities of the nodes, so that the network always remains within a reasonable workload range. Reasonable adjustment of the problem can avoid excessive load or inefficient operation of the network, improve the overall task completion rate and the participation enthusiasm of the nodes. At the same time, by dynamically adjusting the difficulty according to the participation index, the master node can motivate more nodes to participate in the calculation, enhance the synergy of the network, and ensure the efficiency and stability of the system in a multi-node distributed environment. This adjustment mechanism helps to achieve resource balance and load regulation of the network, and improve the robustness and sustainable development of the system.

[0114] In one embodiment, the step of adjusting the difficulty of the current round of puzzles according to the number of completion notifications received by the master node in the previous round and the average participation index includes:

[0115] The difficulty of the current round of puzzles is calculated using the following formula:

[0116]

[0117] in, and Respectively represent the difficulty of the current round of problems and the difficulty of the previous round of problems, and Indicates the number of nodes participating in the last round and the number of nodes in the last two rounds, that is, the number of completion notifications in the last round and the number of completion notifications in the last two rounds. and represents the average engagement index of the last round and the average engagement index of the last two rounds, and is a preset adjustment factor.

[0118] Specifically, if the number of tasks completed in the previous round is greater than the previous two rounds, it means that more nodes actively participated and successfully completed the tasks. The master node will increase the difficulty of the current round of puzzles to increase the challenge of the tasks and prevent the network from being too lax. If the average participation index of the node increases, it means that the node is more active or has more resources to participate in the tasks. The difficulty of the current round of puzzles will be adjusted to ensure that the puzzles will not become too simple due to too low participation, thereby avoiding inefficiency of the system.

[0119] In this embodiment, the difficulty of the current round of puzzles is dynamically adjusted by considering the completion status of the previous two rounds and the changes in node participation, combined with the preset adjustment coefficient. This formula can optimize the difficulty of tasks in real time according to the activity of the network and the completion of tasks, ensure that the difficulty of tasks matches the capabilities and participation of nodes, avoid overloading or underloading the system, and thus improve the stability, efficiency and fairness of the system.

[0120] In one embodiment, the present application also provides a semi-centralized energy-saving federated learning system, the system includes multiple nodes, each node is configured with a machine learning model for federated learning, and the system is used to execute a semi-centralized energy-saving federated learning method such as any of the above embodiments.

[0121] To facilitate understanding of the solution of the present application, specific examples are provided below for illustration.

[0122] like Figure 1 As shown, in this application, the drone group consists of no more than 100 drones, and the performance configurations of each drone are not distinguished. Each drone node consists of a computing module, a data acquisition module, and a communication module. The drone nodes are drones of different types and functions. Let the drone cluster be a set S = {s1, s2, s3…}. Each drone is equipped with sensors and onboard computing capabilities for data collection.

[0123] In an unmanned cluster, each unmanned cluster uses a 5-tuple. Indicates that Indicates its idle computing power, expressed as a multiple of the number of CPU cycles required to process a unit of data; It represents the time energy consumption, which represents the unit energy that must be consumed by the unmanned task as the task time goes by; represents the energy available at the start of the mission; Indicates the remaining energy; Indicates the number of nodes in the local routing table.

[0124] Assume that each unmanned node has completed the model deployment for the FL process before flight. During the flight, data is acquired in real time by the onboard equipment. The unmanned node broadcasts regularly to update the routing table. The node only communicates directly with the neighbor nodes in the routing table, and the communication with nodes other than the neighbor nodes is forwarded by the neighbor nodes according to the latest routing table.

[0125] This application first divides the unmanned nodes into neighboring groups from the perspective of network structure to form multiple neighboring groups. Then, a local training process is carried out, and each unmanned node continues to train the updated local model using its own data. In the aggregation process within the neighboring group, nodes that complete training before the agreed time will participate in the master node competition process to update the model within the group. In the aggregation process between neighboring groups, bidirectional ring communication is used to aggregate the models between groups, and the model update weights of each neighboring group are consistent.

[0126] In the model training step, each node in the neighboring group uses its own collected data and pre-deployed models to complete a round of model training in the onboard computing unit. Figure 2 As shown, the model in the present invention adopts the self-supervised online transfer learning algorithm Fed-deepcluster. This step includes the following sub-steps:

[0127] (1) During each round of training, the unlabeled data of each node is input into a pre-trained deep neural network model to extract data features.

[0128] In order to reduce the computational cost of each node, the data features need to be reduced in dimension. After adopting the max pooling strategy, the parameters of the CNN pooling layer are reduced in dimension through the PCA method. These parameters are regarded as the features of the data. Pooling is a dimensionality reduction operation in convolutional neural networks (CNNs) that is used to reduce the size of feature maps while retaining important feature information. Max pooling selects the largest pixel value as the output of the window within a specified pooling window. This method can retain the strongest features in the feature map because the maximum value represents the most significant feature in the window.

[0129]

[0130] Public is a 1*d dimensional matrix, are the deep neural network parameters, where Represents a specific cell .

[0131] PCA (Principal Component Analysis) is a commonly used data dimensionality reduction method. It transforms the original data into a set of linearly independent representations of each dimension through linear transformation, and is usually used to extract the main characteristic components of the data. The working principle of PCA can be summarized as follows: after subtracting the mean of each dimension in the data set, the covariance matrix is ​​calculated, and the covariance matrix is ​​eigendecomposed to obtain a set of eigenvalues ​​and corresponding eigenvectors. According to the size of the eigenvalue, the eigenvectors corresponding to the first k largest eigenvalues ​​are selected. These eigenvectors are called principal components. The original data is projected onto these principal components to obtain the data representation after dimensionality reduction.

[0132] (2) Input the data features obtained in the previous step into the distributed k-means clustering algorithm to obtain the clustering results, and regard the clustering results as pseudo-labels of the data, thereby transforming the unlabeled classification task into a supervised multi-classification problem. The clustering algorithm used to generate pseudo-labels can be any clustering algorithm suitable for a distributed environment, such as a hierarchical clustering algorithm, a Parallel-DBSCAN algorithm, etc.

[0133] (3) Obtain the classification results of the training data and the model loss function value, and update the neural network parameters through the back propagation algorithm to enhance the feature extraction ability of the model.

[0134] (4) In the local update phase, the model training is repeated for a specified number of rounds. In each round, the feature representation and cluster center are updated using the current network parameters, gradually optimizing the clustering results and the network’s feature extraction capabilities.

[0135] In the master node competition step, if Figure 3 As shown, in each neighboring group, the node selection algorithm esPow is applied during the competition for the master node to determine the master node and the backup node. This step includes the following sub-steps:

[0136] (1) Each node calculates its participation index for this round based on its remaining energy, computing resources, and network connectivity, and then broadcasts it on the network. Setting the participation index provides a reference for the node's decision on whether to participate in the competition, and plays a role in pre-screening high-quality nodes in the process. The index used can be any index that can reflect the performance and reliability of the node.

[0137] (2) After receiving the participation index obtained in step (1) from all nodes, each node calculates the average participation index of the current round.

[0138] (3) Based on the dynamic difficulty of the current round and the current average participation index obtained in step (2), the node decides whether to participate in the competition for the master node according to a predetermined strategy.

[0139] (4) Participating nodes run the PoW algorithm.

[0140] POW (Proof of Work), or proof of work, is usually used in blockchain networks to confirm the nodes that perform accounting within a cycle. The process of proof of work is as follows: First, the blockchain network requires the node to find a hash value of data. This block can be the data of the previous block or other data specified by the designer. The value needs to meet specific conditions, such as the first few bits of the data are zero. The specific number of zeros is called "difficulty". In order to solve this problem, the node will continue to try different input values ​​(nonce) and calculate the hash value of the data until it finds one that meets the conditions. This nonce and the corresponding hash value together constitute the proof of the node's workload.

[0141] When a node finds a hash value that meets the conditions, they will broadcast the result to the network, and other nodes will verify it, that is, recalculate the hash value of the data to ensure that it meets the preset conditions. If the verification is successful, the proof of work is accepted by the network.

[0142] The certification method used may be replaced by any method that can reflect the node's computing resources through computing tasks and the node's communication capabilities through communication processes.

[0143] (5) If (3) is completed before receiving two completion notifications, the participating nodes broadcast the completion notification to the neighboring groups.

[0144] (6) After receiving the first and second completion notifications, it will stop (5) and broadcast the first and second nodes it has received within the group. The node with the largest number of first confirmations will be confirmed as the primary node, and the node with the largest number of second confirmations will be the backup node.

[0145] (7) The master node adjusts the difficulty based on the number of completion notifications received by the master node in the current round and the calculated average participation index to control the number of nodes participating in the competition in each round to 30%-40% of the total network nodes.

[0146] The present invention can meet the needs of large-scale dynamic unmanned clusters and task publishers. Nodes in large-scale dynamic unmanned clusters can progressively obtain a global model with better performance during the task process. At the same time, the self-supervised online transfer learning algorithm Fed-deepcluster proposed by the present invention effectively utilizes unlabeled data and global models to improve the client's processing ability in the face of unknown data on site. The master node competition process of the node selection algorithm esPow effectively guarantees the effective and reliable advancement of the entire training process. The task publisher obtains a global model with high global generalization in the task process, which can effectively process any task-related data and obtain task capabilities online without further subsequent processing.

[0147] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between these entities or operations. Moreover, the term "include", "comprise" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process, method, article or equipment including a series of elements not only includes those elements, but also includes other elements not clearly listed, or also includes elements inherent to such process, method, article or equipment. In the absence of more restrictions, the elements defined by the sentence "including one..." do not exclude the existence of other identical elements in the process, method, article or equipment including the elements. Herein, "one", "one", "said", "the" and "it" may also include plural forms, unless the context clearly indicates another way. A plurality refers to at least two cases, such as 2, 3, 5 or 8, etc. "And / or" includes any and all combinations of the relevant listed items.

[0148] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The various embodiments can be combined as needed, and the same or similar parts can refer to each other.

[0149] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A semi-centralized energy-saving federated learning method, characterized in that: Applied to a semi-centralized energy-saving federated learning system, the system includes a plurality of nodes, each node is configured with a machine learning model for federated learning, and the method includes: After dividing each node into multiple groups, the machine learning model of each node is trained for at least one round using a self-supervised online transfer learning algorithm to obtain a local training model and a local cluster center of each node, wherein the self-supervised online transfer learning algorithm is used to extract data features through a pre-trained convolutional neural network model, and generate pseudo labels through clustering using the data features after dimensionality reduction; In each of the groups, after the node selection algorithm is applied to select the master node and the backup node, the master node of the group performs intra-group update aggregation on each local training model and each local clustering center in the group to obtain the intra-group aggregation model and intra-group clustering center of the group. The backup node is used to take over the master node under the preset takeover conditions; The master nodes of each two groups exchange their own intra-group aggregation models and intra-group clustering centers through bidirectional ring communication; In each of the groups, the master node of the group performs global update aggregation on each of the received intra-group aggregation models and each of the intra-group clustering centers to obtain a global aggregation model and a global clustering center, and synchronizes the global aggregation model and the global aggregation center to each node in the group.

2. The semi-centralized energy-saving federated learning method according to claim 1 is characterized in that: The step of dividing each node into a plurality of groups comprises: Each node updates its own routing table regularly; When it is detected that the position of at least one node or the number of each node has changed, for each node, if the number of neighbor nodes in the routing table of the node is greater than the preset number, the node is used as a temporary group center, and a group invitation is broadcast to each neighbor node in the routing table of the node; For a node that receives multiple group invitations, sending an invitation confirmation message to the node that receives the group invitation earliest; For each node of the temporary group center, after receiving the invitation confirmation message, the group list is broadcast to each node to form multiple groups.

3. The semi-centralized energy-saving federated learning method according to claim 1, characterized in that: The step of training the machine learning model of each node using a self-supervised online transfer learning algorithm for at least one round to obtain a local training model and a local cluster center of each node includes: In each training round, for each node, the unlabeled data of the node is input into the pre-trained convolutional neural network model to obtain data features. After the data features are reduced in dimension, the clustering results obtained by clustering are used as pseudo labels, and the pseudo labels are used to update the machine learning model parameters of the node. If the current training round meets the preset training end conditions, the updated machine learning model of the current training round is used as the local training model, and the clustering results of the current training round are used as the local clustering center.

4. The semi-centralized energy-saving federated learning method according to claim 3 is characterized in that: The step of using the clustering result obtained by clustering as a pseudo label and using the pseudo label to update the machine learning model parameters of the node includes: The distributed k-means clustering algorithm is used to cluster the data features after dimensionality reduction, and the clustering results are used as pseudo labels. The pseudo labels are used to classify the unlabeled data through the machine learning model of the node to obtain the classification results and the model loss function value. The machine learning model parameters of the node are updated according to the classification results and the model loss function value.

5. The semi-centralized energy-saving federated learning method according to claim 1, characterized in that: The step of applying a node selection algorithm to select a master node and a backup node in each of the groups comprises: In each of the groups, each node calculates its participation index and then broadcasts it, and each node calculates the average participation index of the current round based on the received participation indexes, and after determining the nodes participating in the competition based on the average participation index of the current round and the pre-adjusted difficulty of the puzzle, the nodes participating in the competition broadcast a completion notification to each node after successfully completing the proof-of-work algorithm, and each node determines the master node and the backup node among the nodes participating in the competition based on the received completion notification.

6. The semi-centralized energy-saving federated learning method according to claim 5, characterized in that: The step of each node calculating its participation index comprises: Each node calculates its participation index according to the following formula: in, is the node participation indicator, and is the percentage of remaining energy and idle computing power; and It indicates the safety threshold of energy and computing power usage set by the administrator in advance. Lower than When the unmanned node stops the mission and automatically returns or lands on the spot; Indicates the number of nodes in the local routing table, represents the number of all nodes in the unmanned network, It represents the ratio of the number of nodes in the local routing table to the number of all nodes in the unmanned network.

7. The semi-centralized energy-saving federated learning method according to claim 5, characterized in that: The step of each node determining a master node and a backup node among the nodes participating in the competition according to the received completion notification includes: After receiving the first completion notification and the second completion notification, each node takes the node corresponding to the first completion notification as the first candidate node and the node corresponding to the second completion notification as the second candidate node, and broadcasts them; The number of each first candidate node and the number of each second candidate node are counted, and the first candidate node with the largest number is used as the main node, and the second candidate node with the largest number is used as the backup node.

8. The semi-centralized energy-saving federated learning method according to claim 5, characterized in that: The pre-adjustment process of the difficulty of the current round of puzzles includes: Adjust the difficulty of the current round's puzzles based on the number of completion notifications received by the master node in the previous round and the average participation index.

9. The semi-centralized energy-saving federated learning method according to claim 8, characterized in that: The step of adjusting the difficulty of the current round of puzzles according to the number of completion notifications received by the master node in the previous round and the average participation index includes: The difficulty of the current round of puzzles is calculated using the following formula: in, and Respectively represent the difficulty of the current round of problems and the difficulty of the previous round of problems, and Indicates the number of nodes participating in the last round and the number of nodes in the last two rounds, that is, the number of completion notifications in the last round and the number of completion notifications in the last two rounds. and represents the average engagement index of the last round and the average engagement index of the last two rounds, and is a preset adjustment factor.

10. A semi-centralized energy-saving federated learning system, characterized in that: The system includes multiple nodes, each node is configured with a machine learning model for federated learning, and the system is used to execute the semi-centralized energy-saving federated learning method as described in any one of claims 1 to 9.

Images