Decentralized federated learning aggregation method based on trust mechanism and consensus step length

Abstract

The invention relates to the technical field of federated learning, and particularly discloses a decentralized federated learning aggregation method based on a trust mechanism and a consensus step length. According to the method, by introducing the credibility and the dynamic consensus averaging mechanism, model updating can be dynamically adjusted according to the historical performance and similarity of the nodes in the aggregation process, and therefore the information sharing and model parameter aggregation efficiency is optimized. The training speed and the resource utilization efficiency are improved, the robustness and performance of the model in the heterogeneous data environment are enhanced, and more effective decentralized learning is achieved.

Description

Technical Field

[0001] The present invention relates to the field of federated learning technology, and in particular to a decentralized federated learning aggregation method based on a trust mechanism and a consensus step. Background Art

[0002] Federated learning faces many challenges, such as slow training speed, high resource consumption, and susceptibility to extreme clients. In the context of decentralized federated learning, current technologies mainly rely on centralized servers to aggregate and update global model parameters, or use decentralized Gossip protocols to propagate model information. However, these methods have several obvious technical problems and defects: First, the centralized architecture has single point failures and communication bottlenecks. In centralized federated learning, all participating devices must communicate with the central server, which not only increases the communication burden, but also introduces the single point failure problem of the centralized server. Once the server fails, the entire learning process will be interrupted. In addition, as the number of participating nodes increases, the server load and latency will also rise sharply, limiting the scalability of federated learning. Second, the communication efficiency in decentralized methods is low. Decentralized federated learning propagates model parameters between nodes through the Gossip protocol. Although it eliminates the dependence on the central server, since the Gossip protocol usually adopts random neighbor communication, the speed of information transmission is slow, and global convergence in the network requires more communication rounds. This leads to a significant increase in training time in large-scale networks, especially in cases of complex topology or large network latency. In addition, non-IID data is not handled well. In many application scenarios, data distributed on different nodes are often non-IID. Existing decentralized federated learning algorithms have poor model convergence when facing these heterogeneous data, and the final trained global model may not be able to adapt well to the data distribution of all nodes. This is because the model updates of each node vary greatly, resulting in error accumulation during model aggregation, making it difficult to reach an effective consensus.

[0003] A decentralized federated learning model training method is disclosed in CN115860150A, which mainly uses neighbor selection to solve the data heterogeneity problem of decentralized federated learning. Neighbor selection aims to improve the convergence speed and robustness of the model by giving priority to information exchange with nodes with similar local data distribution, but this may lead to unbalanced information sharing, causing some nodes to lag in information update, thereby affecting the performance of the global model. In addition, existing research has also focused on model pruning to solve the communication cost problem of decentralized federated learning. Model pruning reduces communication costs by reducing the number of parameters, thereby improving training efficiency to a certain extent, but this method may lose the expressiveness and accuracy of the model while reducing the complexity of the model, especially when facing non-independent and identically distributed data. In addition, these methods also face the difficulty of balancing training speed, resource consumption, model robustness and model performance in practical applications. In order to ensure that the model can work effectively under the data distribution of different nodes, it is often difficult to meet all requirements by relying solely on neighbor selection or model pruning.

[0004] Therefore, it is difficult for existing methods to find a reasonable balance among these aspects. There is an urgent need to explore improved aggregation methods to optimize the collaboration between nodes through more flexible and adaptive strategies, thereby improving the overall performance and efficiency of decentralized federated learning in heterogeneous environments. Summary of the invention

[0005] In order to overcome the problems of uncoordinated collaboration between nodes and error accumulation during model aggregation in the existing decentralized federated learning technology, the present invention provides a decentralized federated learning aggregation method based on a trust mechanism and a consensus step.

[0006] In order to achieve the above-mentioned object of the invention, the present invention provides the following technical solutions:

[0007] In a first aspect, the present invention provides a decentralized federated learning aggregation method based on a trust mechanism and a consensus step, the method comprising:

[0008] S1. Perform corresponding training on each node through the local data set of each node to obtain feature parameters generated by each node; the feature parameters include feature output and model parameters;

[0009] S2, sending the characteristic parameters of each node to neighboring nodes;

[0010] S3. Each node calculates the cosine similarity of the feature output based on the received feature parameters of the neighboring nodes, adjusts the trust of each neighboring node according to the cosine similarity, allocates the aggregation weight of each neighboring node according to the trust, updates the fusion model parameters according to the aggregation weight, and controls the update interval by the consensus step size during the update process;

[0011] S4. Dynamically update the consensus step size based on the index scheduler, and repeat S1 to S4 until the consensus step size reaches a preset convergence value, completing aggregation.

[0012] According to a specific implementation, in the above aggregation method, adjusting the trust of each neighbor node according to the cosine similarity specifically includes:

[0013] When the cosine similarity exceeds a preset threshold, increasing the trust degree;

[0014] When the cosine similarity is lower than a preset threshold, the trust level is reduced.

[0015] According to a specific implementation, in the above aggregation method, the calculation process of the trust degree includes:

[0016]

[0017] in, is the trust of the i-th node to the j-th neighbor node in the t-th round of update, is the trust of the i-th node to the j-th neighbor node in the t+1th round of update, η is the learning rate, is the trust increment, and the calculation formula of the trust increment is:

[0018]

[0019] in, represents the feature output of the i-th node at time t, represents the feature output of the jth neighbor node at time t, is the cosine similarity of the feature output of the i-th node and the j-th neighbor node, is the cosine similarity variable, τ is the preset threshold, and α and β are constant coefficients.

[0020] According to a specific implementation, in the above aggregation method, the trust is set with an upper threshold and a lower threshold; when the increased trust exceeds the upper threshold, the upper threshold is taken; when the reduced trust is less than the lower threshold, the lower threshold is taken.

[0021] According to a specific implementation, in the above aggregation method, the aggregation weight fusion model parameters are updated, and the calculation formula is:

[0022]

[0023] in, is the model parameter of the i-th node in round t, η is the learning rate, is the loss function The gradient of the local dataset at the i-th node, is the model parameter of the jth neighbor node in round t, γ is the consensus step length, For trust, is the model parameter updated by the i-th node in round t+1.

[0024] According to a specific implementation, in the above aggregation method, the communication used to send the characteristic parameters of each node to the neighboring nodes includes a Gossip protocol.

[0025] According to a specific implementation, in the above aggregation method, dynamically updating the consensus step size based on the index scheduler includes:

[0026] The consensus step size is set based on a preset initial value, and the consensus step size is updated to a preset convergence value based on an exponential scheduler. The calculation formula is:

[0027]

[0028] Among them, γ t is the consensus step length of round t, γ 0 is the preset initial value, r t is the growth rate, r is a constant, m is the number of update rounds at convergence, and ε is the preset convergence value.

[0029] According to a specific implementation, in the above aggregation method, the preset convergence value of the consensus step is 1.

[0030] In a second aspect, the present invention provides a computer-readable medium comprising instructions, which, when executed by a processor of a computing system, cause the computing system to execute a decentralized federated learning aggregation method based on a trust mechanism and a consensus step as described in any one of the above items.

[0031] Compared with the prior art, the present invention has the following beneficial effects:

[0032] For the federated learning scenario of multiple users, the client / server architecture is generally adopted, and the server serves as the center of the federated learning for coordination. Decentralized federated learning supports the multi-party clients participating in the federated learning to communicate with each other, and can effectively process heterogeneous data on different nodes. The original data of the participants does not need to be shared with other participants, and only the model parameters are exchanged. The present invention combines the trust mechanism and the aggregation strategy to ensure that the models of multiple users can still maintain high accuracy and consistency when facing diversified data. First, the present invention dynamically adjusts the contribution of each node in the model aggregation by calculating the cosine similarity, which can effectively reduce the influence of extreme nodes on the global model, thereby improving the robustness of the model under different data distributions, especially in the non-independent and identically distributed (Non-IID) environment. In addition, the present invention uses a dynamic consensus step size design, and the decentralized federated learning can flexibly adjust the update rate of the model parameters at different stages, so that the algorithm converges quickly in the early stage, and makes full use of local data in the later stage, thereby improving the overall training efficiency. The decentralized federated learning architecture provided by the present invention can be applied to various different devices and application scenarios. Each node can be a mobile device of an individual user, an edge computing node or a cloud server, and has good flexibility and portability. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 A flowchart of a decentralized federated learning aggregation method based on a trust mechanism and a consensus step size provided in an embodiment of the present invention;

[0034] Figure 2 A schematic diagram of a decentralized federated learning framework provided by an embodiment of the present invention;

[0035] Figure 3 A schematic diagram of the communication process of gossip communication provided by an embodiment of the present invention;

[0036] Figure 4 A schematic diagram of experimental results of changing the number of nodes provided by an embodiment of the present invention;

[0037] Figure 5 A schematic diagram of experimental results of the impact of heterogeneous data provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0038] The present invention is further described in detail below in conjunction with test examples and specific implementation methods. However, this should not be understood as the scope of the above subject matter of the present invention being limited to the following embodiments, and all technologies realized based on the content of the present invention belong to the scope of the present invention.

[0039] In order to better understand the technical solution provided by the present invention, some terms are explained below.

[0040] First, decentralized federated learning is a distributed machine learning method that aims to allow multiple devices (such as edge devices or nodes) to collaborate on training models without centralizing data. In traditional federated learning, there is usually a central server to coordinate model updates, and each node uploads the model parameters calculated locally to the central server for aggregation. In decentralized federated learning, there is no central server involved, and nodes communicate directly point-to-point and share model parameters with each other. This approach reduces dependence on central servers, enhances the privacy protection of the system, reduces communication bottlenecks, and improves fault tolerance and scalability. Each node independently trains the model on local data, and then exchanges updated model parameters with neighboring nodes. Through multiple rounds of communication and parameter exchange, the model in the entire system is gradually made consistent.

[0041] The role of consensus step size in decentralized federated learning has been widely studied. It can effectively control the convergence rate of each node and its neighbor model parameters in each round of model update. This concept can be traced back to decentralized optimization algorithms such as EXTRA and Push-Sum protocol, where consensus step size is used as a key hyperparameter to control the rate of information convergence in multi-node systems. The core idea is to ensure the stability and global consistency of information propagation while avoiding too fast or too slow model convergence.

[0042] In practical applications, static consensus step size settings are not always optimal, especially when dealing with non-independent and identically distributed (non-IID) data. It has been pointed out in the literature that static step size may cause the model to converge too quickly in heterogeneous data scenarios, thus causing local optimality problems. Therefore, more and more studies tend to adopt dynamic consensus step size adjustment strategies. For example, Solmaz et al. proposed a dynamic consensus mechanism in which the consensus step size gradually increases according to the differences between nodes, ensuring that the system can retain more local information in the early stages and accelerate convergence by increasing the step size in the later stages.

[0043] For example, in the study of Kolomvakis et al., the consensus step size was applied to the distributed optimization problem of smart grids, simulating the process of model training of multiple nodes (i.e., smart devices) without relying on a central controller. By adjusting the consensus step size, the system can fuse heterogeneous data sources in the power grid environment, allowing each node to gradually adapt to each other's differences and ultimately achieve global optimization.

[0044] Another related application comes from Cai et al., who adopted dynamic consensus step scheduling in a decentralized Internet of Things (IoT) environment to address the challenge of heterogeneous data from a large number of edge devices. By adjusting the consensus step size, the system can avoid the rapid convergence of device parameters in the early stage of training, and accelerate the convergence speed in the later stage, improving the efficiency of the entire system.

[0045] The Gossip protocol is a communication protocol used for information dissemination and data synchronization in distributed systems. Its essence is based on point-to-point communication between nodes. By continuously exchanging information with neighboring nodes, the entire network eventually converges to a global consensus. The protocol is widely used in distributed computing, decentralized networks, blockchains, and decentralized federated learning due to its high efficiency, robustness, and scalability.

[0046] The basic idea of ​​the Gossip protocol is similar to the virus propagation model: each node randomly selects a node from its own set of neighbor nodes within a periodic time window and exchanges information with it (such as model parameters, local gradients, features, etc.). After the exchange, both nodes update their own information and further exchange with their neighbor nodes. This process is repeated continuously, and eventually all nodes reach a consensus after multiple rounds of communication.

[0047] In decentralized federated learning, the Gossip protocol is used to exchange model parameters between nodes, eliminating the reliance on central servers. After each round of training, the client exchanges its updated model parameters or feature outputs with neighboring nodes through the Gossip protocol to merge local models. Through multiple rounds of iterations, the Gossip protocol can gradually make the model parameters of all nodes consistent, ensuring the convergence of the global model.

[0048] In this application, the local communication characteristics of the Gossip protocol can reduce bandwidth consumption while maintaining the scalability of the system. Especially in the face of large-scale heterogeneous devices (such as IoT devices) and complex network topology environments, the distributed architecture of the Gossip protocol can adapt to the heterogeneity of devices and avoid the server bottleneck and single point failure problems in traditional federated learning.

[0049] Specifically, the technical solution provided by the embodiment of the present invention is further introduced below.

[0050] Please refer to Figure 1 , which shows a flow chart of a decentralized federated learning aggregation method based on a trust mechanism and a consensus step provided by an embodiment of the present invention, the method comprising:

[0051] S1. Perform corresponding training on each node through the local data set of each node to obtain feature parameters generated by each node; the feature parameters include feature output and model parameters.

[0052] Each node i is from its own local data set D i Get samples for training and update the model parameters w through the local optimization algorithm SGD i After training, each node generates its local feature output and calculates the local loss.

[0053] S2. Send the characteristic parameters of each node to its neighboring nodes.

[0054] After training, each node communicates with neighboring nodes to exchange model parameters and feature outputs. In this embodiment, the communication adopts a Gossip protocol. This point-to-point communication does not rely on a central server, which enhances the scalability and robustness of the system.

[0055] S3. Each node calculates the cosine similarity of the feature output based on the feature parameters of the received neighboring nodes, adjusts the trust in each neighboring node according to the cosine similarity, allocates the aggregation weight of each neighboring node according to the trust, updates the model parameters according to the aggregation weight fusion, and controls the update interval through the consensus step size during the update process.

[0056] Specifically, after obtaining the feature output of the neighbor node, each node calculates the cosine similarity between the local feature output and the feature output of the neighbor node. According to the similarity result, the node will dynamically adjust the trust of the neighbor. This is the trust mechanism proposed in the embodiment of the present invention. The trust mechanism dynamically adjusts the contribution of the neighbor through the similarity of the feature output and plays a core role in DFL. Cosine similarity measures the alignment between two feature vectors, regardless of their size. This makes it particularly useful for processing heterogeneous data: even if the clients experience different degrees of updates, they are still considered to be aligned if their updates follow similar directions. In a non-independent and identically distributed setting, model outputs from different clients may show significant changes. However, if the feature outputs are consistent (i.e., follow the same trend), these nodes can still be considered as trusted neighbors. Compared with Euclidean distance or parameter-based differences, cosine similarity can better capture data heterogeneity, reduce the impact of data imbalance and improve the aggregation process. This enables the model to converge faster and improve accuracy. Further, when the cosine similarity exceeds a preset threshold, the trust is increased; when the cosine similarity is lower than a preset threshold, the trust is reduced. The trust update formula is:

[0057]

[0058] in, is the trust of the i-th node to the j-th neighbor node in the t-th round of update, is the trust of the i-th node to the j-th neighbor node in the t+1th round of update, η is the learning rate, To increase the trust;

[0059] Further, Calculate according to the following formula

[0060]

[0061] in, represents the feature output of the i-th node at time t, represents the feature output of the jth neighbor node at time t, is the cosine similarity of the feature output of the i-th node and the j-th neighbor node, is the cosine similarity variable, τ is the preset threshold, and α and β are constant coefficients.

[0062] To prevent excessive oscillation, the confidence are constrained within upper and lower limits:

[0063]

[0064] After local training, each node i outputs its feature f i With each neighbor node j∈N i The characteristic output f j The comparison is performed and the trust score is updated based on the trust update formula. If the similarity exceeds the threshold τ, the trust score is increased:

[0065]

[0066] If the similarity is below a threshold, the trust score is decreased:

[0067]

[0068] When the increased trust exceeds the upper threshold, the upper threshold is taken; when the decreased trust is less than the lower threshold, the lower threshold is taken.

[0069] This reward and punishment mechanism dynamically adjusts the contribution of each neighbor node, helping the system adapt to the challenges of non-independent and identically distributed data. The trust mechanism enables the DFL system to dynamically adjust the influence of neighbor nodes in each round to ensure the robustness of the aggregation process. This adaptability is crucial to solving the challenges of non-independent and identically distributed data, improving convergence speed, and enhancing model performance.

[0070] Furthermore, in the aggregation phase, the node calculates and updates the model parameters of itself and its neighboring nodes. The aggregation formula is:

[0071]

[0072] in, is the model parameter of the i-th node in round t, η is the learning rate, is the loss function The gradient of the local dataset at the i-th node, is the model parameter of the jth neighbor node in round t, γ is the consensus step length, For trust, is the model parameter updated by the i-th node in round t+1.

[0073] S4. Dynamically update the consensus step size based on the index scheduler, and repeat S1 to S4 until the consensus step size reaches a preset convergence value, completing aggregation.

[0074] Specifically, each node uses an exponential scheduler to dynamically update the consensus step size γ. Initially, the consensus step size is set based on a preset initial value and gradually increases to a preset convergence value with each round of training. The dynamic update of the consensus step size enables the system to gradually adapt to heterogeneous data differences in the early stages of training and accelerate model fusion in the later stages of training.

[0075] It can be understood that the consensus step size (also known as the convergence rate) is a key hyperparameter in DFL, which controls the rate at which each node's model converges to the average of its neighbor models. A fixed consensus step size may cause convergence problems in non-independent and identically distributed settings, either slowing down the convergence speed or causing overfitting. To solve this problem, an embodiment of the present invention introduces an exponential scheduler that gradually increases the consensus step size from a preset initial value to a preset convergence value during training. In the early stages, setting a higher consensus step size will lead to rapid convergence, thereby ignoring valuable local information from each node. As training progresses and the models become more consistent, gradually increasing the consensus step size can accelerate model convergence and ensure efficient convergence. The consensus step size update formula is:

[0076] γ t =γ 0 × t ,

[0077] γ t is the consensus step length of round t, γ 0 is the preset initial value, r t is the growth rate, where r is a constant.

[0078] The step size is updated according to the above formula, and each training iteration brings it closer to the preset convergence value:

[0079]

[0080] Among them, m is the number of update rounds at convergence, and ε is the preset convergence value.

[0081] In a possible implementation, this embodiment sets the preset initial value to 0.1 and the preset convergence value to 1.

[0082] This gradual increase enables the system to balance exploration and exploitation, ensuring independence early on while promoting faster convergence later on. The exponential scheduler ensures that the DFL system maintains sufficient independence early on to avoid local optima and converges quickly to a global solution as training progresses.

[0083] To better understand the technical solution provided by the present invention, please refer to Figure 2 , which shows a schematic diagram of a decentralized federated learning framework provided by an embodiment of the present invention. The decentralized federated learning framework provided by an embodiment of the present invention is designed based on trust and dynamic consensus step with a reward and punishment mechanism. In this architecture, each node exchanges model information with its neighbors through point-to-point communication without relying on coordination from a central server. In order to meet the challenges brought by heterogeneous data distribution, the present invention introduces a trust mechanism and a consensus step in model aggregation. The trust mechanism is used to measure the similarity of feature outputs between the client and neighboring nodes, and to assign different weights to the contributions of model parameters; the consensus step controls the rate of parameter updates to balance the fusion progress of the model.

[0084] It should be noted that the decentralized federated learning architecture provided by the present invention can be applied to a variety of different devices and application scenarios. Each node can represent any computing device, such as a server, a personal computer, a computer workstation, a smart phone, a tablet computer, a smart camera, a smart car or other types of cellular phones, media consumption devices, wearable devices, set-top boxes, game consoles, etc., or a personal user's mobile device, edge computing node, client or cloud server. It should be noted that the nodes of the method described in the embodiments of the present invention are intended to include but are not limited to these and any other suitable types of hardware computing devices.

[0085] Furthermore, the performance of DFL-ATC proposed in the embodiment of the present invention is compared with two well-known algorithms: the centralized federated learning method FedAvg and the decentralized D-PSGD, and they are used as baselines.

[0086] Specifically, basic experiments: In the first experiment, the performance of DFL-ATC is evaluated over 100 rounds of global training on the MNIST, CIFAR-10, and CIFAR-100 datasets. Each node is trained locally for 10 rounds and then synchronized with its neighbors through gossip communication. The communication process is as follows Figure 3As shown. The model exchanges parameters with neighbors, and the consensus step size gradually increases from 0.08 to 1 throughout the training process. The results show that DFL-ATC significantly improves the prediction performance and accelerates convergence. As shown in Table 1, DFL-ATC not only accelerates the convergence rate, but also significantly improves the prediction accuracy. Compared with D-PSGD, DFL-ATC has an accuracy improvement of 17.4%, while being 9.7% higher than FedAvg under certain conditions. These improvements highlight the ability of DFL-ATC to improve the efficiency and effectiveness of decentralized training.

[0087] Table 1 Schematic diagram of the final accuracy of different algorithms

[0088]

[0089]

[0090] Experiments with varying number of nodes: Additional experiments were conducted using 4, 6, and 8 nodes to examine how the number of nodes affects performance. As the number of nodes increases, the heterogeneity of local data increases, making model training more challenging. As a result, the model accuracy decreases slightly. However, as Figure 4 As shown in the figure, when α = 0.1, the accuracy of different models trained for 100 epochs on different datasets and different numbers of clients, DFL-ATC consistently outperforms FedAvg and D-PSGD in all configurations. This shows that DFL-ATC can maintain strong performance even as the network size continues to grow, providing a robust solution for decentralized learning under high heterogeneity.

[0091] Impact of Heterogeneous Data: Dirichlet distributed data with α values ​​of 0.1, 0.5, and 1 are used to simulate different degrees of data heterogeneity. Smaller values ​​of α lead to more unbalanced data distribution among nodes, similar to the non-IID scenario. As expected, model accuracy decreases with decreasing α values, but DFL-ATC still outperforms both baselines in all settings. In the most challenging scenario for model test accuracy (α = 0.1), DFL-ATC is tested on MNIST, CIFAR-10, and CIFAR-100 and consistently achieves superior performance, demonstrating its wide applicability and robustness against extreme heterogeneity. The results are shown in Figure 2. Figure 5 As shown in Figure 1, the accuracy of different models trained for 100 epochs on different datasets and different numbers of clients when α = 0.1, using 8 nodes for experiments. When training on the MNIST dataset, due to the simplicity of the MNIST dataset, overfitting is prone to occur, so the accuracy is observed to drop subsequently. However, no such drop is observed for other datasets.

[0092] Experimental results confirm that trust mechanisms and consensus step sizes have a positive impact on decentralized federated learning in non-IID environments. DFL-ATC not only accelerates convergence but also improves accuracy on a variety of datasets and configurations. Its performance outperforms D-PSGD by 17.4% and FedAvg by 9.7%, highlighting its robustness, scalability, and effectiveness for decentralized training in heterogeneous environments.

[0093] Taking into account the above numerical experimental results, the method provided by the embodiment of the present invention can produce relatively good application effects in some areas where federated learning is applicable and where some high-quality public data sets currently exist, for example:

[0094] 1. Satellite network

[0095] In the application of satellite clusters, the DFL-ATC algorithm allows satellites to share information in a decentralized manner without relying on a central server. The reason for this approach is that the satellite cluster cannot decide which satellite to serve as the central server, because the orbits and positions of all satellites may not be fixed, and any satellite may have limited communication with the ground station at a certain moment. Therefore, the decentralized architecture can effectively avoid the single point of failure and communication bottlenecks caused by centralization, thereby improving the reliability and scalability of the system. Satellites usually face problems of high latency and limited bandwidth. Decentralized federated learning enables satellites to update and optimize models in real time, improving the efficiency of tasks such as environmental monitoring and climate change research. At the same time, the trust mechanism dynamically evaluates the quality of cooperation between satellites to ensure that important tasks can rely on those satellites that perform well, thereby improving the collaborative work capabilities of the entire satellite cluster.

[0096] 2. Smart healthcare

[0097] In the field of smart healthcare, decentralized federated learning significantly enhances data privacy protection. Hospitals and medical devices often face strict privacy and compliance requirements. The DFL-ATC algorithm can perform collaborative model training without sharing sensitive patient information. Since medical data is usually distributed among multiple hospitals and devices, no centralized organization can securely aggregate all data, so a decentralized approach can effectively reduce the potential risk of data leakage. The trust mechanism enables the system to dynamically identify and utilize medical devices that provide high-quality data, thereby improving the accuracy of disease prediction and diagnosis models. At the same time, the application of dynamic consensus step size helps to deal with the inconsistency of data distribution in different hospitals, so that the model can better adapt to the needs of various patients and improve the personalization and accuracy of medical services.

[0098] 3. Drone swarms

[0099] In the application of drone swarms, the DFL-ATC algorithm improves the efficiency of collaborative work in a decentralized manner. When drones perform tasks, such as disaster relief or environmental monitoring, they can share perception information and model parameters in real time without relying on the ground control center. Since drones may be in different locations when performing tasks, and the ground control center may not be able to receive data from all drones in real time, decentralized federated learning can reduce communication delays and enhance real-time decision-making capabilities. In addition, the introduction of the trust mechanism helps drones dynamically evaluate and select reliable neighbors for information exchange, ensuring that each drone can more effectively utilize the environmental information around it and improve the overall efficiency of task completion. The dynamic consensus step size enables drones to flexibly adjust the model fusion speed according to task complexity and environmental changes, and adapt to different operating scenarios.

[0100] Specifically, for the federated learning scenario of multiple users, the client / server architecture is generally adopted, and the server serves as the center of the federated learning for coordination. Decentralized federated learning supports the multi-party clients participating in the federated learning to communicate with each other, and can effectively process heterogeneous data on different nodes. The original data of the participants does not need to be shared with other participants, and only the model parameters are exchanged. The present invention combines the trust mechanism and the aggregation strategy to ensure that the models of multiple users can still maintain high accuracy and consistency when facing diversified data. Based on the deployment of the DFL-ATC method in the process of solving the above problems and implementing the technical solution provided by the embodiment of the present invention, the expected practical effects that can be obtained are as follows:

[0101] 1. High training efficiency: Through the design of dynamic consensus step size, DFL-ATC can flexibly adjust the update rate of model parameters at different stages. This adaptability enables the algorithm to converge quickly in the early stage, while making full use of local data in the later stage, improving the overall training efficiency.

[0102] 2. Enhance model robustness: The trust mechanism dynamically adjusts the contribution of each node in the model aggregation by evaluating the similarity of feature outputs between neighboring nodes. This method can effectively reduce the impact of extreme participants on the global model, thereby improving the robustness of the model under different data distributions, especially in non-independent and identically distributed (Non-IID) environments.

[0103] 3. Optimize resource utilization: DFL-ATC reduces the reliance on central servers through decentralized point-to-point communication, thereby reducing communication costs and resource consumption. This makes the algorithm more scalable in large-scale distributed systems and suitable for application in resource-constrained environments such as the Internet of Things and edge computing.

[0104] 4. Adapt to heterogeneous data: The algorithm can effectively process heterogeneous data on different nodes. Through the combination of trust mechanism and aggregation strategy, it ensures that the model can still maintain high accuracy and consistency when facing diversified data.

[0105] 5. Improve user experience: In practical applications, DFL-ATC can achieve faster model training and updates while ensuring data privacy and security, thereby improving user experience. For example, in smart devices and applications, users can get more accurate personalized services while protecting their sensitive information from being leaked.

[0106] Through these characteristics, the DFL-ATC algorithm not only promotes the development of decentralized federated learning technology, but also provides strong support for practical applications in many fields such as medical health, financial risk control, and intelligent transportation, heralding a broad future prospect.

[0107] In addition, an embodiment of the present invention also provides a computer-readable medium, comprising instructions, which, when executed by a processor of a computing system, enable the computing system to execute a decentralized federated learning aggregation method based on a trust mechanism and a consensus step as described in any one of the above items.

[0108] In the embodiments of the present invention, the processor may be an integrated circuit chip having the ability to process signals. The processor may be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0109] The methods, steps and logic block diagrams disclosed in the embodiments of the present invention can be implemented or executed. The general processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiments of the present invention can be directly embodied as a hardware decoding processor for execution, or can be executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The processor reads the information in the storage medium and completes the steps of the above method in combination with its hardware.

[0110] The storage medium may be a memory, which may be, for example, a volatile memory or a nonvolatile memory, or may include both volatile and nonvolatile memory.

[0111] Among them, the non-volatile memory can be a read-only memory (ROM), a programmable ROM (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory.

[0112] The volatile memory may be a random access memory (RAM) which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), sync link DRAM (SLDRAM), and direct rambus RAM (DRRAM).

[0113] The storage media described in the embodiments of the present invention are intended to include, but are not limited to, these and any other suitable types of memory.

[0114] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A decentralized federated learning aggregation method based on trust mechanism and consensus step length, characterized in that: The method comprises: S1. Perform corresponding training on each node through the local data set of each node to obtain feature parameters generated by each node; the feature parameters include feature output and model parameters; S2, sending the characteristic parameters of each node to neighboring nodes; S3. Each node calculates the cosine similarity of the feature output based on the received feature parameters of the neighboring nodes, adjusts the trust of each neighboring node according to the cosine similarity, allocates the aggregation weight of each neighboring node according to the trust, updates the fusion model parameters according to the aggregation weight, and controls the update interval by the consensus step size during the update process; S4. Dynamically update the consensus step size based on the index scheduler, and repeat S1 to S4 until the consensus step size reaches a preset convergence value, completing aggregation.

2. According to claim 1, a decentralized federated learning aggregation method based on trust mechanism and consensus step length is characterized in that: The trust level of each neighbor node is adjusted according to the cosine similarity, specifically including: When the cosine similarity exceeds a preset threshold, increasing the trust degree; When the cosine similarity is lower than a preset threshold, the confidence level is reduced.

3. According to claim 2, a decentralized federated learning aggregation method based on trust mechanism and consensus step length is characterized in that: The calculation process of the trust degree includes: in, is the trust of the i-th node to the j-th neighbor node in the t-th round of update, is the trust of the i-th node to the j-th neighbor node in the t+1th round of update, η is the learning rate, is the trust increment, and the calculation formula of the trust increment is: Among them, f i t represents the feature output of the i-th node at time t, represents the feature output of the jth neighbor node at time t, Sim(f i t ,f j t ) is the cosine similarity of the feature output of the i-th node and the j-th neighbor node, ΔSim(f i t ,f j t ) is the cosine similarity variable, τ is the preset threshold, and α and β are constant coefficients.

4. According to claim 2, a decentralized federated learning aggregation method based on trust mechanism and consensus step length is characterized in that: The trust degree is provided with an upper threshold and a lower threshold; when the increased trust degree exceeds the upper threshold, the upper threshold is taken; when the reduced trust degree is less than the lower threshold, the lower threshold is taken.

5. The decentralized federated learning aggregation method with any mechanism and consensus step size according to claim 1, characterized in that: The parameters of the aggregation weight fusion model are updated, and the calculation formula is: in, is the model parameter of the i-th node in round t, η is the learning rate, is the loss function The gradient of the local dataset at the i-th node, is the model parameter of the jth neighbor node in round t, γ is the consensus step length, For trust, is the model parameter updated by the i-th node in round t+1.

6. According to claim 1, a decentralized federated learning aggregation method based on trust mechanism and consensus step length is characterized in that: The communication used to send the characteristic parameters of each node to the neighboring nodes includes the Gossip protocol.

7. According to claim 1, a decentralized federated learning aggregation method based on trust mechanism and consensus step length is characterized in that: Dynamically updating the consensus step size based on the exponential scheduler includes: The consensus step size is set based on a preset initial value, and the consensus step size is updated to a preset convergence value based on an exponential scheduler. The calculation formula is: Among them, γ t is the consensus step length of round t, γ 0 is the preset initial value, r t is the growth rate, r is a constant, m is the number of update rounds at convergence, and ε is the preset convergence value.

8. A decentralized federated learning aggregation method based on trust mechanism and consensus step length according to claim 7, characterized in that: The preset convergence value is 1.

9. A computer-readable medium, characterized in that The method comprises instructions which, when executed by a processor of a computing system, cause the computing system to perform the method of any one of claims 1 to 8.

Images