Decentralization fairness federated learning method based on DAG chain
Through multi-dimensional evaluation of client contribution and dynamic reward mechanisms, combined with the cutting-edge roaming fairness method of DAG chain, the problems of unfair contribution assessment and high resource consumption in decentralized federated learning are solved, and the system's fairness and efficiency improvement is achieved.
Patent Information
- Application Number
- CN202510551847.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-04-29
- Publication Date
- 2025-08-01
AI Technical Summary
In the existing decentralized federated learning, the contribution evaluation and incentive mechanism are single, which is difficult to fully reflect the actual contribution of the client. The DAG structure consumes a large-scale distributed environment and has low aggregation convergence efficiency. There is bias in global model selection, resulting in insufficient system fairness and performance.
By evaluating client contributions in multiple dimensions, designing dynamic adjustment reward mechanisms and sparse technology, combining the cutting-edge roaming fairness method of DAG chains, optimizing the reward distribution process, ensuring that the contributions of each client are reasonably evaluated and fairly recognized, and controlling the expansion speed and growth pattern of DAG chains.
It realizes a comprehensive quantitative and fair evaluation of client contributions, improves the fairness and efficiency of the system, reduces communication and computing overhead, optimizes the scalability and aggregation efficiency of the DAG chain, and ensures the overall performance of the system.
Smart Images
Figure CN120409746A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the fields of blockchain and federated learning, and specifically to a decentralized fairness federated learning method based on a DAG chain. Background Art
[0002] With the development of artificial intelligence and blockchain technologies, federated learning (FL), as a distributed machine learning framework, has received extensive attention due to its ability to achieve efficient collaborative training while protecting data privacy. It allows multiple clients to jointly train a machine learning model without sharing raw data. In recent years, decentralized federated learning combined with blockchain technology has further enhanced the robustness and privacy protection capabilities of the system.
[0003] In decentralized federated learning, the directed acyclic graph (DAG) structure has gradually become a research hotspot due to its advantages such as high concurrency, low latency, and strong scalability. The DAG records the update and interaction process of model parameters through distributed ledger technology to ensure the immutability and transparency of data. However, with the expansion of application scenarios, especially in large-scale distributed environments, the existing technologies face some challenges: on the one hand, most of the existing contribution evaluation and incentive mechanisms are measured from a single dimension (such as data volume or gradient quality), making it difficult to comprehensively reflect the actual contributions of clients to model training. On the other hand, the existing reward allocation mechanisms are relatively single and lack flexibility, making it difficult to adapt to diverse application scenarios. In addition, in large-scale distributed scenarios, the scale of the DAG structure may expand rapidly, resulting in problems such as large resource consumption and low aggregation convergence efficiency.
[0004] At the same time, in the DAG structure, the selection and update mechanism of the global model is crucial for the fairness and performance of the system. When existing DAG algorithms select the global model, there may be a bias towards certain nodes or branches, resulting in the neglect of the contributions of some clients. By optimizing the DAG algorithm, the global model can be selected more fairly to ensure that the contributions of each client can be reasonably evaluated and recognized. Summary of the Invention
[0005] The present invention proposes a decentralized fairness federated learning method based on a DAG chain, aiming to optimize the reward allocation process, improve the fairness and efficiency of the system by evaluating client contributions from multiple dimensions, dynamically adjusting the reward mechanism, and sparsification technology, so as to solve the problems raised in the above background art. The technical solutions provided by the present invention are as follows:
[0006] A decentralized fairness federated learning method based on a DAG chain, comprising the following steps:
[0007] Step 1: Divide a large number of clients into multiple independent shards. The client data within each shard is similar. The clients perform model training locally to obtain local model parameters.
[0008] Step 2: Calculate the data contribution of the clients by comprehensively considering the dataset scale, class distribution difference, and data diversity; calculate the model contribution of the clients by comprehensively considering the gradient contribution and parameter update contribution.
[0009] Step 3: The clients within each shard elect a leader for that shard. The shard leader assigns weights based on the data contribution and model contribution of each client, and weighted aggregates the updated gradients uploaded by the clients according to this weight to obtain the globally model parameters preliminarily optimized for that shard.
[0010] Step 4: The shard leader uploads the globally model parameters preliminarily optimized to the DAG chain, and uses the TIP tip-walking fairness method based on the DAG chain to obtain multiple TIP nodes that have passed the review and verification.
[0011] Step 5: Evaluate the globally model parameters preliminarily optimized to obtain their accuracy rate; select multiple TIPs that have passed the review and verification for fusion and evaluation to obtain the first global fusion parameters and their accuracy rate; perform a second fusion and evaluation on the first global fusion parameters and the globally model parameters preliminarily optimized to obtain the second global fusion parameters and their accuracy rate; compare the accuracy rates of the three, and use the parameters corresponding to the highest accuracy rate as the globally model parameters recorded on the DAG chain; the DAG chain distributes the updated globally model parameters to the shard leader.
[0012] Step 6: After receiving the globally model parameters, the shard leader obtains the global gradient vector, determines its sparsification parameter according to the comprehensive performance of each client, and performs a sparsification operation; the shard leader distributes the sparsified gradient vector to each client, and the client uses the sparsified gradient vector for the next round of training to continuously adjust the model.
[0013] Preferably, the calculation formula for the data contribution of the client is:
[0014]
[0015] where the right side of the equation is the three dimensions of dataset scale, class distribution difference, and data diversity respectively, w1 + w2 + w3 = 1, and the weights are used to adjust the importance of each dimension.
[0016] Dataset scale dimension m is the total number of clients within the shard. By comparing the size n of the dataset of client i i with the size n of the dataset of other client j j, reflecting the relative proportion of the dataset size of client i in the whole;
[0017] The category distribution difference dimension uses the Earth Mover's Distance (EMD) to measure the actual difference between distributions, d i is the EMD value between the data distribution of client i and the global data distribution: d i = EMD(P i , Q), where P i represents the data distribution of client i, and Q represents the global data distribution;
[0018] The data diversity dimension uses the information entropy H(D i ) as an evaluation metric. The larger the value of the information entropy, the more evenly distributed the samples of each category are, and the higher the data diversity.
[0019] Preferably, the calculation formula for the model contribution of the client is:
[0020]
[0021] where the right side of the equation is two dimensions, namely the gradient contribution and the parameter update contribution. α + β = 1, and the weights are used to balance the proportions of different contribution types in the model contribution evaluation;
[0022] The gradient contribution dimension calculates the cosine similarity between the client gradient vector and the global gradient vector:
[0023]
[0024] where g i is the gradient vector calculated by the client based on the local model, and g global is the global gradient vector obtained by the weighted average of all client gradient vectors;
[0025] The calculation formula for the parameter update contribution dimension is:
[0026] where K represents the number of iterations of the client itself, represents the parameter update amount of the client in the k-th iteration.
[0027] Preferably, the steps to obtain the global model parameters in step 3 are:
[0028] Combining the data contribution and model contribution of the client, the comprehensive contribution of client i to the global model is obtained:
[0029] W i = C data,i + C model,i
[0030] Calculate the total sum of the comprehensive contributions of all clients within the shard to obtain the weight of the model parameters of client i in the global model aggregation:
[0031]
[0032] For the local model parameters of each client Perform weighted aggregation to obtain the preliminarily optimized global model parameters
[0033] Preferably, the TIP tip walking fairness method based on the DAG chain described in step 4 is specifically as follows: Use the walking algorithm to determine the parent node of each node; Design a dynamically updated set "TIP pool" for storing the evaluated model parameters; Use the TIP selection algorithm to regularly select the optimal model parameters from the "TIP pool" for global aggregation.
[0034] Preferably, use three walking algorithms to divide the entire training process into different algorithm stages, and adjust the algorithm ratio by adjusting the weights at different stages to control the expansion speed and growth form of the DAG chain.
[0035] Preferably, the comprehensive performance S of client i in step 6 i Is calculated by the following formula:
[0036] S i = λV i +(1 - λ)τ i
[0037] Where, on the right side of the equation are the reputation value and the response time factor respectively, λ is the weight for the importance of the reputation value and the response time factor, and its value range is [0,1];
[0038] The reputation value V of client i i The calculation formula is as follows:
[0039]
[0040] Where, ψ is the initial reputation value used to avoid the situation where the reputation value is zero; α is the decay factor used to reduce the influence of early contributions on the current reputation value; β1 and β2 are weights used to balance the importance of data contributions and model contributions, j is other clients other than client i, and t represents the iteration round;
[0041] The response time factor τ of client i i Is calculated by it, and the specific formula is as follows:
[0042]
[0043] Where, T iis the average response time, and ∈1 is an estimated value used to avoid a zero denominator.
[0044] Preferably, the sparsification operation in step 6 is specifically as follows: The comprehensive performance S of client i i determines the sparsification parameter q i , and the calculation formula is:
[0045]
[0046] where D is the dimension of the gradient, and β is a hyperparameter that controls the degree of sparsification; the gradient vector g downloaded by client i i is a sparsified version of the global gradient vector g global , and the sparsification formula is as follows:
[0047] g i = sparsify(g global , q i )
[0048] where sparsify(·) is the sparsification operation, and the specific implementation steps are as follows:
[0049] Sort the global gradient vector g global in descending order of absolute value to determine the importance of each gradient component;
[0050] According to the sparsification parameter q i , select the top q i most important gradient components;
[0051] Set the remaining gradient components to zero to generate the sparsified gradient vector g i .
[0052] Compared with the prior art, the beneficial effects achieved by the present invention are:
[0053] The present invention comprehensively evaluates the contributions of clients in multiple dimensions, combines data diversity, class distribution differences, as well as gradient contributions and parameter update contributions in model training, and comprehensively quantifies the actual contributions of each client. By designing a dynamic adjustment reward mechanism, based on the reputation value and response time factor of the client, combined with the sparsification technology, gradients with different degrees of sparsification are allocated according to the comprehensive performance of the client to encourage its active participation and improve system efficiency. At the same time, the sparsified gradient not only retains the core information of the global gradient but also reduces communication and computational overhead. By proposing a cutting-edge walk fairness method based on the DAG structure, using the centralized random walk algorithm to select the optimal parent nodes in the DAG, combined with the similarity and accuracy of model parameters for efficient aggregation and update, the scalability and aggregation efficiency of the DAG are optimized, and the expansion speed and growth form of the DAG are controlled to avoid the rapid expansion of the DAG scale and ensure the fairness and overall performance of the system. Brief Description of the Drawings
[0054] The drawings are used to provide a further understanding of the present invention and form a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation to the present invention. In the drawings:
[0055] Figure 1 is the flowchart of the federated learning of the present invention;
[0056] Figure 2 is the diagram of the decentralized federated learning framework of the present invention. Detailed Description of the Embodiments
[0057] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.
[0058] In order to make the above objects, features and effects of the present invention more obvious and understandable, the present invention will be further described in detail below with reference to the drawings and specific embodiments.
[0059] Embodiment 1:
[0060] A decentralized fairness federated learning method based on a DAG chain, as Figure 1 shown, includes the following steps:
[0061] Step 1, according to factors such as data characteristics and network topology, divide a large-scale client group into multiple independent shards. The client data within each shard has a certain similarity to reduce internal communication overhead and improve training efficiency. Each client uses its own data to train the model locally.
[0062] Step 2, use EMD to evaluate the class distribution difference between the client data and the global data, calculate the information entropy to measure the data diversity, and comprehensively consider the dataset size, class distribution difference and data diversity to calculate the data contribution of the client. Evaluate the gradient contribution through the cosine similarity between the client gradient vector and the global gradient vector; measure the parameter update contribution by calculating the amount of model parameter update in each iteration; determine the model contribution of the client by comprehensively considering the gradient contribution and the parameter update contribution.
[0063] Step 2.1, the data contribution of the client is specifically calculated in the following way:
[0064] Dataset size: n is the total number of clients. By comparing the size n of the dataset of client ii The size n of the dataset of other client j j , reflecting the relative proportion of the dataset scale of client i in the whole.
[0065] Category distribution difference: To accurately measure the difference in category distribution between client data and global data, the present invention uses the Earth Mover’s Distance (EMD) to measure the actual difference between distributions:
[0066]
[0067] where P and Q represent two distributions respectively, and f ij represents the flow of moving the i-th category in distribution P to the j-th category in distribution Q, and d ij represents the specificity between category i and category j. The goal of EMD is to minimize the total moving cost.
[0068] If there is a large difference in category distribution between client data and global data, it means that the client data may contain category information missing in the global data, which has important supplementary value for the training of the global model. The present invention defines d i as the EMD value between the data distribution of client i and the global data distribution, that is:
[0069] d i = EMD(P i , Q)
[0070] where P i represents the data distribution of client i, Q represents the global data distribution, and by calculating d i the actual difference in category distribution between the data of client i and the global data can be quantitatively measured.
[0071] Data diversity: The present invention uses information entropy as an evaluation index for data diversity. Information entropy is a statistic that measures the degree of uniformity of data distribution, and its calculation formula is:
[0072]
[0073] where p ij represents the probability of the j-th class sample appearing in the dataset of client i. By counting all k class samples, the larger the value of information entropy, the more uniform the distribution of various class samples in the dataset, and the higher the data diversity. By calculating the information entropy, the present invention can quantitatively evaluate the diversity of client data, thereby providing an important basis for the evaluation of data contribution.
[0074] Considering the three dimensions of dataset scale, category distribution difference and data diversity comprehensively, the calculation formula for data contribution is:
[0075]
[0076] Among them, w1 + w2 + w3 = 1, and these weights are used to adjust the importance of each dimensional feature in the evaluation.
[0077] Step 2.2, the model contribution of the client is specifically calculated in the following way:
[0078] Gradient contribution:
[0079] Cosine similarity is a measure used to measure the similarity degree of two vectors in direction, and it reflects the similarity by calculating the cosine value of the included angle between two vectors. For two vectors A and B, their cosine similarity is defined as:
[0080]
[0081] Among them, A·B is the dot product of vectors A and B, and the calculation formula is:
[0082]
[0083] |A| and |B| are the Euclidean norms of vectors A and B respectively, and the calculation formulas are:
[0084]
[0085] Therefore, the cosine similarity between the client gradient vector and the global gradient vector is calculated through the above formula to evaluate the contribution of the client to the global model:
[0086]
[0087] Among them, g i is the gradient vector calculated by the client according to the local model, and g global is the global gradient vector obtained by the weighted average of all client gradient vectors.
[0088] Parameter update contribution:
[0089] During the local training process of the client, the update amplitude of the model parameters can, to a certain extent, reflect its contribution to the model training. In the present invention, the norm of the model parameter update amount of client i in each iteration is calculated and denoted as Δθ i . The greater the parameter update amplitude of the client, it indicates that it has made a relatively large adjustment to the model according to the local data, and the promotion effect on the model training is also stronger. In order to better evaluate the contribution of the client at different iteration stages, based on the client's own iteration history, the normalized contribution is calculated to ensure that the contributions of different clients are compared on the same scale, and to avoid the contribution evaluation deviation caused by different iteration times:
[0090]
[0091] Among them, K represents the number of iterations of the client itself, represents the parameter update amount of the client in the k-th iteration.
[0092] Combining the two dimensions of gradient contribution and parameter update contribution, the calculation formula for the model contribution is:
[0093]
[0094] Among them, α + β = 1. By adjusting the weights, according to the actual application scenarios and requirements of federated learning, the proportion of different contribution types in the model contribution evaluation can be balanced. Through this comprehensive calculation formula, the present invention can comprehensively and accurately evaluate the model contribution of the client, so as to more comprehensively consider the contribution of each client when aggregating the global model and achieve better fairness performance.
[0095] Step 3, within each shard, a committee is elected according to indicators such as the computing resources, historical training stability, and contribution enthusiasm of the clients. The committee consists of multiple leaders and is responsible for supervising the training process of the clients within the shard to ensure the fairness and efficiency of training. As Figure 2 shown, the shard leader collects the model parameters of all clients within this shard, and according to the client data contribution and model contribution calculated in Step 2, assigns weights to the model parameters of each client. The shard leader aggregates these weighted model parameters to obtain the global model parameters within the shard, completing the global model update and preliminary optimization within the shard. The global model update is specifically as follows:
[0096] Combining the data contribution and model contribution of the client, the comprehensive contribution of the client to the global model is obtained:
[0097] W i = C data,i + C model,i
[0098] Calculate the sum of the comprehensive contributions of all clients within the shard, and obtain the weight of the model parameters of client i in the global model aggregation:
[0099]
[0100] The greater the weight, the greater the influence of the client in the global model aggregation. In the global model aggregation stage, assuming that the local model parameters of each client are Then the updated global model parameters:
[0101]
[0102] The meaning of this formula is to take the local model parameters of each client and perform weighted summation according to their respective corresponding weights w i to obtain the global model parameters W within the shard after update global (0) .
[0103] Step 4, the shard leader uploads the obtained global model parameters to the DAG chain. In large-scale distributed scenarios, decentralized federated learning significantly improves the processing capacity and efficiency of the system with the help of sharding and parallel processing technologies, providing strong support for model training and optimization; however, with the continuous increase in the number of clients, direct on-chain interaction has triggered new challenges, the scale of the DAG chain has expanded rapidly, resource consumption has increased sharply, and the aggregation convergence efficiency has also been greatly reduced. To solve these problems, the present invention adopts the TIP tip-walking fairness method based on the DAG chain, specifically as follows:
[0104] In the DAG chain, TIP (Top Intersection Point) refers to those end nodes that have not been referenced by other nodes. These nodes are key points for model parameter update and aggregation because they represent the latest information that has not yet been integrated into the global model. When a new transaction is submitted, it is added to the DAG chain as a new node. If this transaction is not referenced by any other transaction, then this node becomes a TIP. When a new node is added, one or more TIPs are randomly selected as its parent nodes through random walk, and these parent nodes will be referenced by the new node and thus are no longer TIPs. By aggregating the model parameters of the selected TIPs, an update of the global model can be generated. By reasonably selecting and managing TIPs, the expansion speed and direction of the DAG chain can be effectively controlled, thereby optimizing the aggregation process of model parameters.
[0105] The tip-walking fairness method is a consensus algorithm based on the DAG chain. By performing random walks in the DAG chain to select the optimal parent nodes and combining key factors such as the similarity and accuracy of model parameters, it ensures the efficient aggregation and update of model parameters while achieving fairness. The present invention constructs a DAG chain by integrating multiple participants or groups of participants and defining the paths for model parameter transfer and update between them. Based on the DAG chain, a centralized random walk algorithm is adopted to determine the parent nodes of each node. This step is crucial for accelerating the aggregation and update process of model parameters because it can ensure that each node can obtain the latest model parameters from the nodes most similar to it. Subsequently, a dynamically updated set called the "TIP pool" is designed to store the evaluated model parameters. By introducing an efficient consensus or TIP selection algorithm, the optimal model parameters can be periodically selected from the "TIP pool" for global aggregation. This step ensures that the models in the network can be continuously optimized, thereby continuously improving the performance of federated learning.
[0106] The main process on the DAG chain is the asynchronous model parameter fusion process. Each TIP node on the DAG usually can have 2 - 8 parent nodes, but 2 parent nodes often have relatively good verification efficiency. Having more than 2 parent nodes can make the model fusion more comprehensive, but the low time efficiency is not conducive to the rapid expansion of the DAG. Therefore, 2 parent nodes are often used. The main consensus algorithm of the DAG chain is to find the parent nodes through the TIP selection algorithm. At the same time, the parent nodes find their approvers and are added to the "TIP pool" after evaluation. According to the overall system structure of the present invention, three walking algorithms are designed, dividing the entire training process into different algorithm stages. In different stages, the algorithm ratio is adjusted by adjusting the λ1 and λ2 control algorithms, thereby improving the expansion speed of the DAG chain, the growth form of the DAG chain, and the training efficiency of the task.
[0107] The first walking algorithm is through multiple rounds of walks. In each round, two TIPs are selected and their performance is evaluated. Finally, the TIP with the best performance is selected as the parent node. This method can explore simultaneously in multiple directions, enhancing the horizontal concurrency of the DAG. Walk p rounds, and the committee evaluates the performance of the model based on the global test set. In each round of walk, two TIPs are selected and evaluated. After walking p rounds, the group of TIPs with the highest accuracy is recorded as its own parent node, and it is added to the approval pools of the two parent nodes. This algorithm requires a relatively long evaluation time. The more rounds p, the longer the evaluation time, and the greater the possibility of finding a more suitable and better-performing model. Therefore, the selection of p needs to consider the balance between time and performance for the purpose of the entire system task.
[0108] The random walk algorithm 2 selects the child node with a large weight according to the established weight. However, an algorithm that only relies on weight for random walk is likely to cause the continuation of the advantage of a certain branch. It is possible that the model trained at a certain node has good initial performance, and following the weight for random walk will make the advantage grow like a snowball. Moreover, a model with good initial performance does not necessarily mean good subsequent training performance. The previous random walk algorithms basically only perform random walk according to weight, which is risky. Therefore, the fairness of the entire system needs to be considered comprehensively.
[0109] The random walk algorithm 3 conducts a single-round random walk to deeply explore the vertical structure of the DAG, and selects a TIP with the optimal performance as the parent node. This method optimizes the selection of TIPs by deeply exploring the vertical structure of the DAG, and the time cost is relatively low. The specific process is similar to that of the random walk algorithm 1. The TIP with the highest accuracy will be found after p rounds. However, the random walk algorithm 1 focuses on adjusting the horizontal structure of the DAG chain, while the random walk algorithm 3 focuses on adjusting the vertical structure of the DAG chain. By controlling the random walk algorithms 1 and 3, the trend of the entire DAG chain can be adjusted.
[0110] Step 5: Evaluate the global model parameters W uploaded by the shard leader to obtain its accuracy acc1; select multiple verified TIPs from the "TIP pool" for the first fusion and evaluation to obtain the first global fusion parameters W global (0) and its accuracy acc2; perform the second fusion and evaluation on the first global fusion parameters W global (1) and the global model parameters W uploaded by the shard leader global (1) to obtain the second global fusion parameters W global (0) and its accuracy acc3. Compare acc1, acc2, and acc3, record the highest accuracy in the transaction information, and use the corresponding parameters as the global model parameters W recorded on the DAG chain global (2) ; The DAG chain distributes the updated global model parameters to the shard leader to provide the global model gradient for the next round of training. global Specifically, m verified TIPs are defined as P
[0111] (k, v), i ∈ 1... m, and its parameters include all key-value pairs. The total parameter set is Key = {k|k ∈ P1 ∪ P2... ∪ P i}, and parameter alignment is performed: m}
[0112]
[0113] After alignment, update the total weight parameter dictionary:
[0114]
[0115] Average the parameters according to the FedAvg algorithm:
[0116]
[0117] Fuse and evaluate W avg to obtain the first global fusion parameter W global (1) and its accuracy acc2.
[0118] Step 6: After receiving the global model gradient, the shard leader determines the sparsification parameter according to the comprehensive performance of each client and sparsifies the global model; the leader distributes the sparsified model gradient to each client within the shard, and the client uses the sparsified gradient for the next round of training. During the training process, it continues to evaluate its own contribution and continuously adjusts the model, forming a complete closed-loop iterative process to continuously improve the performance of the global model and the overall efficiency of the federated learning system.
[0119] Regarding the sparsification degree of the model assigned to each client, the present invention uses the reputation value and response time index of each client to measure. The reputation value V of client i i is calculated based on its historical performance and contribution stability. The specific formula is as follows:
[0120]
[0121] where ψ is the initial reputation value, usually a small positive number used to avoid the case where the reputation value is zero; α is the decay factor used to reduce the influence of early contributions on the current reputation value; β1 and β2 are weights used to balance the importance of data contributions and model contributions, j is other clients other than client i, and t represents the iteration round.
[0122] The response time factor τ of client i i is calculated through its average response time T i The specific formula is as follows:
[0123]
[0124] where ∈1 is a very small positive number used to avoid division by zero. The comprehensive performance S of client i i is calculated by the following formula:
[0125] S i = λV i +(1 - λ)τ i
[0126] Among them, λ is a weight used to balance the importance of the reputation value and the response time factor, and its value range is [0, 1].
[0127] To allocate gradients according to the comprehensive performance of the client, the present invention adopts an improved sparsification method. Sparsification is a common optimization technique that improves computational efficiency and resource utilization by reducing data redundancy and complexity. Specifically, the comprehensive performance S of the client i determines the sparsification parameter q i , which represents the number of gradient components to be retained, and the calculation formula is:
[0128]
[0129] where D is the dimension of the gradient, that is, the length of the gradient vector, and β is a hyperparameter that controls the degree of sparsification and is used to adjust the sensitivity of the sparsification operation.
[0130] The gradient gi downloaded by client i i is a sparsified version of the global gradient vector g global , and the sparsification formula is as follows:
[0131] gi i = sparsify(g global , q i )
[0132] where sparsify(·) is the sparsification operation used to retain the most important components in the gradient vector. The specific implementation steps of the sparsification operation are as follows:
[0133] Sort the global gradient vector g global in descending order of absolute value to determine the importance of each gradient component. Components with larger absolute values usually have a greater impact on model updates;
[0134] According to the sparsification parameter q i , select the top q i most important gradient components;
[0135] Set the remaining gradient components to zero to generate the sparsified gradient gi i .
[0136] In this way, the sparsification operation can effectively reduce the redundant information in the gradient vector while retaining the components that are most critical for model updates. The present invention can dynamically adjust the degree of sparsification according to the comprehensive performance of the client. Clients with higher comprehensive performance will obtain more gradient components and thus be able to update the local model more completely.
[0137] Example 2:
[0138] The computer-readable storage medium of this embodiment stores a computer program, and when the program is executed by a processor, it implements the steps in a decentralized fairness federated learning based on a DAG chain in Embodiment 1.
[0139] The computer-readable storage medium of this embodiment can be an internal storage unit of the terminal, such as the hard disk or memory of the terminal; the computer-readable storage medium of this embodiment can also be an external storage device of the terminal, such as a plug-in hard disk, a smart memory card, a secure digital card, a flash card, etc. equipped on the terminal; further, the computer-readable storage medium can also include both the internal storage unit and the external storage device of the terminal.
[0140] The computer-readable storage medium of this embodiment is used to store the computer program and other programs and data required by the terminal, and the computer-readable storage medium can also be used to temporarily store the data that has been output or will be output.
[0141] Embodiment 3:
[0142] The computer device of this embodiment includes a processor, a memory, and a computer program stored on the memory and executable on the processor. When the processor executes the program, it implements the steps in a decentralized fairness federated learning based on a DAG chain in Embodiment 1.
[0143] In this embodiment, the processor can be a central processing unit, or other general-purpose processors, digital signal processors, application-specific integrated circuits, off-the-shelf programmable gate arrays, or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc.; the memory can include a read-only memory and a random access memory, and provides instructions and data to the processor. A part of the memory can also include a non-volatile random access memory. For example, the memory can also store information about the device type.
[0144] Those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on such an understanding, the above technical solutions, in essence, or the part that contributes to the prior art can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.
[0145] The above are only the preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or perform equivalent replacements for some of the technical features. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.
Claims
1. A decentralized fairness federated learning method based on the DAG chain, characterized in that, It includes the following steps: Step 1: Divide a large number of clients into multiple independent shards. The client data within each shard is similar. The clients perform model training locally to obtain local model parameters; Step 2: Calculate the data contribution of the clients by comprehensively considering the dataset scale, class distribution difference, and data diversity; Calculate the model contribution of the clients by comprehensively considering the gradient contribution and parameter update contribution; Step 3: The clients within each shard elect a leader for that shard. The shard leader assigns weights based on the data contribution and model contribution of each client, and weighted aggregates the updated gradients uploaded by the clients according to the weights to obtain the globally model parameters preliminarily optimized for that shard; Step 4: The shard leader uploads the preliminarily optimized globally model parameters to the DAG chain, and uses the TIP tip-walking fairness method based on the DAG chain to obtain multiple TIP nodes that have passed the review and verification; Step 5: Evaluate the preliminarily optimized globally model parameters to obtain their accuracy; Select multiple TIPs that have passed the review and verification for fusion and evaluation to obtain the first global fusion parameters and their accuracy; Perform a second fusion and evaluation on the first global fusion parameters and the preliminarily optimized globally model parameters to obtain the second global fusion parameters and their accuracy; Compare the accuracies of the three, and use the parameters corresponding to the highest accuracy as the globally model parameters recorded on the DAG chain; The DAG chain distributes the updated globally model parameters to the shard leader; Step 6: After receiving the globally model parameters, the shard leader obtains the global gradient vector, determines its sparsification parameters according to the comprehensive performance of each client, and performs a sparsification operation; The shard leader distributes the sparsified gradient vector to each client, and the client uses the sparsified gradient vector for the next round of training to continuously adjust the model.
2. The decentralized fairness federated learning method based on the DAG chain according to claim 1, characterized in that The calculation formula for the data contribution of the client is: Where the right side of the equation is the three dimensions of dataset scale, class distribution difference, and data diversity respectively, w1 + w2 + w3 = 1, and the weights are used to adjust the importance of each dimension; Dataset scale dimension m is the total number of clients within a shard. By comparing the size n of the dataset of client i i with the size n of the dataset of other client j j , it reflects the relative proportion of the dataset scale of client i in the whole; The category distribution difference dimension uses the earth mover's distance EMD to measure the actual difference between distributions, d i is the EMD value between the data distribution of client i and the global data distribution: d i = EMD(P i , Q), where P i represents the data distribution of client i, and Q represents the global data distribution; The data diversity dimension uses the information entropy H(D i ) as an evaluation metric. The larger the value of the information entropy, the more uniform the distribution of various samples, and the higher the data diversity.
3. The decentralized fairness federated learning method based on DAG chain according to claim 2, characterized in that, The calculation formula for the model contribution of the client is: Where the right side of the equation is the two dimensions of gradient contribution and parameter update contribution respectively, α + β = 1, and the weights are used to balance the proportion of different contribution types in the model contribution evaluation; The gradient contribution dimension calculates the cosine similarity between the client gradient vector and the global gradient vector: Among them, g i is the gradient vector calculated by the client based on the local model, and g global is the global gradient vector obtained by the weighted average of all client gradient vectors; The calculation formula for the parameter update contribution dimension is as follows: Among them, K represents the number of iterations of the client itself, represents the parameter update amount of the client in the k-th iteration.
4. The decentralized fairness federated learning method based on the DAG chain according to claim 3, wherein, The steps to obtain the globally model parameters in Step 3 are: Comprehensively consider the data contribution and model contribution of the clients to obtain the comprehensive contribution of client i to the global model: W i = C data,i + C model,i Calculate the sum of the comprehensive contributions of all clients within the shard to obtain the weight of the model parameters of client i in the global model aggregation; The local model parameters of each client are weighted and aggregated to obtain the preliminarily optimized global model parameters 5. The decentralized fairness federated learning method based on the DAG chain according to claim 1, characterized in that The TIP tip-walking fairness method based on the DAG chain described in Step 4 is specifically: Use a walking algorithm to determine the parent node of each node; Design a dynamically updated set "TIP pool" for storing the model parameters that have passed the evaluation; Use the TIP selection algorithm to regularly select the optimal model parameters from the "TIP pool" for global aggregation.
6. The decentralized fairness federated learning method based on the DAG chain according to claim 5, characterized in that, Three random walk algorithms are used to divide the entire training process into different algorithm phases. In different phases, the algorithm ratio is adjusted by adjusting the weights to control the expansion speed and growth form of the DAG chain.
7. A decentralized fairness federated learning method based on the DAG chain according to claim 1, characterized in that, The comprehensive performance S of client i in step 6 i is calculated by the following formula: S i = λV i +(1 - λ)τ i Among them, the right side of the equation is the reputation value and the response time factor respectively. λ is the weight for the importance of the reputation value and the response time factor, and its value range is [0, 1]. The credit value V of client i i The calculation formula is as follows: Among them, ψ is the initial reputation value used to avoid the situation where the reputation value is zero; α is the decay factor used to reduce the influence of early contributions on the current reputation value; β1 and β2 are weights used to balance the importance of data contributions and model contributions. j is other clients other than client i, and t represents the iteration round. Response time factor τ of client i i It is used for calculation, and the specific formula is as follows: Among them, T i is the average response time, and ∈1 is an estimated value used to avoid a zero denominator.
8. A decentralized fairness federated learning method based on a DAG chain according to claim 7, characterized in that, The sparsification operation in step 6 is specifically as follows: the comprehensive performance S of client i i determines the sparsification parameter q i , and the calculation formula is: where D is the dimension of the gradient, and β is a hyperparameter that controls the degree of sparsification; the gradient vector gi downloaded by client i i is the sparsified version of the global gradient vector g global and the sparsification formula is as follows: g i = sparsify(g global , q i ) Among them, sparsify(·) is a sparsification operation, and the specific implementation steps are as follows: For the global gradient vector g global Sort by absolute value to determine the importance of each gradient component; According to the sparsification parameter q i , select the top q i most important gradient components; Set the remaining gradient components to zero to generate the sparsified gradient vector g i .
9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps in a decentralized fairness federated learning method based on a DAG chain as described in any one of claims 1-8.
10. A computer device, comprising a processor, a memory, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps in a decentralized fairness federated learning method based on a DAG chain as described in any one of claims 1-8.