Method and system for evaluating multi-dimensional participant contribution, and model aggregation optimization method
By using a federated learning framework that dynamically adjusts the weights of magnitude, direction, and uniqueness dimensions, the static and single-dimensional problems of participant contribution evaluation are solved. This improves the model's convergence speed and accuracy, prevents malicious attacks, and ensures the transparency and fairness of the evaluation through blockchain notarization.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- BEIJING ELECTRONIC DIGITAL INTELLIGENCE TECHNOLOGY CO LTD
- Filing Date
- 2026-03-31
- Publication Date
- 2026-07-03
AI Technical Summary
In the existing federated learning framework, the evaluation of participants' contributions suffers from problems such as staticity, single dimension, delayed feedback, and unreliable evidence, resulting in a disconnect between the evaluation results and the actual value of the data, and making it impossible to build a fair and efficient federated collaboration loop.
By dynamically adjusting the weights of amplitude, direction, and uniqueness in three dimensions, the contribution of participants is evaluated in real time based on the rate of change of model accuracy. Transparency is ensured through blockchain notarization, thus synchronizing contribution evaluation with the training process and intervening in the model aggregation process in real time.
It improves model convergence speed and accuracy, prevents malicious attacks, achieves fair contribution assessment and efficient data value aggregation, and reduces audit costs.
Smart Images
Figure CN122332865A_ABST
Abstract
Description
Technical Field
[0001] This disclosure relates to the fields of data element circulation and privacy computing technology, and in particular to a method and system for evaluating the contribution of multi-dimensional participants, and a model aggregation optimization method. Background Technology
[0002] Against the backdrop of increasingly stringent data privacy protection, federated learning, as a distributed training paradigm where data is usable but not visible, has been widely applied in various fields. By having multiple participants retain their private data locally and collaboratively train a global model, it avoids the risk of data leakage while aggregating the value of multi-source data. The evaluation of participant contributions is crucial for the stable operation of a federated learning system, directly impacting participation enthusiasm, training efficiency, and model performance. A reasonable evaluation mechanism can incentivize the input of high-quality data and suppress malicious behavior.
[0003] However, the current federated learning framework still suffers from several prominent problems in evaluating the contributions of participants. First, it suffers from static bias; existing solutions often price data based on volume or pre-set quality in the early stages of training, failing to dynamically adjust based on real feedback from the data throughout the training process, leading to a disconnect between evaluation results and the actual value of the data. Second, the evaluation dimensions are singular, focusing only on data scale and failing to differentiate between diverse contributions such as data size, gradient direction guidance, and feature uniqueness. Third, it lacks a closed-loop incentive and constraint mechanism; evaluation lags behind training, making it impossible to intervene in the aggregation process in real time, resulting in slow model convergence and vulnerability to free-rider attacks. Fourth, the evidence is unreliable; the transparency of contribution calculation is insufficient, transaction records are easily tampered with, and audit costs are increased. These problems severely restrict the large-scale implementation and sustainable development of federated learning, necessitating a robust contribution evaluation mechanism to address them. Summary of the Invention
[0004] In view of this, the present disclosure provides a method and system for evaluating the contribution of multi-dimensional participants, as well as a model aggregation optimization method. This method can solve the four major pain points that existing federated learning frameworks in the prior art generally face in evaluating the contribution of participants: static evaluation, single dimension, delayed feedback, and unreliable evidence. These problems lead to a disconnect between contribution evaluation and the true value of data, and make it impossible to build a fair and efficient federated collaboration loop.
[0005] In a first aspect, embodiments of this disclosure provide a method for evaluating the contribution of multi-dimensional participants, including: Initialize the global model and distribute it to the multi-dimensional participants; Each participant performs iterative training of the global model based on its corresponding local data, and obtains local model information and uploads it to the server after each iteration. The overall accuracy improvement factor for each client is determined based on the local model information. Obtain the model accuracy change rate corresponding to the comprehensive accuracy improvement factor for each client, and dynamically adjust the magnitude dimension weight, the direction dimension weight, and the uniqueness dimension weight according to the model accuracy change rate; Based on the dynamically adjusted magnitude dimension weights, direction dimension weights, and uniqueness dimension weights, the dynamically adjusted comprehensive accuracy improvement factor is obtained. The dynamically adjusted comprehensive accuracy improvement factor and the locally trained model parameters corresponding to all clients are dynamically aggregated to obtain the updated global model parameters, which are then distributed to the multi-dimensional participants for local training. The dynamic adjustment of the overall accuracy improvement factor for the corresponding round when the training stopping condition is met is obtained and denoted as the target factor. The model contribution information for each participant is determined based on the target factor and the historical cumulative contribution of decay.
[0006] Secondly, embodiments of this disclosure also provide a system for evaluating the contributions of multi-dimensional participants, including: An initialization unit is used to initialize the global model and distribute the global model to the multi-dimensional participants; The training unit is used by each participant to perform iterative training of the global model based on the corresponding local data, and to obtain local model information and upload it to the server after each iteration. The comprehensive accuracy improvement factor acquisition unit is used to determine the comprehensive accuracy improvement factor for each client based on the local model information. The weight dynamic acquisition unit is used to acquire the model accuracy change rate corresponding to the comprehensive accuracy improvement factor for each client, and dynamically adjust the magnitude dimension weight, the direction dimension weight, and the uniqueness dimension weight according to the model accuracy change rate. The factor dynamic update unit is used to obtain the dynamically adjusted comprehensive accuracy improvement factor based on the dynamically adjusted magnitude dimension weight, the direction dimension weight, and the uniqueness dimension weight. The dynamic aggregation unit is used to dynamically aggregate the dynamically adjusted comprehensive accuracy improvement factor and the locally trained model parameters corresponding to all clients, obtain the updated global model parameters, and distribute them to the multi-dimensional participants for local training. The target factor acquisition unit is used to obtain the dynamically adjusted comprehensive accuracy improvement factor for the corresponding round when the training stopping condition is met, denoted as the target factor; The contribution information analysis unit is used to determine the model contribution information of each participant based on the target factor and the historical decay cumulative contribution.
[0007] Thirdly, embodiments of this disclosure also provide a model aggregation optimization method, including: Initialize the global model and distribute it to the multi-dimensional participants; Each participant performs iterative training of the global model based on its corresponding local data, and obtains local model information and uploads it to the server after each iteration. The overall accuracy improvement factor for each client is determined based on the local model information. Obtain the model accuracy change rate corresponding to the comprehensive accuracy improvement factor for each client, and dynamically adjust the magnitude dimension weight, the direction dimension weight, and the uniqueness dimension weight according to the model accuracy change rate; Based on the dynamically adjusted magnitude dimension weights, direction dimension weights, and uniqueness dimension weights, the dynamically adjusted comprehensive accuracy improvement factor is obtained. The dynamically adjusted comprehensive accuracy improvement factor and the locally trained model parameters corresponding to all clients are dynamically aggregated to obtain the updated global model parameters, which are then distributed to the multi-dimensional participants for local training. Training is stopped when the training stopping condition is met.
[0008] The multi-dimensional contribution evaluation method disclosed in this application analyzes the rate of change of model accuracy in each round and dynamically adjusts the weights of the three dimensions of amplitude, direction, and uniqueness at different training stages. The comprehensive accuracy improvement factor obtained in each round can most realistically reflect the comprehensive contribution value of the corresponding participant in the current iteration round, rather than a coarse statistical analysis of a single dimension. This application thoroughly deconstructs the data value from the three dimensions of amplitude, direction, and uniqueness, and the dynamically adjusted evaluation factor directly participates in the aggregation. Compared with the traditional Fed Avg, the convergence speed is significantly improved and the accuracy is higher. At the same time, it is difficult for malicious nodes to simultaneously forge gradients with high similarity and high uniqueness, effectively preventing attacks such as gradient flipping. This method can accurately obtain the contribution information of each participant.
[0009] The above description is merely an overview of the technical solution disclosed herein. In order to better understand the technical means of this disclosure and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this disclosure more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description
[0010] To more clearly illustrate the technical solutions of the embodiments of this disclosure, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0011] Figure 1A flowchart illustrating the method for evaluating the contribution of multi-dimensional participants provided in this embodiment of the disclosure.
[0012] Figure 2 A flowchart illustrating the method for obtaining the overall accuracy enhancement factor for each client provided in this embodiment of the disclosure.
[0013] Figure 3 A flowchart illustrating the method for obtaining the information contribution capability of each client provided in this embodiment of the disclosure.
[0014] Figure 4 This is a flowchart illustrating the method for obtaining the direction consistency information of each local gradient and the global gradient provided in this embodiment of the disclosure.
[0015] Figure 5 This is a flowchart illustrating a method for dynamically adjusting dimensional weights based on the rate of change of model accuracy, as provided in an embodiment of this disclosure.
[0016] Figure 6 A flowchart illustrating the method for obtaining model contribution information for each participant provided in this embodiment of the disclosure.
[0017] Figure 7 This is a schematic flowchart of the model aggregation optimization method provided in the embodiments of this disclosure. Detailed Implementation
[0018] The embodiments of this disclosure will now be described in detail with reference to the accompanying drawings.
[0019] It should be understood that the following specific examples illustrate the implementation of this disclosure, and those skilled in the art can easily understand other advantages and effects of this disclosure from the content disclosed in this specification. Obviously, the described embodiments are only a part of the embodiments of this disclosure, and not all of them. This disclosure can also be implemented or applied through other different specific implementation methods, and the details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this disclosure. It should be noted that, in the absence of conflict, the following embodiments and features in the embodiments can be combined with each other. Based on the embodiments in this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.
[0020] It should be noted that various aspects of embodiments within the scope of the appended claims are described below. It will be apparent that the aspects described herein can be embodied in a wide variety of forms, and any particular structure and / or function described herein is merely illustrative. Based on this disclosure, those skilled in the art will understand that one aspect described herein can be implemented independently of any other aspect, and two or more of these aspects can be combined in various ways. For example, any number of aspects set forth herein can be used to implement the device and / or practice the method. Additionally, this device and / or method can be implemented using structures and / or functionalities other than one or more of the aspects set forth herein.
[0021] It should also be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of this disclosure. The drawings only show the components related to this disclosure and are not drawn according to the number, shape and size of the components in actual implementation. In actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.
[0022] Furthermore, specific details are provided in the following description to facilitate a thorough understanding of the examples. However, those skilled in the art will understand that the described aspects can be practiced without these specific details.
[0023] Reference Figure 1 This application discloses a method for evaluating the contributions of multidimensional participants, including: S100: The server initializes the global model and distributes it to the multi-dimensional participants.
[0024] In S200, each participant performs iterative training of the global model based on its corresponding local data, and obtains local model information after each iteration and uploads it to the server.
[0025] The local model information includes the model parameter update amount and individual gradient for each client.
[0026] S300 determines the overall accuracy enhancement factor for each client based on local model information.
[0027] S400 obtains the model accuracy change rate corresponding to the comprehensive accuracy improvement factor for each client, and dynamically adjusts the magnitude dimension weight, direction dimension weight, and uniqueness dimension weight based on the model accuracy change rate.
[0028] S500 obtains the dynamically adjusted comprehensive accuracy improvement factor based on the dynamically adjusted amplitude dimension weight, direction dimension weight, and uniqueness dimension weight.
[0029] The S600 dynamically aggregates the dynamically adjusted comprehensive accuracy improvement factor and the locally trained model parameters for all clients to obtain updated global model parameters and distributes them to multi-dimensional participants for local training.
[0030] S700 is the dynamic adjustment factor for the overall accuracy improvement in the corresponding round when the training stopping condition is met, denoted as the target factor.
[0031] S800 determines the model contribution information for each participant based on the target factor and the historical cumulative contribution of decay.
[0032] The multi-dimensional contribution evaluation method disclosed in this application analyzes the rate of change of model accuracy in each round and dynamically adjusts the weights of the three dimensions of amplitude, direction, and uniqueness at different training stages. The comprehensive accuracy improvement factor obtained in each round can most realistically reflect the comprehensive contribution value of the corresponding participant in the current iteration round, rather than a single-dimensional coarse statistics. This application thoroughly decomposes the data value from the three dimensions of amplitude, direction, and uniqueness. The dynamically adjusted evaluation factor directly participates in the aggregation, that is, the contribution evaluation is synchronized with the training process, and it can intervene in the model aggregation process in real time. Compared with the traditional Fed Avg, the convergence speed is significantly improved and the accuracy is higher. At the same time, it is difficult for malicious nodes to simultaneously forge gradients with high similarity and high uniqueness, effectively preventing gradient flip attacks. This method can accurately obtain the contribution information of each participant.
[0033] For S200, local model information includes the model parameter update amount and individual gradients for each client.
[0034] No. The model parameter update amount for each client in round t is: The corresponding individual gradient is .
[0035] The individual gradient is obtained by differentiating the local data with respect to the global model, and the parameter update is obtained by multiplying the individual gradient by the learning rate. The client does not upload data or the complete model, but only the gradient or update.
[0036] The parameter update increment is -(individual gradient × learning rate), and the core logic is to take the negative value along the gradient descent direction.
[0037] Specifically, the methods for obtaining individual gradients include: 1) server-side broadcasting of the first... Round global model parameters: 2) Client Load local dataset 3) Global model Locally, forward inference yields the predicted value: 4) Use local tags Calculate the loss: 5) Backpropagation, for Differentiate to obtain the first The individual gradient of a client in round t: That is, the individual gradient of the i-th client in the t-th round is calculated by the client i using the t-th batch of local data to run the local model once.
[0038] For specific references to S300 Figure 2 The methods for obtaining the overall accuracy improvement factor for each client include: S310 obtains the information contribution capability of the corresponding client based on the model parameter update amount.
[0039] Simultaneous parameters Figure 3 The methods for obtaining each client's information contribution capability include: S311, obtain the L2 norm of the corresponding client based on the model parameter update amount, i.e. .
[0040] S312, obtain the sum of the norms of all participants, i.e. ; S313, determine the information contribution capability of each client based on the L2 norm and the sum of norms.
[0041] The information contribution capability of the i-th client is : , This represents the amount of model parameter updates for the i-th client.
[0042] In this embodiment, the client's information contribution capability is measured in terms of magnitude, which measures the physical scale of parameter updates. Essentially, this embodiment measures the update step size (magnitude) provided by the participant. In federated learning, a larger step size typically indicates that the participant has updated more model parameters, carrying a more active information flow. Normalization eliminates the interference of overall model scaling on contribution evaluation, making the contribution comparable across rounds.
[0043] S320 obtains the direction consistency information between each local gradient and the global gradient based on the individual gradient.
[0044] In this embodiment, the direction consistency information between each local gradient and the global gradient is the information of the direction dimension.
[0045] Simultaneous parameters Figure 4 The method for obtaining the direction consistency information between each local gradient and the global gradient specifically includes: S321, gradient for all individuals Perform a weighted summation to obtain the weighted global gradient. ; S322, based on individual gradient Weighted global gradient Determine the degree of directional overlap for each local gradient.
[0046] The degree of overlap of the directions corresponding to the local gradient of the i-th client is: , The individual gradient corresponding to the i-th client , This is a weighted global gradient.
[0047] S323, Obtain individual gradients respectively Weighted global gradient The L2 norm is calculated, and the similarity information between each local gradient and the global gradient is obtained based on the degree of directional overlap.
[0048] The similarity information between the local gradient and the global gradient of the i-th client is as follows: .
[0049] S324, normalize the similarity information to obtain directional consistency information.
[0050] The direction consistency information corresponding to the i-th client is: : .
[0051] In this embodiment, the angle between the local gradient and the global weighted gradient is calculated using a cosine similarity function, and the result is linearly mapped to an interval. The smaller the angle (the closer the cosine value is to 1), the closer the direction derived by the participants is to the globally optimal path. If the cosine value is negative (deviation from the direction), the mapping result will approach 0, thereby achieving automatic weight reduction for updates due to directional deviation or abnormal perturbations, ensuring the convergence stability of model training.
[0052] S330: Select any client as the target party and obtain the average gradient of all other participants globally. And based on the average gradient Individual gradient of the target party This allows for the identification of unique characteristics for each client.
[0053] In this embodiment, the unique capability corresponding to each client is the information of the uniqueness dimension, used to identify non-redundant unique information. The unique capability corresponding to the i-th client is... : .
[0054] This embodiment utilizes the gradient inner product to calculate the orthogonal component between the local gradient and the average gradient of the other participants. It removes redundant parts from individual gradients that are homogeneous with the group, leaving behind unique, differentiated information. This embodiment is specifically designed to identify long-tail features and scarce data. In multi-party data collaboration, highly homogeneous data is often present. Using this formula, the system can accurately locate unique and valuable sample providers and award them high scores, thus completely solving the free-rider problem.
[0055] S340, based on the information contribution capability of each client. , Directional consistency information Uniqueness The overall accuracy improvement factor for each client is determined by weighting the magnitude dimension, direction dimension, and uniqueness dimension.
[0056] The overall accuracy improvement factor for the i-th client is : , , , These are the magnitude dimension weight, direction dimension weight, and uniqueness dimension weight, respectively.
[0057] It should be noted that when calculating the overall accuracy improvement factor for each client for the first time, the magnitude dimension weight, direction dimension weight, and uniqueness dimension weight used are all initially set values. Preferably, the magnitude dimension weight is initially set to 0.2, the direction dimension weight is initially set to 0.5, and the uniqueness dimension weight is initially set to 0.3.
[0058] To address the issue of existing evaluation dimensions being singular and focusing solely on data scale, this application introduces three weighted dimensions: magnitude (ω1), direction (ω2), and uniqueness (ω3). The weighting of these three dimensions can be dynamically adjusted based on accuracy improvements (e.g., increasing the weighting of direction and uniqueness if accuracy improvement is insufficient). This design effectively distinguishes between different types of contributors, such as those contributing based on data scale, gradient direction guidance, and feature uniqueness, breaking away from the sole focus on data volume. It accurately identifies contributors to high-quality, high-value data, improving the comprehensiveness and accuracy of contribution evaluation.
[0059] For S400, referencing Figure 5 The method of dynamically adjusting dimension weights based on the rate of change of model accuracy specifically includes: S410 obtains the rate of change for each client based on the comprehensive accuracy improvement factor calculated in each round.
[0060] The change rate ∆acc corresponding to each client is the ratio of the factor difference (i.e., the difference between the comprehensive accuracy improvement factors in the current round and the previous round) to the comprehensive accuracy improvement factor in the previous round. The calculation formula is: ∆acc = (difference) / comprehensive accuracy improvement factor in the previous round.
[0061] S420, the server obtains the weight dynamic adjustment strategy corresponding to the change rate according to the preset threshold, and determines the adjusted amplitude dimension weight, direction dimension weight, and uniqueness dimension weight based on the weight dynamic adjustment strategy.
[0062] Specifically, 1) preset the accuracy improvement rate threshold T, and compare the accuracy improvement rate ∆acc calculated by each client with the threshold T; 2) If the accuracy improvement rate ∆acc of any client is < T, automatically trigger the weight regulation logic to adjust the amplitude dimension weight, direction dimension weight, and uniqueness dimension weight corresponding to this client; the specific adjustment method is: keep the sum of the weights of the three unchanged, adjust the weights of the direction dimension weight and uniqueness dimension weight upward by a preset ratio respectively, and synchronously adjust the weight of the amplitude dimension weight downward, and the total amount of the weights of the direction dimension weight and uniqueness dimension weight increased is equal to the total amount of the weight of the amplitude dimension weight decreased.
[0063] The existing solutions lack a real-time intervention mechanism, and the evaluation lag leads to slow model convergence and vulnerability to free-rider attacks. In this application, by presetting the accuracy improvement rate threshold, when the client accuracy improvement rate is lower than the threshold, the weight regulation logic is automatically triggered, and a weight verification step is added (if the improvement requirement is not met, the adjustment is repeated), forming a real-time closed loop of evaluation-regulation-verification. This closed loop can intervene in the global model aggregation process in real time, encourage participants to continuously provide high-quality data (improve their own accuracy improvement rate to obtain more reasonable weights), inhibit malicious behaviors such as free-riding, and at the same time accelerate the model convergence speed and improve the training efficiency and global model performance.
[0064] S430, determine the comprehensive accuracy improvement factor of each client according to the information contribution ability, direction consistency information, uniqueness ability, adjusted amplitude dimension weight, adjusted direction dimension weight, and adjusted uniqueness dimension weight corresponding to each client.
[0065] Further, for S420, it specifically includes: S421, if the change rate ∆acc is less than the preset threshold, call the first weight dynamic adjustment strategy, and adjust the current direction dimension weight and uniqueness dimension weight upward according to the first weight dynamic adjustment strategy, and adjust the current amplitude dimension weight downward; S422, if the rate of change ∆acc is greater than the preset threshold, the second weight dynamic adjustment strategy is invoked, and the current magnitude dimension weight is adjusted upward according to the second weight dynamic adjustment strategy, while the current direction dimension weight and uniqueness dimension weight are both adjusted downward.
[0066] The adjustment ratios for the directional dimension weights and the uniqueness dimension weights are the same, and the sum of the adjusted magnitude dimension weights, directional dimension weights, and uniqueness dimension weights is 1.
[0067] Furthermore, the adjustment ratio of the directional dimension weight is 10% to 20% of the current corresponding directional dimension weight.
[0068] Furthermore, the adjustment ratio of the amplitude dimension weight is 50% to 60% of the current corresponding amplitude dimension weight.
[0069] Existing solutions mostly use static pricing in the early stages of training, failing to incorporate feedback and adjustments throughout the training process, leading to a disconnect between evaluation and the actual value of the data. This application calculates a comprehensive accuracy improvement factor in each round and dynamically triggers weight adjustments based on the accuracy improvement rate (the percentage difference between the current and previous round's factor). This ensures that contribution evaluation is integrated throughout the entire training process, matching the true value of data at different iteration stages in real time, avoiding the lag and bias of static evaluation, and making the evaluation results more closely reflect the actual training effect.
[0070] For S500, the dynamically adjusted overall accuracy improvement factor for the i-th participant is: .
[0071] For S600, the updated global model parameters are: : That is, to calculate the overall accuracy improvement factor of each participant. They are then aggregated as weights to generate a new global model.
[0072] Then, for S700, the model parameters are re-issued based on the new round of global model, and the cycle execution of S200-S600 is repeated until the training of each participant meets the training stopping condition. The dynamic adjustment of the comprehensive accuracy improvement factor of the corresponding round is obtained and recorded as the target factor. At the same time, the smart contract is triggered to distribute the benefits, that is, to execute S800.
[0073] For S800, also refer to Figure 6 The method for obtaining model contribution information for each participant specifically includes: S810 determines the cumulative contribution of each participant in the current round based on the target factor and the historical decay cumulative contribution.
[0074] The cumulative contribution of the i-th participant (i.e., participant i) in the current round throughout the entire collaboration cycle is: : ; The historical decay factor is preferably 0.7 to 0.9; T is the total number of iterations for the federated learning training task, which is also the end point of the model training cycle. In a set of training projects, when the model reaches the preset accuracy or the preset maximum number of iterations, the training stops, and the number of iterations at this time is T. It is the current iteration round, with a value range of [1,T]. When locating a specific training round, the system can capture and calculate the real-time contribution indicators made by the participants in each specific "round". Let i be the target factor for the i-th participant in the current round; The preset total revenue represents the total commercial value or incentive pool generated by this multi-party collaboration, which is the pre-agreed revenue distribution base and represents the total value created by the circulation of data elements.
[0075] S820: Obtain the percentage of each participant's cumulative contribution in the current round.
[0076] Wherein, the percentage corresponding to the cumulative contribution of the i-th participant in the current round is: , It is the sum of the cumulative contributions of all participants. This percentage can achieve a fair distribution ratio, ensuring that every bit of profit strictly corresponds to its weight in the overall value enhancement.
[0077] S830 determines the model contribution information for each participant based on the preset total revenue and percentage.
[0078] The model contribution information corresponding to the i-th participant is: : , This is the preset total revenue.
[0079] The method disclosed in this embodiment can accumulate contribution factors calculated in real time in each round using an exponential moving average (EMA). Simultaneously, a decay factor ensures that more recent contributions have a larger weighting in the final allocation. This embodiment effectively solves the problem of temporal consistency in contribution assessment, acknowledging the long-term contributions of participants in building the initial basic model while assigning higher weight to recent model fine-tuning (addressing bottlenecks) through a decay mechanism. This guarantees that the logic of profit distribution has a complete time dimension, making the final profit distribution not only fair but also auditable and verifiable.
[0080] Furthermore, this application also includes: in each round, writing the information contribution capability, directional consistency information, uniqueness capability, comprehensive accuracy improvement factor, updated magnitude dimension weight, updated directional dimension weight, updated uniqueness dimension weight, and model contribution information of each participant to the blockchain for real-time notarization. The on-chain recording of details for each iteration provides a reliable auditing foundation for data element transactions. Compared to existing solutions that suffer from unreliable notarization and insufficient transparency, this design improves the transparency and auditability of contribution evaluation, reduces the risk of transaction record tampering, lowers auditing costs, provides a reliable basis for calculating participant contributions and resolving disputes, and enhances the practicality and credibility of the solution.
[0081] The proposed solution fully adheres to the core paradigm of federated learning, which states that data is available but not visible. Participants only need to calculate the comprehensive accuracy improvement factor based on local data, without having to upload private raw data. This achieves accurate contribution assessment and weight adjustment while avoiding the risk of data leakage. It not only meets the strict requirements for data privacy protection but also fully aggregates the value of multi-source data, promoting the large-scale implementation and sustainable development of federated learning.
[0082] The following details the evaluation method for the contributions of the multidimensional participants that this application seeks to protect, using specific examples. Example 1: Training of a medical joint diagnostic imaging model; Participants: Hospital A (a large general hospital with 5,000 cases of routine pneumonia data), Hospital B (a specialized hospital with 200 cases of rare lung disease data), Hospital C (an imaging center with a moderate amount of data but high-quality data manually annotated by senior experts), and AI company D (providing a privacy computing platform and computing resources).
[0083] Initial parameters: Set the evaluation factor weights ω1=0.2 (amplitude), ω2=0.5 (direction), ω3=0.3 (uniqueness); and the attenuation factor γ=0.8.
[0084] The dynamic evaluation and coefficient adjustment process specifically includes: Phase 1: Basic Feature Construction Phase (Rounds 1-20) Feature Performance: During this phase, the model is in the global contour learning phase, requiring a large number of samples for support. Due to its massive data volume, Hospital A's uploaded parameter update ∆θ is extremely significant, and the L2 norm (amplitude dimension) indicator accounts for more than 60% of all parties. Dynamic Adjustment: The system detects a rapid increase in model accuracy (acc) (i.e., greater than the preset threshold), triggering an adaptive weight adjustment mechanism, moderately increasing ω1 (amplitude weight) to 0.3, and decreasing ω2 to 0.45 (direction) and ω3 to 0.25 (uniqueness). Return Coefficient Fluctuation: During this phase, Hospital A leads in single-round accuracy improvement factors, and its cumulative contribution is rapidly increasing, temporarily giving it a leading weight in return allocation.
[0085] Phase Two: Accuracy Platform Breakthrough Phase (Rounds 21-80) Characteristics: Routine pneumonia identification approaches saturation, and model progress slows. At this point, although the gradient generated by the rare disease cases provided by Hospital B has a small amplitude (L2 norm), its gradient direction is highly orthogonal to other directions, resulting in a significantly higher gradient inner product (uniqueness dimension) after normalization. Meanwhile, Hospital C's annotations are accurate, with its gradient direction highly consistent with the global aggregation direction, exhibiting excellent cosine similarity (direction dimension). Dynamic Adjustment: The system detects that ∆acc (accuracy improvement rate) is below the threshold and automatically triggers logic: increasing the weights of ω2 (direction) and ω3 (uniqueness) while compressing the weight of ω1 (amplitude). Fluctuations in Revenue Factors: Due to the weights shifting towards "uniqueness," the single-round comprehensive accuracy improvement factor corresponding to Hospital B begins to surpass it. Through exponential moving average calculations, the cumulative contribution weight of Hospital B steadily increases, reflecting dynamic compensation for the value of scarce data.
[0086] Phase 3: Convergence and Settlement Period (Round 81 and beyond) Stopping Determination: When the model's AUC on the validation set reaches 0.95 and remains stable for 5 consecutive rounds, the system determines that the preset accuracy has been reached and training stops. Final Coefficient Generation: The smart contract automatically extracts the 80 rounds of historical data stored on the blockchain and calculates the weighted sum. Blockchain Evidence Records: The L2 norm, cosine value, and inner product value of all rounds are uploaded to the blockchain in real time at the end of each round, ensuring that the allocation basis cannot be tampered with.
[0087] Secondly, this application discloses a system for evaluating the contributions of multi-dimensional stakeholders, used to perform the evaluation method for multi-dimensional stakeholder contributions disclosed in the first aspect of this application. The system includes: The initialization unit is used by the server to initialize the global model and distribute the global model to the multi-dimensional participants. The training unit is used by each participant to perform iterative training of the global model based on the corresponding local data. After each iteration, the local model information is obtained and uploaded to the server. The overall accuracy improvement factor acquisition unit is used to determine the overall accuracy improvement factor for each client based on local model information. The dynamic weight acquisition unit is used to obtain the model accuracy change rate corresponding to the comprehensive accuracy improvement factor for each client, and dynamically adjust the magnitude dimension weight, direction dimension weight, and uniqueness dimension weight according to the model accuracy change rate. The factor dynamic update unit is used to obtain the dynamically adjusted comprehensive accuracy improvement factor based on the dynamically adjusted magnitude dimension weight, direction dimension weight, and uniqueness dimension weight. The dynamic aggregation unit is used to dynamically aggregate the dynamically adjusted comprehensive accuracy improvement factor and the locally trained model parameters corresponding to all clients, obtain the updated global model parameters, and distribute them to the multi-dimensional participants for local training. The target factor acquisition unit is used to obtain the dynamically adjusted comprehensive accuracy improvement factor for the corresponding round when the training stopping condition is met, denoted as the target factor; The contribution information analysis unit is used to determine the model contribution information of each participant based on the target factor and the historical cumulative contribution.
[0088] Reference Figure 7 Thirdly, this application discloses a model aggregation optimization method, including: S10, the server initializes the global model and distributes the global model to the multi-dimensional participants; S20, each participant performs iterative training of the global model based on the corresponding local data, and obtains local model information after each iteration and uploads it to the server; S30, determine the overall accuracy improvement factor for each client based on local model information; S40: Obtain the model accuracy change rate corresponding to the comprehensive accuracy improvement factor for each client, and dynamically adjust the magnitude dimension weight, direction dimension weight, and uniqueness dimension weight based on the model accuracy change rate. S50, based on the dynamically adjusted amplitude dimension weight, direction dimension weight, and uniqueness dimension weight, obtains the dynamically adjusted comprehensive accuracy improvement factor; S60 dynamically aggregates the dynamically adjusted comprehensive accuracy improvement factor and the locally trained model parameters for all clients to obtain the updated global model parameters and distributes them to the multi-dimensional participants for local training. Training stops when the training stopping condition is met.
[0089] The model aggregation optimization method disclosed in this application directly involves evaluation factors in the aggregation process, resulting in a significantly faster convergence speed and higher accuracy compared to the traditional Fed Avg method. The specific implementation methods for S10-S60 are consistent with the implementation methods for evaluating the contributions of multi-dimensional participants disclosed in the first aspect of this application, and therefore will not be elaborated upon here.
[0090] The basic principles of this disclosure have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in this disclosure are merely examples and not limitations, and should not be considered as essential features of each embodiment of this disclosure. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the scope of this disclosure to the necessity of employing the aforementioned specific details for implementation.
[0091] In this disclosure, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. The block diagrams of devices, apparatuses, devices, and systems involved in this disclosure are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as "comprising," "including," "having," etc., are open-ended terms meaning "including but not limited to," and are used interchangeably with them. The terms "or" and "and" as used herein refer to the terms "and / or," and are used interchangeably with them unless the context clearly indicates otherwise. The term "such as" as used herein refers to the phrase "such as but not limited to," and is used interchangeably with it.
[0092] Additionally, as used herein, the "or" used in a list of items beginning with "at least one" indicates a separate list, such that a list of, for example, "at least one of A, B, or C" means A or B or C, or AB or AC or BC, or ABC (i.e., A and B and C). Furthermore, the word "exemplary" does not imply that the described example is preferred or better than other examples.
[0093] It should also be noted that in the systems and methods of this disclosure, the components or steps can be decomposed and / or recombined. These decompositions and / or recombinations should be considered as equivalent solutions to this disclosure.
[0094] Various changes, substitutions, and modifications can be made to the technology described herein without departing from the teachings defined by the appended claims. Furthermore, the scope of the claims of this disclosure is not limited to the specific aspects of the processes, machines, manufactures, events, means, methods, and actions described above. Currently existing or later-developed processes, machines, manufactures, events, means, methods, or actions that perform substantially the same function or achieve substantially the same result as the corresponding aspects described herein can be utilized. Therefore, the appended claims include such processes, machines, manufactures, events, means, methods, or actions within their scope.
[0095] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use this disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of this disclosure. Therefore, this disclosure is not intended to be limited to the aspects shown herein, but rather to be carried out within the widest scope consistent with the principles and novel features disclosed herein.
[0096] The above description has been given for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of this disclosure to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations therein.
Claims
1. A method for evaluating multi-dimensional participant contribution, characterized in that, include: Initialize the global model and distribute it to the multi-dimensional participants; Each participant performs iterative training of the global model based on its corresponding local data, and obtains local model information and uploads it to the server after each iteration. The overall accuracy improvement factor for each client is determined based on the local model information. Obtain the model accuracy change rate corresponding to the comprehensive accuracy improvement factor for each client, and dynamically adjust the magnitude dimension weight, direction dimension weight, and uniqueness dimension weight based on the model accuracy change rate. Based on the dynamically adjusted magnitude dimension weights, direction dimension weights, and uniqueness dimension weights, the dynamically adjusted comprehensive accuracy improvement factor is obtained. The dynamically adjusted comprehensive accuracy improvement factor and the locally trained model parameters corresponding to all clients are dynamically aggregated to obtain the updated global model parameters, which are then distributed to the multi-dimensional participants for local training. The dynamic adjustment of the overall accuracy improvement factor for the corresponding round when the training stopping condition is met is obtained and denoted as the target factor. The model contribution information for each participant is determined based on the target factor and the historical cumulative contribution of decay.
2. The method of claim 1, wherein, The local model information includes the model parameter update amount and individual gradient for each client; The step of determining the overall accuracy improvement factor for each client based on the local model information includes: The information contribution capability of the corresponding client is obtained based on the update amount of the model parameters. Based on the individual gradients, obtain the direction consistency information between each local gradient and the global gradient; Choose any client as the target party, obtain the average gradient of all other participants, and determine the unique capability of each client based on the average gradient and the individual gradient of the target party. Based on the information contribution capability, directional consistency information, uniqueness capability, magnitude dimension weight, directional dimension weight, and uniqueness dimension weight corresponding to each client, the comprehensive accuracy improvement factor for each client is determined.
3. The method for evaluating the contribution of multi-dimensional participants according to claim 2, characterized in that, The step of obtaining the information contribution capability of the corresponding client based on the model parameter update includes: Obtain the L2 norm of the corresponding client based on the model parameter update amount; Obtain the sum of the norms of all the aforementioned participants; The information contribution capability of each client is determined based on the L2 norm and the sum of the norms.
4. The method for evaluating the contribution of multi-dimensional participants according to claim 3, characterized in that, The step of obtaining the direction consistency information between each local gradient and the global gradient based on the individual gradient includes: The weighted global gradient is obtained by summing all the individual gradients using weighted methods. Based on the individual gradient and the weighted global gradient, determine the degree of directional overlap corresponding to each local gradient; The L2 norms of the individual gradient and the weighted global gradient are obtained respectively, and the similarity information between each local gradient and the global gradient is obtained based on the degree of directional overlap. The similarity information is normalized to obtain directional consistency information.
5. The method for evaluating the contribution of multi-dimensional participants according to claim 2, characterized in that, The step of obtaining the model accuracy change rate corresponding to the comprehensive accuracy improvement factor for each client, and dynamically adjusting the magnitude dimension weight, the direction dimension weight, and the uniqueness dimension weight based on the model accuracy change rate, includes: Based on the comprehensive accuracy improvement factor calculated in each round, the rate of change corresponding to each client is obtained; The dynamic weight adjustment strategy corresponding to the rate of change is obtained according to a preset threshold, and the adjusted magnitude dimension weight, direction dimension weight, and uniqueness dimension weight are determined based on the dynamic weight adjustment strategy. Based on the information contribution capability, directional consistency information, uniqueness capability, adjusted magnitude dimension weight, adjusted directional dimension weight, and adjusted uniqueness dimension weight corresponding to each client, the comprehensive accuracy improvement factor for each client is determined.
6. The method for evaluating the contribution of multi-dimensional participants according to claim 5, characterized in that, The step of obtaining the weight dynamic adjustment strategy corresponding to the rate of change according to a preset threshold, and determining the adjusted magnitude dimension weight, direction dimension weight, and uniqueness dimension weight based on the weight dynamic adjustment strategy, includes: If the rate of change is less than a preset threshold, the first weight dynamic adjustment strategy is invoked, and the current directional dimension weight and uniqueness dimension weight are both adjusted upward according to the first weight dynamic adjustment strategy, while the current amplitude dimension weight is adjusted downward. If the rate of change is greater than a preset threshold, the second weight dynamic adjustment strategy is invoked, and the current magnitude dimension weight is adjusted upward according to the second weight dynamic adjustment strategy, while the current direction dimension weight and uniqueness dimension weight are both adjusted downward. The adjustment ratios of the directional dimension weight and the uniqueness dimension weight are the same, and the sum of the adjusted amplitude dimension weight, directional dimension weight, and uniqueness dimension weight is 1.
7. The method for evaluating the contribution of multi-dimensional participants according to claim 6, characterized in that, The adjustment ratio of the directional dimension weight is 10% to 20% of the current corresponding directional dimension weight; The adjustment ratio of the amplitude dimension weight is 50% to 60% of the current corresponding amplitude dimension weight.
8. The method for evaluating the contribution of multi-dimensional participants according to claim 5, characterized in that, The step of determining the model contribution information for each participant based on the target factor and historical cumulative contribution includes: The cumulative contribution of each participant in the current round is determined based on the target factor and the historical decay cumulative contribution. Obtain the percentage of each participant's cumulative contribution in the current round; Based on the preset total revenue and the aforementioned percentage, the model contribution information corresponding to each participant is determined.
9. A system for evaluating the contributions of multi-dimensional participants, characterized in that, include: An initialization unit is used to initialize the global model and distribute the global model to the multi-dimensional participants; The training unit is used by each participant to perform iterative training of the global model based on the corresponding local data, and to obtain local model information and upload it to the server after each iteration. The comprehensive accuracy improvement factor acquisition unit is used to determine the comprehensive accuracy improvement factor for each client based on the local model information. The weight dynamic acquisition unit is used to acquire the model accuracy change rate corresponding to the comprehensive accuracy improvement factor for each client, and dynamically adjust the magnitude dimension weight, direction dimension weight, and uniqueness dimension weight according to the model accuracy change rate. The factor dynamic update unit is used to obtain the dynamically adjusted comprehensive accuracy improvement factor based on the dynamically adjusted magnitude dimension weight, the direction dimension weight, and the uniqueness dimension weight. The dynamic aggregation unit is used to dynamically aggregate the dynamically adjusted comprehensive accuracy improvement factor and the locally trained model parameters corresponding to all clients, obtain the updated global model parameters, and distribute them to the multi-dimensional participants for local training. The target factor acquisition unit is used to obtain the dynamically adjusted comprehensive accuracy improvement factor for the corresponding round when the training stopping condition is met, denoted as the target factor; The contribution information analysis unit is used to determine the model contribution information of each participant based on the target factor and the historical decay cumulative contribution.
10. A model aggregation optimization method, characterized in that, include: Initialize the global model and distribute it to the multi-dimensional participants; Each participant performs iterative training of the global model based on its corresponding local data, and obtains local model information and uploads it to the server after each iteration. The overall accuracy improvement factor for each client is determined based on the local model information. Obtain the model accuracy change rate corresponding to the comprehensive accuracy improvement factor for each client, and dynamically adjust the magnitude dimension weight, direction dimension weight, and uniqueness dimension weight based on the model accuracy change rate. Based on the dynamically adjusted magnitude dimension weights, direction dimension weights, and uniqueness dimension weights, the dynamically adjusted comprehensive accuracy improvement factor is obtained. The dynamically adjusted comprehensive accuracy improvement factor and the locally trained model parameters corresponding to all clients are dynamically aggregated to obtain the updated global model parameters, which are then distributed to the multi-dimensional participants for local training. Training is stopped when the training stopping condition is met.