Federated learning contribution assessment methods and devices

Abstract

This application provides a federated learning contribution evaluation method and device. Multiple edge computing devices are grouped to obtain multiple sub-federated learning sets. For a target sub-federated learning set, model update information corresponding to each edge computing device in the target sub-federated learning set is aggregated. Based on the performance metrics of the updated global model on a public test set, the collaborative contribution value of the target sub-federated learning set is determined. Using a first linear programming solver, the maximum loss value corresponding to all sub-federated learning sets is obtained based on the collaborative contribution value of the multiple sub-federated learning sets, and the maximum loss value is optimized to obtain the minimized maximum loss value. Using a second linear programming solver, based on the minimized maximum loss value of all sub-federated learning sets and the reference contribution value corresponding to the multiple edge computing devices, the target contribution vector corresponding to each of the multiple edge computing devices is determined, accurately quantifying the contribution of each edge computing device in the training process.

Description

Technical Field

[0001] This application relates to the field of federal learning technology, and more particularly to a method and apparatus for evaluating federal learning contributions. Background Technology

[0002] With the development of edge computing and privacy-preserving computing technologies, federated learning, as a distributed machine learning paradigm, is widely used in data privacy-sensitive fields such as healthcare, finance, and the Internet of Things. In distributed training systems, the federated learning framework can be used to achieve collaborative training of the global model. Multiple edge computing devices train the model locally and upload the model updates to the federated server for aggregation, thereby completing the collaborative training of the global model without sharing the original data.

[0003] However, in actual deployment, there are significant differences in data quality, computing power, and participation frequency among different edge computing devices. Some edge computing devices may upload low-quality or malicious model updates, affecting the convergence speed and final accuracy of the global model. In addition, the lack of a reasonable evaluation mechanism for the contributions of edge computing devices may lead to uneven resource allocation, low training efficiency, and even "free-riding" problems, weakening the overall performance of the system.

[0004] Traditional federated learning contribution evaluation schemes often use the Shapley value to assess the contribution of each participant. This is done by calculating each participant's marginal contribution to the consortium and then weighting the average to determine their share of the benefits in the collaboration. However, this scheme relies on the premise that each participant's local training data is stable. In federated learning, the data distribution of each participant is often dynamic. Therefore, if the data distribution changes during actual training, the original calculation results will no longer accurately reflect the actual contribution of each participant, leading to a mismatch between contribution and benefit. This can cause participants to withdraw from the collaboration, affecting not only the model training effect but also the stability of the distributed training system. Summary of the Invention

[0005] This application provides a federated learning contribution evaluation method and device, which can not only accurately quantify the contribution of each edge computing device in the global model training process, but also ensure the stability of the distributed training system.

[0006] In a first aspect, embodiments of this application provide a federated learning contribution evaluation method, applied to a federated server in a distributed training system, wherein the distributed training system further includes multiple edge computing devices, the multiple edge computing devices collaboratively training a global model, and the method includes:

[0007] Receive local model update information uploaded by the multiple edge computing devices;

[0008] The multiple edge computing devices are grouped to obtain multiple sub-federated learning sets;

[0009] For the target sub-federated learning set, the local model update information corresponding to each edge computing device in the target sub-federated learning set is aggregated to generate an updated first global model, and the collaborative contribution value corresponding to the target sub-federated learning set is determined based on the performance index of the first global model on the public test set; the target sub-federated learning set is any one of the multiple sub-federated learning sets;

[0010] Using the first linear programming solver, based on the collaborative contribution values ​​of each of the multiple sub-federated learning sets, the maximum loss value corresponding to all sub-federated learning sets is obtained, and the maximum loss value is optimized to obtain the minimized maximum loss value.

[0011] Using the second linear programming solver, based on the minimized maximum loss value corresponding to all sub-federated learning sets, the error between the contribution allocation of the multiple edge computing devices and the reference contribution value corresponding to the multiple edge computing devices is minimized, so as to determine the target contribution vector corresponding to each of the multiple edge computing devices, and the target contribution vector corresponding to the multiple edge computing devices is sent to their respective edge computing devices.

[0012] Wherein, the collaborative contribution value is used to characterize the model performance level achieved by the target sub-federated learning set through joint training of the global model; the maximum loss value is used to characterize the difference between the collaborative contribution value of the sub-federated learning set and the sum of the contribution values ​​obtained by the edge computing devices it contains under the current global contribution allocation scheme; the reference contribution value is used to characterize the performance level achieved by the edge computing devices training the global model independently.

[0013] Secondly, embodiments of this application provide a federated learning contribution evaluation device, applied to a federated server in a distributed training system. The distributed training system further includes multiple edge computing devices that collaboratively train a global model. The device includes:

[0014] A receiving module is used to receive local model update information uploaded by the multiple edge computing devices;

[0015] The grouping module is used to group the multiple edge computing devices to obtain multiple sub-federated learning sets;

[0016] An aggregation module is used to aggregate local model update information corresponding to each edge computing device in the target sub-federated learning set to generate an updated first global model, and determine the collaborative contribution value corresponding to the target sub-federated learning set based on the performance index of the first global model on a public test set; the target sub-federated learning set is any one of the multiple sub-federated learning sets.

[0017] The optimization module is used to obtain the maximum loss value corresponding to all sub-federated learning sets based on the collaborative contribution value of each of the multiple sub-federated learning sets through the first linear programming solver, and optimize the maximum loss value to obtain the minimized maximum loss value.

[0018] The determination module is used to minimize the error between the contribution allocation of the multiple edge computing devices and the reference contribution value of the multiple edge computing devices based on the minimized maximum loss value corresponding to all sub-federated learning sets through the second linear programming solver, so as to determine the target contribution vector corresponding to each of the multiple edge computing devices, and send the target contribution vector corresponding to the multiple edge computing devices to their respective edge computing devices.

[0019] Wherein, the collaborative contribution value is used to characterize the model performance level achieved by the target sub-federated learning set through joint training of the global model; the maximum loss value is used to characterize the difference between the collaborative contribution value of the sub-federated learning set and the sum of the contribution values ​​obtained by the edge computing devices it contains under the current global contribution allocation scheme; the reference contribution value is used to characterize the performance level achieved by the edge computing devices training the global model independently.

[0020] Thirdly, embodiments of this application provide an electronic device applied to a sender, comprising: a memory, a processor, and a communication interface; wherein, the memory stores executable code, and when the executable code is executed by the processor, the processor can at least implement the federated learning contribution evaluation method as described in the first aspect.

[0021] Fourthly, embodiments of this application provide a non-transitory machine-readable storage medium storing executable code, which, when executed by a processor of an electronic device, enables the processor to at least implement the federated learning contribution evaluation method as described in the first aspect.

[0022] Fifthly, embodiments of this application provide a computer program product, including: a computer program or instructions that, when executed by a processor of an electronic device, enable the processor to at least implement the federated learning contribution evaluation method as described in the first aspect.

[0023] In the federated learning contribution evaluation scheme provided in this application embodiment, the scheme is applied to a federated server in a distributed training system. This distributed training also includes multiple edge computing devices. These edge computing devices independently train the model using local data, generating local training results which are then uploaded to the federated server. The federated server aggregates the training results from all edge computing devices, generates global model parameters, and distributes them to each edge computing device to iteratively and collaboratively complete the training of the global model. After each edge computing device has trained for multiple rounds, it uploads its local training results to the federated server. The federated server can accurately evaluate the contribution value of each edge computing device based on its local training results and provide feedback to each edge computing device. This prevents malicious interference or partial withdrawal of edge computing devices during training, thereby ensuring the stability of the distributed training system.

[0024] Specifically, firstly, local model update information uploaded by multiple edge computing devices is received to trigger a contribution evaluation of these devices. Next, the edge computing devices are grouped to obtain multiple sub-federated learning sets. Then, the local model update information corresponding to each edge computing device in each sub-federated learning set is aggregated to generate an updated first global model for each sub-federated learning set. Based on the performance metrics of the first global model on a public test set, the collaborative contribution value for each sub-federated learning set is determined. This collaborative contribution value characterizes the model performance level achieved by each sub-federated learning set through joint training of the global model. Next, using a first linear programming solver, the maximum loss value corresponding to all sub-federated learning sets is obtained based on their respective collaborative contribution values. This maximum loss value is then optimized to obtain a minimized maximum loss value. This maximum loss value characterizes the difference between the collaborative contribution value of a sub-federated learning set and the sum of the contribution values ​​obtained by its constituent edge computing devices under the current global contribution allocation scheme. Then, using a second linear programming solver, based on the minimized maximum loss value corresponding to all sub-federated learning sets, the error between the contribution allocation of multiple edge computing devices and the reference contribution value corresponding to multiple edge computing devices is minimized to determine the target contribution vector corresponding to each of the multiple edge computing devices. These target contribution vectors are then sent to their respective edge computing devices. The reference contribution value is used to characterize the performance level achieved by an edge computing device training the global model independently.

[0025] In the above scheme, a first linear programming solver obtains the maximum loss value corresponding to all sub-federated learning sets based on their respective collaborative contribution values, and optimizes the maximum loss value to obtain the minimized maximum loss value. Then, a second linear programming solver minimizes the error between the contribution allocation of multiple edge computing devices and their corresponding reference contribution values ​​based on the minimized maximum loss value for all sub-federated learning sets, thereby determining the target contribution vector for each edge computing device. In other words, the first linear programming solver, combined with the collaborative contribution values ​​of multiple sub-federated learning sets, solves and optimizes the maximum loss value of all sub-federated learning sets, achieving... To minimize the maximum loss value at each sub-federated learning set level, a precise and reasonable constraint benchmark is provided for subsequent optimization of the contribution allocation of edge computing devices. At the same time, a second linear programming solver is used to optimize the error between the contribution allocation of multiple edge computing devices and the reference contribution value based on the minimized maximum loss value. This achieves the accurate determination of the target contribution vector of each edge computing device under the premise of meeting the overall loss control requirements of each sub-federated learning set, so as to accurately quantify the contribution of each edge computing device in the global model training process. This not only ensures the rationality and accuracy of the contribution allocation of each edge computing device, but also improves the collaborative efficiency and operational stability of the entire distributed training system. Attached Figure Description

[0026] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0027] Figure 1 A schematic diagram of a federated learning contribution evaluation system provided in this application embodiment;

[0028] Figure 2 A flowchart illustrating a method for evaluating federated learning contributions, provided as an embodiment of this application;

[0029] Figure 3 A flowchart illustrating another method for evaluating federated learning contributions provided in this application embodiment;

[0030] Figure 4 A schematic diagram illustrating the application of a federated learning contribution evaluation method provided in this application embodiment;

[0031] Figure 5 A schematic diagram of a federated learning contribution evaluation device provided in an embodiment of this application;

[0032] Figure 6To and Figure 5 The illustrated embodiment provides a schematic diagram of the electronic device corresponding to the federated learning contribution evaluation device. Detailed Implementation

[0033] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0034] The terminology used in the embodiments of this application is for the purpose of describing particular embodiments only and is not intended to limit the application. The singular forms “a,” “the,” and “the” used in the embodiments of this application and the appended claims are also intended to include the plural forms, unless the context clearly indicates otherwise. “Multiple” generally includes at least two, but does not exclude the inclusion of at least one. It should be understood that the term “and / or” as used herein is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, and B alone. Additionally, the character “ / ” in this document generally indicates that the preceding and following related objects are in an “or” relationship. Depending on the context, the words “if” or “when” as used herein can be interpreted as “when…” or “when…”.

[0035] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a product or system comprising a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a product or system. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the product or system that includes said element.

[0036] Furthermore, the timing of the steps in the following method embodiments is merely an example and not a strict limitation.

[0037] Traditional federated learning contribution evaluation schemes often use the Shapley value to assess the contribution of each participant. This is done by calculating each participant's marginal contribution to the consortium and then weighting the average to determine their share of the benefits in the collaboration. However, this scheme relies on the premise that each participant's local training data is stable. In federated learning, the data distribution of each participant is often dynamic. Therefore, if the data distribution changes during actual training, the original calculation results will no longer accurately reflect the actual contribution of each participant, leading to a mismatch between contribution and benefit. This can cause participants to withdraw from the collaboration, affecting not only the model training effect but also the stability of the distributed training system.

[0038] To address the aforementioned technical issues, this application proposes a novel federated learning contribution evaluation scheme. This scheme primarily constructs a two-layer linear programming optimization framework: "sub-federated learning set loss optimization and edge device contribution value calibration." First, the first-stage linear programming focuses on overall loss control at the sub-federated level, resolving the issues of uneven loss among sub-federated learning sets and high overall risk during federated learning collaboration. Then, the second-stage linear programming correlates the optimization results of the first stage, accurately calibrating the contribution allocation of each edge computing device. This addresses the issue of excessive deviation between the actual contribution allocation and the reference contribution value of each edge computing device. This approach accurately quantifies the contribution of each edge computing device in the global model training process while ensuring the overall performance stability of the distributed training system and maintaining fairness in the contribution evaluation of each edge computing device, thereby incentivizing continuous participation in federated learning and the provision of high-quality samples.

[0039] The following embodiments illustrate the federated learning contribution evaluation scheme.

[0040] Figure 1 A schematic diagram of a federated learning contribution evaluation system provided in this application embodiment is shown below. Figure 1 As shown, the contribution evaluation system includes a security value assessment module, a two-stage minimum kernel solution module, a contribution distribution module, an auxiliary module, and third-party security hosting. Furthermore, this contribution evaluation system can primarily be applied to federated servers in distributed training systems to quantify the contributions of multiple edge computing devices in the joint training of the global model. This not only improves the efficiency and quality of global model training but also ensures the stability of the distributed training system.

[0041] The federated server acts as the core coordinator, responsible for the global control of multiple edge computing devices in the distributed training system and aggregating model update information uploaded by these devices after local training is completed. Furthermore, the federated server communicates with each edge computing device via encrypted channels.

[0042] Multiple edge computing devices can act as data holders, securely storing their raw data locally and performing operations such as local training of the global model, gradient calculation, and response to contribution evaluation requests, while their corresponding raw data never leaves their local area.

[0043] Alternatively, a two-stage minimum kernel solver module and a security value assessment module can be integrated into the federated server to directly execute the execution logic corresponding to the two-stage minimum kernel solver module and the security value assessment module through the federated server.

[0044] The security value assessment module is primarily used to evaluate the model performance achievable when the various sub-federated learning sets jointly train the global model. Optionally, this security value assessment module can be deployed on a federated server or on a separate, trusted third-party secure hosting facility.

[0045] The two-stage minimum kernel solver module is primarily responsible for optimizing the loss of each sub-federated learning set and calibrating the contribution values ​​of each edge device. Furthermore, this two-stage minimum kernel solver module can internally include a first-stage minimum loss solver and a second-stage deviation minimization solver, both of which can be linear programming solvers. For ease of description, the following description will use the first and second linear programming solvers.

[0046] The first linear programming solver is primarily responsible for solving the standard minimum kernel linear programming problem to obtain the minimized maximum loss value corresponding to all current sub-federated learning sets. The second linear programming solver is primarily responsible for receiving the minimized maximum loss value determined by the first linear programming solver and the reference contribution value of each edge computing device. Under the premise of satisfying the minimum loss constraint, it solves a secondary linear programming problem to minimize the deviation between the minimized contribution allocation of each edge computing device and the reference contribution value, thereby obtaining the unique contribution allocation corresponding to each edge device. Optionally, this two-stage minimum kernel solving module can be deployed on the federated server.

[0047] The contribution distribution module is mainly responsible for sending the target contribution vectors corresponding to each edge computing device to the corresponding edge computing device. It can also distribute corresponding incentives to each edge computing device according to the target contribution vectors corresponding to each edge computing device.

[0048] The auxiliary module is mainly used for incentive regulation and feedback on the behavior of each edge computing device to work in conjunction with other modules in the contribution evaluation system to form a complete incentive closed loop. Additionally, the third-party secure hosting is mainly responsible for storing a public test set used to evaluate the model performance achievable through joint training of each sub-federated learning set, and for evaluating the model performance achievable by each sub-federated learning set.

[0049] In practical applications, when quantifying the contributions of multiple edge computing devices in a distributed training system to the joint training of a global model, this contribution evaluation system first sends contribution evaluation requests to these devices. Upon receiving these requests, each edge computing device sends its corresponding local model update information to the contribution evaluation system. This local model update information may include the local model's weight parameters, model gradients, or model increments.

[0050] Then, local model update information uploaded by multiple edge computing devices is received to quantify the contribution of the joint training of the global model by the multiple edge computing devices. Specifically, the multiple edge computing devices are grouped to obtain multiple sub-federated learning sets. The collaborative contribution value of each sub-federated learning set is then determined when each edge computing device in each sub-federated learning set jointly trains the global model. The collaborative contribution value characterizes the model performance level achieved by the sub-federated learning sets through joint training of the global model. The process of determining the collaborative contribution value for each sub-federated learning set is roughly the same; the following explanation uses any target sub-federated learning set as an example.

[0051] For the target sub-federated learning set, the model update information corresponding to each edge computing device in the target sub-federated learning set is aggregated to generate an updated first global model. Based on the performance index of the first global model on the public relations test set, the collaborative contribution value corresponding to the target sub-federated learning set is determined.

[0052] Next, using the first linear programming solver, based on the collaborative contribution values ​​of each of the multiple sub-federated learning sets, the maximum loss value corresponding to all sub-federated learning sets is obtained, and the maximum loss value is optimized to obtain the minimized maximum loss value. The maximum loss value characterizes the difference between the collaborative contribution value of a sub-federated learning set and the sum of the contribution values ​​obtained by its constituent edge computing devices under the current global contribution allocation scheme.

[0053] Then, using a second linear programming solver, based on the minimized maximum loss value corresponding to all sub-federated learning sets, the error between the contribution allocation of multiple edge computing devices and the reference contribution value corresponding to each edge computing device is minimized to determine the target contribution vector for each edge computing device. These target contribution vectors are then sent to their respective edge computing devices. This allows the edge computing devices to adjust their training data in subsequent local training processes based on their corresponding target contribution vectors, thus actively participating in the training of the global model. The reference contribution value is used to characterize the performance level achieved by an edge computing device training the global model independently.

[0054] As described above, by applying minimum kernel optimization from cooperative game theory to federated learning contribution evaluation, we first solve for the maximum loss value corresponding to all possible sub-federated learning sets, and minimize this maximum loss value to obtain the minimized maximum loss value and the stable contribution allocation interval corresponding to each edge computing device. Then, we select the target allocation vector with the smallest deviation from the benchmark contribution of each edge computing device, and generate a unique contribution allocation scheme that optimizes stability and fairness. This not only accurately quantifies the contribution of each edge computing device in the global model training process, but also ensures the overall performance stability of the distributed training system, while taking into account the fairness of the contribution evaluation of each edge computing device.

[0055] To facilitate understanding of the specific process of the above contribution evaluation system, the following examples illustrate the specific process of federated learning contribution evaluation.

[0056] Figure 2 A flowchart of a federated learning contribution evaluation method provided for embodiments of this application is shown below. Figure 2 As shown, this method is applied to a federated server in a distributed training system. This distributed training system also includes multiple edge computing devices, which collaboratively train a global model. Specifically, the method may include the following steps:

[0057] 201. Receive local model update information uploaded by multiple edge computing devices.

[0058] 202. Group multiple edge computing devices to obtain multiple sub-federated learning sets.

[0059] 203. For the target sub-federated learning set, aggregate the local model update information corresponding to each edge computing device in the target sub-federated learning set to generate an updated first global model. Based on the performance metrics of the first global model on the public test set, determine the collaborative contribution value corresponding to the target sub-federated learning set. The target sub-federated learning set can be any one of multiple sub-federated learning sets.

[0060] 204. Using the first linear programming solver, based on the collaborative contribution values ​​of each of the multiple sub-federated learning sets, obtain the maximum loss value corresponding to all sub-federated learning sets and optimize the maximum loss value to obtain the minimized maximum loss value.

[0061] 205. Using the second linear programming solver, based on the minimized maximum loss value corresponding to all sub-federated learning sets, minimize the error between the contribution allocation of multiple edge computing devices and the reference contribution value corresponding to multiple edge computing devices, so as to determine the target contribution vector corresponding to each of the multiple edge computing devices, and send the target contribution vector corresponding to each of the multiple edge computing devices to their respective edge computing devices.

[0062] The federated learning contribution evaluation scheme provided in this application can be applied to various distributed training systems. Regardless of whether the federated learning scenario in the distributed training system is horizontal or vertical, this scheme can be used to evaluate the contribution of each edge computing device in the distributed training system to jointly train the global model.

[0063] The local model update information includes the weight parameters, model gradients, and model increments of the local models corresponding to each edge computing device. Upon receiving local model update information from multiple edge computing devices, the devices are grouped to determine the possible sub-federated learning sets. These sub-federated learning sets can be empty, contain a subset of edge computing devices in a joint training combination, or contain all edge computing devices in a joint training combination. In other words, all possible combinations of edge computing devices are enumerated.

[0064] For example, suppose three hospitals participate in federated learning to build a hearing prediction model. That is, suppose the current distributed training system includes edge computing devices A, B, and C, deployed in these three hospitals respectively. Grouping these three edge computing devices yields eight sub-federated learning sets. Sub-federated learning set 1 includes only edge computing device A; sub-federated learning set 2 includes only edge computing device B; sub-federated learning set 3 includes only edge computing device C; sub-federated learning set 4 includes edge computing devices A and B; sub-federated learning set 5 includes edge computing devices A and C; sub-federated learning set 6 includes edge computing devices B and C; sub-federated learning set 7 includes edge computing devices A, B, and C; and sub-federated learning set 8 does not contain any edge computing devices. Since any sub-federated learning set may detach from the entire distributed training system and form its own combination for independent training, this will affect the overall stability of the distributed training system. When quantifying the contribution of multiple edge computing devices in jointly training the global model, we can first determine the collaborative contribution value of each of the multiple sub-federated learning sets when they are separated from the distributed training system and each edge computing device within the sub-federated learning sets independently trains the global model. The collaborative contribution value characterizes the model performance level achieved by the sub-federated learning sets through joint training of the global model, i.e., the value created by the sub-federated learning sets when jointly training the global model. For an empty set, if the sub-federated learning set is empty, the collaborative contribution value corresponding to that sub-federated learning set can be defined as a randomly guessed baseline performance. The process of determining the collaborative contribution value for each sub-federated learning set containing edge computing devices is roughly the same. The following explanation uses any target sub-federated learning set from multiple sub-federated learning sets as an example.

[0065] For the target sub-federated learning set, the local model update information corresponding to each edge computing device in the target sub-federated learning set is aggregated to generate the updated first global model. Specifically, when the local model update information is the model gradient, one possible implementation for aggregating the local model update information corresponding to each edge computing device in the target sub-federated learning set to generate the updated first global model is as follows: Obtain the amount of local training data for each edge computing device in the target sub-federated learning set; determine the second aggregation weights corresponding to each edge computing device in the target sub-federated learning set based on the proportion of the local training data amount of each edge computing device in the target sub-federated learning set to the total training data amount of the target sub-federated learning set; perform a weighted average of the model gradients corresponding to each edge computing device in the target sub-federated learning set according to the second aggregation weights to obtain the second global gradient; apply the second global gradient to the parameter update of the current global model to generate the updated first global model.

[0066] In addition, when determining the second aggregation weights corresponding to each edge computing device in the target sub-federated learning set, the second aggregation weights can be determined not only based on the amount of local training data of each edge computing device in the target sub-federated learning set, but also by combining information such as the historical contribution vectors of each edge computing device in the target sub-federated learning set.

[0067] One possible implementation for generating the updated first global model when the local model update information is model parameters is as follows: Obtain the local training data volume of each edge computing device in the target sub-federated learning set; determine the second aggregation weight corresponding to each edge computing device in the target sub-federated learning set based on the proportion of the local training data volume of each edge computing device in the target sub-federated learning set to the total training data volume of the target sub-federated learning set; perform a weighted average of the model parameters corresponding to each edge computing device in the target sub-federated learning set according to the second aggregation weight, to obtain the global model parameters; update the parameters of the current global model to these global model parameters, and generate the updated first global model.

[0068] Then, based on the performance metrics of the first global model on the public test set, the collaborative contribution value corresponding to the target sub-federated learning set is determined to simulate the model performance achieved when the global model is jointly trained by various edge computing devices within the target sub-federated learning set. The performance metrics include task performance metrics and test uncertainty metrics. Task performance metrics can be models such as accuracy, for example, AUC. Test uncertainty metrics can be the loss value corresponding to the first global model, for example, negative cross-entropy loss. Additionally, performance metrics can also include data statistics metrics, model similarity, etc., all of which can be used to evaluate the collaborative contribution value corresponding to the target sub-federated learning set and can also serve as the basis for dynamically adjusting the preset weights corresponding to task performance metrics and test uncertainty metrics.

[0069] In one optional embodiment, the specific implementation process of simulating the model performance achieved by each edge computing device in the target sub-federated learning set when jointly training the global model can be as follows: inputting a common test set into the updated first global model to determine the prediction result corresponding to the common test set; determining the task performance index corresponding to the target sub-federated learning set based on the prediction result and the reference result corresponding to the common test set; determining the test uncertainty index corresponding to the target sub-federated learning set; and determining the collaborative contribution value corresponding to the target sub-federated learning set based on the task performance index and the test uncertainty index.

[0070] In determining the collaborative contribution value of the target sub-federated learning set based on task performance indicators and test uncertainty indicators, the collaborative contribution value of the target sub-federated learning set can also be determined by combining the preset weights corresponding to the task performance indicators and the preset weights corresponding to the test uncertainty indicators.

[0071] Specifically, the first product of the preset weights corresponding to the task performance indicators and the task performance indicators is obtained, and the second product of the preset weights corresponding to the test uncertainty indicators and the test uncertainty indicators is obtained. The sum of the first product and the second product is determined, and this sum is determined as the collaborative contribution value corresponding to the target sub-federated learning set.

[0072] For example, suppose the task performance metric is Vperf, the test uncertainty metric is Vuncert, and the preset weights for the task performance metric are Wperf and the preset weights for the test uncertainty metric are Wuncert. The collaborative contribution value corresponding to the target sub-federated learning set is the weighted sum of these metrics: that is, the collaborative contribution value V(S) = Wperf⋅ Vperf(S) + Wuncert⋅ Vuncert(S).

[0073] Next, using the first linear programming solver, based on the collaborative contribution values ​​of each of the multiple sub-federated learning sets, the maximum loss value corresponding to all sub-federated learning sets is obtained, and the maximum loss value is optimized to obtain the minimized maximum loss value. The maximum loss value characterizes the difference between the collaborative contribution value of a sub-federated learning set and the sum of the contribution values ​​obtained by its constituent edge computing devices under the current global contribution allocation scheme.

[0074] The loss value refers to the difference between the maximum gain that a sub-federated learning set could obtain by acting alone and the actual gain that the sub-federated learning set obtains from the overall contribution allocation corresponding to the joint training of the global model by all current edge computing devices. If this difference, i.e., the loss value, is positive, it indicates that the sub-federated learning set has incurred a loss due to participating in the overall cooperation (i.e., the cooperation gain is lower than the gain from acting alone). In this case, there is a high probability that each edge computing device within the sub-federated learning set will detach from the distributed training system and train the global model independently, thereby threatening the stability of the distributed training system.

[0075] To ensure the stability of the distributed training system, a minimum kernel can be introduced. By optimizing the contribution allocation scheme, the loss of the most unprofitable sub-federated learning set in all sub-federated learning sets can be minimized, thereby ensuring the stability of the overall cooperation and preventing the sub-federated learning set from leaving the cooperation due to excessive loss.

[0076] In cooperative game theory, the ideal core solution requires that all sub-federations have no losses (i.e., the cooperative distribution of benefits is greater than or equal to the individual action benefits). However, in many scenarios, the core solution does not exist (e.g., some sub-alliances will inevitably incur losses). In this case, the minimum core solution becomes the approximately stable optimal solution.

[0077] In practice, we can first define the maximum possible loss of each sub-federated learning set. For each sub-federated learning set, we calculate the difference between its individual action gain and its allocated gain, and take the maximum value among all differences (i.e., the loss of the most unprofitable sub-federated learning set). Then, we optimize this maximum loss value to minimize it, and determine the minimized maximum loss value.

[0078] In other words, by introducing the core idea of ​​the minimum kernel and optimizing the contribution allocation scheme, the loss of the most unprofitable sub-federated learning set is compressed to the minimum. This process is essentially about finding the minimum value corresponding to the maximum loss value in all sub-federated learning sets under the premise of satisfying the overall benefit constraint (such as total allocation benefit = overall cooperation maximum benefit), so as to ensure that the contribution allocation can suppress the motivation of any sub-federated learning set to leave the distributed training system to the greatest extent.

[0079] Simultaneously, it is also possible to determine the contribution value of each edge computing device under the minimized maximum loss value. That is, to determine the stable solution (the contribution value of each edge computing device) corresponding to the minimized maximum loss value.

[0080] However, due to the core defect that the minimum kernel solution is not unique, the incentive allocation becomes unclear and unreliable. Therefore, in practical applications, in order to solve the core defect that the minimum kernel solution is not unique, when the maximum loss value after minimization is determined, that is, among all contribution allocations that satisfy the maximum loss value after minimization, the contribution vector that is unique and closest to the reference contribution value of each edge computing device can be selected. Thus, the deviation minimization criterion is introduced as a secondary optimization objective.

[0081] In practice, the second linear programming solver minimizes the error between the contribution allocation of multiple edge computing devices and the reference contribution value of multiple edge computing devices based on the minimized maximum loss value corresponding to all sub-federated learning sets, thereby determining the target contribution vector corresponding to each of the multiple edge computing devices, and sending the target contribution vectors corresponding to the multiple edge computing devices to their respective edge computing devices.

[0082] The reference contribution value is used to characterize the performance level achieved by an edge computing device training the global model independently, that is, the value that an edge computing device can create when training the global model independently. Optionally, the individual contribution can be roughly estimated based on the Shapley value principle, for example, by approximating it with a small number of permutations, to obtain the reference contribution value corresponding to each edge computing device.

[0083] In a specific implementation, in an optional embodiment, the reference contribution value corresponding to multiple edge computing devices is implemented as follows: random sampling is performed on all possible participation orders of multiple edge computing devices to generate multiple device permutation sequences; for each permutation sequence, the marginal performance gain brought by each edge computing device when it is added to the current federated learning set is calculated; the average value of the marginal performance gain corresponding to each edge computing device in all sampled permutations is determined as the reference contribution value corresponding to each edge computing device.

[0084] In other words, in the second optimization stage, the deviation minimization criterion can be used to minimize the deviation between the actual contribution allocation of each edge computing device and the reference contribution value. In this way, the optimal contribution allocation scheme can be selected from the stable solution set determined in the first optimization stage, and the target contribution vectors corresponding to multiple edge computing devices can be determined. This allows for the selection of a unique contribution vector from all stable solutions that can maximize individual fairness, minimizing the deviation between the contribution obtained by each edge computing device and the benchmark individual value, thereby improving the transparency of the incentive mechanism and the acceptance of each edge computing device.

[0085] In summary, this embodiment of the application uses a first linear programming solver to obtain the maximum loss value corresponding to all sub-federated learning sets based on their respective collaborative contribution values, and optimizes the maximum loss value to obtain the minimized maximum loss value. Then, using a second linear programming solver, based on the minimized maximum loss value corresponding to all sub-federated learning sets, the error between the contribution allocation of multiple edge computing devices and the reference contribution value corresponding to multiple edge computing devices is minimized to determine the target contribution vector corresponding to each of the multiple edge computing devices. In other words, the first linear programming solver, combined with the collaborative contribution values ​​of multiple sub-federated learning sets, solves and optimizes the maximum loss value of all sub-federated learning sets. This approach minimizes the maximum loss value at each sub-federated learning set level, providing a precise and reasonable constraint benchmark for subsequent optimization of edge computing device contribution allocation. Simultaneously, using a second linear programming solver based on the minimized maximum loss value, it specifically optimizes the error between the contribution allocation and reference contribution value of multiple edge computing devices. This achieves accurate determination of the target contribution vector for each edge computing device while meeting the overall loss control requirements of each sub-federated learning set, thus precisely quantifying the contribution of each edge computing device in the global model training process. This not only ensures the rationality and accuracy of the contribution allocation of each edge computing device but also improves the collaborative efficiency and operational stability of the entire distributed training system.

[0086] The above embodiments describe the process of obtaining the maximum loss value corresponding to all sub-federated learning sets using a first linear programming solver, optimizing this maximum loss value to obtain the minimized maximum loss value corresponding to all sub-federated learning sets, and the process of minimizing the error between the contribution allocation of multiple edge computing devices and the reference contribution value corresponding to multiple edge computing devices using a second linear programming solver. To facilitate a better understanding of this specific process, the following provides a detailed explanation of the specific process of obtaining the maximum loss value corresponding to all sub-federated learning sets using a first linear programming solver, optimizing this maximum loss value, to obtain the minimized maximum loss value corresponding to all sub-federated learning sets.

[0087] Before determining the minimized maximum loss value for all sub-federated learning sets, we can first construct the corresponding standard minimum kernel linear programming problem based on the collaborative contribution values ​​of each of the multiple sub-federated learning sets to obtain the minimized maximum loss value.

[0088] Specifically, the process begins by aggregating the local model update information for all edge computing devices to generate an updated third global model. Based on the performance metrics of this updated third global model on a public test set, the global collaborative contribution value for each edge computing device is determined. This global collaborative contribution value characterizes the performance level achieved after all edge computing devices jointly train the global model. Then, the individual contribution values ​​of each edge computing device and the maximum loss value for all sub-federated learning sets are used as the first decision variables. Minimizing the maximum loss value is then used as the first objective function. Based on the first decision variables, the collaborative contribution values ​​for each sub-federated learning set, and the global collaborative contribution value, the first constraints are determined. Finally, based on the first decision variables, the first objective function, and the first constraints, a first linear programming model is constructed.

[0089] The first constraint includes equality constraints, inequality constraints, and a first variable boundary. Specifically, the equality constraint states that the sum of the individual contribution values ​​of all edge computing devices equals the global collaborative contribution value. The inequality constraint states that for the target sub-federated learning set, the sum of the individual contribution values ​​of the edge computing devices included in the target sub-federated learning set and the sum of the maximum loss values ​​corresponding to all sub-federated learning sets are not less than the collaborative contribution value of the target sub-federated learning set. The first variable boundary states that the individual contribution values ​​are any real numbers, and the maximum loss value is non-negative.

[0090] It should be noted that the inequality constraint must be followed for any sub-federated learning set. For details, please refer to the inequality constraints of the target sub-federated learning set.

[0091] After constructing the first linear programming model, the first linear programming solver is invoked to solve the model, obtaining the first solution result. Based on this result, the minimized maximum loss value is then obtained. The first linear programming solver is configured to output the optimal solution of the first objective function while satisfying all first constraints, thereby obtaining the minimized maximum loss value for all sub-federated learning sets.

[0092] As described above, by constructing the first linear programming model to focus on the overall loss control at the level of sub-federated learning sets, the problem of uneven loss among sub-federated learning sets and high overall risk during the federated learning collaboration process is solved, thus ensuring the stability of the distributed training system.

[0093] The following section details the specific implementation process of minimizing the error between the contribution allocation of multiple edge computing devices and the reference contribution value corresponding to the multiple edge computing devices using the second linear programming solver.

[0094] Before determining the target contribution vector for each edge computing device by minimizing the error between the contribution allocation of multiple edge computing devices and the reference contribution value corresponding to the multiple edge computing devices, a deviation minimization linear problem can be constructed based on the minimized maximum loss value corresponding to all sub-federated learning sets to determine the target contribution vector for each edge computing device.

[0095] Specifically, based on the reference contribution values ​​corresponding to multiple edge computing devices, first and second auxiliary variables are constructed for each edge computing device. The contribution deviations of each edge computing device are then determined based on these first and second auxiliary variables. The individual contribution values ​​of each edge computing device, along with their corresponding first and second auxiliary variables, are used as the second decision variables. The second objective function is to minimize the sum of the absolute values ​​of the contribution deviations of all edge computing devices. The second constraint is determined based on the second decision variables, the reference contribution values ​​of the edge computing devices, and the maximum loss value after minimizing all sub-federated learning sets. Finally, a second linear programming model is constructed based on the second decision variables, the second objective function, and the second constraint.

[0096] The second constraint includes a minimum loss constraint, a deviation definition constraint, and a second variable boundary. Specifically, the minimum loss constraint is the first constraint, inheriting all the first constraints from the first stage but fixing their maximum loss value to the minimized maximum loss value. The deviation definition constraint states that for the target edge computing device, the difference between the individual contribution value and the reference contribution value of the target edge computing device is equal to the difference between the first auxiliary value and the second auxiliary value of the target edge computing device. The target edge computing device can be any one of multiple edge computing devices, and the deviation definition constraints for other edge computing devices are referenced from those for the target edge computing device. The second variable boundary states that the individual contribution value is any real number, and both the first and second auxiliary values ​​are non-negative.

[0097] The purpose of constructing this second linear programming model is to find a unique contribution allocation for each edge computing device, given the known maximum loss value after minimization, so as to minimize the total deviation between the actual contribution allocation of each edge computing device and the predefined reference contribution value of each edge computing device.

[0098] However, the minimization problem in constructing this second linear programming model is not a linear programming problem. Therefore, it can be transformed into a linear programming problem by introducing auxiliary variables corresponding to each edge computing device. In specific implementation, the L1 norm can be used as a deviation measure to transform it into a linear programming problem.

[0099] Specifically, two auxiliary variables can be introduced for each edge computing device, namely the first auxiliary variable and the second auxiliary variable, to represent the error between the actual contribution allocation and the reference contribution value corresponding to the edge computing device.

[0100] That is, the first auxiliary variable corresponding to any edge computing device is ,in, Indicated

[0101] It represents the actual contribution allocation corresponding to the i-th edge computing device. This represents the reference contribution value corresponding to the i-th edge computing device. The second auxiliary variable for any edge computing device is... .

[0102] Once the 2n auxiliary variables corresponding to the n edge computing devices are determined, the actual contribution of all edge computing devices is minimized.

[0103] The second objective function, calculated by taking the absolute value of the error between the contribution allocation and the reference contribution value, is as follows:

[0104] Among them, p i This represents the i-th edge computing device.

[0105] The deviation constraint for each edge computing device is defined as follows: .

[0106] After constructing the second linear programming model, the second linear programming solver is invoked to solve the model, obtaining the second solution result. Based on this result, the target contribution vectors for each of the multiple edge computing devices are determined. The second linear programming solver is configured to output the optimal solution of the second objective function while satisfying all second constraints, thereby determining the target contribution vectors for each edge computing device.

[0107] As described above, by linking the optimization results of the first linear programming model when constructing the second linear programming model, the contribution allocation of each edge computing device is accurately calibrated to solve the problem of excessive deviation between the actual contribution allocation of each edge computing device and the reference contribution value. The unique deviation that is closest to the reference contribution value of each edge computing device is selected, thus taking into account the fair perception of each edge computing device.

[0108] In practical applications, after determining the target contribution vectors corresponding to multiple edge computing devices, these vectors are sent to the respective edge computing devices to incentivize them to continue participating in the current joint training of the global model. This also incentivizes the edge computing device to adjust its local training data, accelerating the training efficiency of the entire model and improving its training performance. Furthermore, based on the target contribution vectors corresponding to multiple edge computing devices, the aggregation weights when aggregating the model gradients of these devices can be dynamically adjusted to improve the training efficiency and performance of the global model. The following embodiments will provide a detailed explanation of this process.

[0109] Figure 3 A flowchart illustrating another federated learning contribution evaluation method provided in this application embodiment; see attached document. Figure 3 As shown, specifically, based on the above embodiments, the method may further include the following steps:

[0110] 301. Based on the target contribution vectors corresponding to multiple edge computing devices, determine the first aggregation weight corresponding to each of the multiple edge computing devices during the aggregation process.

[0111] 302. According to the first aggregation weight corresponding to each of the multiple edge computing devices, the model gradients corresponding to the multiple edge computing devices are weighted and averaged to obtain the first global gradient.

[0112] 303. Apply the first global gradient to update the parameters of the current global model to generate the updated second global model.

[0113] 304. Distribute the model parameters corresponding to the second global model to multiple edge computing devices so that the multiple edge computing devices can update their local models based on the model parameters corresponding to the second global model.

[0114] First, based on the target contribution vectors corresponding to multiple edge computing devices, the first aggregation weights for each edge computing device are adjusted during the aggregation of model gradients for those devices. Additionally, the first aggregation weights can be dynamically adjusted by incorporating information such as the historical contribution vectors of the multiple edge computing devices.

[0115] Then, according to the first aggregation weight corresponding to each of the multiple edge computing devices, the model gradients corresponding to the multiple edge computing devices are weighted and averaged to obtain the first global gradient. The first global gradient is then applied to update the parameters of the current global model to generate the updated second global model.

[0116] Finally, the model parameters corresponding to the second global model are distributed to multiple edge computing devices so that the multiple edge computing devices can update their local models based on the model parameters corresponding to the second global model.

[0117] As described above, when aggregating the local model gradients or local model parameters uploaded by each edge computing device, the corresponding aggregation weight is proportional to the target contribution vector corresponding to that edge computing device, thus the aggregated second global model performs better.

[0118] The specific implementation process involved in the embodiments of this application can be referred to the content of the above embodiments, and will not be repeated here.

[0119] To facilitate understanding of the above-described process of evaluating federated learning contributions, an example of this process in a specific application scenario is provided. In a concrete application, the implementation process may include the following steps:

[0120] Step 1: Send contribution evaluation requests to multiple edge computing devices in the distributed training system. Receive local model update parameters uploaded by multiple edge computing devices.

[0121] Step 2: Initialize the federated learning tasks in the distributed training system and define the corresponding value metrics.

[0122] Specifically, the entire set of participants is determined. All potential edge computing devices participating in federated learning within the distributed training system are identified and registered. Core value metrics are defined: key indicators for evaluating the value of each sub-federated learning set are specified, such as task performance metrics (e.g., AUC, denoted as Vperf) and test uncertainty metrics (e.g., negative cross-entropy loss, denoted as Vuncert). The overall value (collaborative contribution value) of the sub-federated learning set is defined as the weighted sum of these indicators: Collaborative contribution value V(S) = Wperf⋅ Vperf(S) + Wuncert⋅ Vuncert(S). A reference individual value (reference contribution value) is defined for each edge computing device: to introduce a bias minimization criterion into the minimum kernel set, this application defines the reference contribution value of each edge computing device as the value it can create when participating in federated learning alone, typically a rough estimate of the individual contribution based on the Shapley value principle (e.g., approximation through a small number of sampling permutations). This needs to be calculated under privacy protection through a security value evaluation module.

[0123] Configure federated learning model and parameters: Select the machine learning model suitable for the current federation and configure its training parameters.

[0124] Prepare a securely shared test set: Ensure that the public test set can be used for model evaluation while protecting privacy.

[0125] Step 3: Determine the collaborative contribution value of each sub-federated learning set.

[0126] Specifically, the training and evaluation of simulated sub-federated learning sets are as follows: For each sub-federated learning set, the security value assessment module initiates a virtual / simulated federated learning assessment task. This task only involves edge computing devices within that sub-federated learning set. In the simulated assessment task, the edge computing devices in the sub-federated learning set perform privacy-preserving joint predictions on a secure shared test set based on their local data, extracting core value indicators. The final value (collaborative contribution value) of each sub-federated learning set is then calculated through a weighted sum of these indicators.

[0127] If a sub-federated learning set is an empty federation, then the sub-federated learning set can be defined as the baseline performance of random guessing.

[0128] Finally, the final values ​​of all sub-federated learning sets are recorded, and all calculated final values ​​are stored to form a value function.

[0129] Step 4: The first-stage minimum loss solver in the two-stage minimum kernel solver module of the federated server is executed. Based on the collaborative contribution values ​​of each of the multiple sub-federated learning sets, the maximum loss value corresponding to all sub-federated learning sets is obtained and optimized to obtain the minimized maximum loss value.

[0130] Step 5: The second-stage deviation minimization solver in the two-stage minimum kernel solver module of the federated server performs the following: Based on the maximum loss value after minimizing all sub-federated learning sets, it minimizes the error between the contribution allocation of multiple edge computing devices and the reference contribution value corresponding to multiple edge computing devices, so as to determine the target contribution vector corresponding to each of the multiple edge computing devices, and sends the target contribution vector corresponding to the multiple edge computing devices to their respective edge computing devices.

[0131] The specific implementation process and the specific technical effects achieved in the embodiments of this application can be referred to the content of the above embodiments, and will not be repeated here.

[0132] The following is combined with Figure 4 This paper illustrates the federated learning contribution evaluation process in a horizontal federated learning application scenario. In practical applications, the specific implementation process may include the following steps:

[0133] Step S1: Obtain the global model through horizontal federated learning.

[0134] The Paillier-based FedAvg algorithm is adopted to meet privacy protection requirements, as detailed below:

[0135] 1. Initialize global model parameters, generate key pairs, and distribute them to each participant.

[0136] Specifically, after the participants determine the machine learning algorithm (usually supervised learning), the federated server performs the following operations:

[0137] Randomly initialize global model parameters Set training rounds Local training epochs Learning rate .

[0138] Generate Paillier homomorphic encryption key pair: public key and private key and public key Distribute the private key to all participants. Securely stored on a federal server. Paillier encryption satisfies additive homomorphism, meaning that for plaintext... ,have

[0139] (That It is the modulus of the Paillier algorithm, and also supports scalar multiplication homomorphism, i.e. ( (is a constant).

[0140] 2. Each participant trains the global model locally.

[0141] Specifically, in the During training rounds: The federated server will provide the global model Distribute to all participants.

[0142] Participants based on local sample sets Train the model using a classification or regression task to obtain locally updated parameters. The calculation formula is:

[0143]

[0144] in, This represents the gradient of the loss function with respect to the parameters.

[0145] 3. Each parameter provider encrypts and uploads the model parameters obtained after training. The model parameters uploaded by each parameter provider are aggregated to obtain the updated encrypted global model.

[0146] Specifically, the participants use public keys. Updated parameters locally Perform Paillier encryption to obtain the encrypted parameters. The data is then encrypted and uploaded to the federated server. The federated server determines the amount of data based on the sample size of each participant. Calculate aggregate weights .

[0147] Using Paillier's additive and scalar multiplication homomorphisms, the encrypted parameters are aggregated:

[0148] First, the encryption parameters for each participant. Perform scalar multiplication to obtain (Here it is assumed) (If it is an integer, it can be processed by scaling or other methods if it is a decimal).

[0149] Then, all participating parties Perform multiplication operations (based on the homomorphism of addition, corresponding to the addition of plaintext) to obtain the encrypted global model. :

[0150]

[0151] 4. Decrypt the encrypted global model and update the global model.

[0152] The server uses a private key. right Paillier decryption is performed to obtain the global model of the plaintext. .

[0153] 5. When the global model's performance converges on the secure shared test set (performance improvement less than p for n consecutive rounds) or reaches the preset number of rounds. When the training terminates, the global model at this point is the final joint model.

[0154] Step S2: Determine the collaborative contribution value v(S) for each sub-alliance.

[0155] The collaborative contribution value v(S) is used to characterize the value of the sub-alliance. Different performance metrics can be selected as the core measure of v(S) based on the task type (classification / regression) of the horizontal federated learning. For example, if the task type is binary classification, then AUC (Area Under ROC Curve) can be used as the core metric.

[0156]

[0157] Its AUC measures the model's ability to distinguish between positive and negative samples, with a value range of [0.5, 1]. A higher value indicates better model performance. Here, D_test^+ / D_test^- are positive and negative samples in the test set, f(x) is the model's positive prediction probability for sample x, and I(⋅) is an indicator function (1 if the condition is met, 0 otherwise).

[0158] When the task type is multi-category, then Macro-F1 can be used as the core metric.

[0159] , The model comprehensively considers the precision (P_i) and recall (R_i) of each category to avoid bias caused by class imbalance; C is the number of categories, with a value range of [0,1], and a higher value indicates better model performance.

[0160] When the task type is regression, RMSE (Root Mean Squared Error) can be used as the core metric.

[0161]

[0162] Its RMSE can measure the model's predicted value ( The deviation between the true value (y(x)) and the true value (y(x)) is in the range [0, +∞). The lower the value, the better the model performance; m is the number of samples in the test set.

[0163] Once the core metrics are set, the collaborative contribution value v(S) for each sub-alliance can be determined based on these metrics. Specifically,

[0164] 1. Divide the participants into groups to obtain multiple sub-alliances.

[0165] Federation servers determine sub-federations The set of participants (assuming) (This refers to the sub-alliance formed by participants 1 and 2), ensuring It is the entire league Non-empty proper subsets ( ).

[0166] 2. Initialize the sub-alliance model.

[0167] The federated server is based on a full federation convergence model. The parameters are used to initialize the sub-alliance model. (Structure and) Consistent), participants load local training data .

[0168] 3. Generate global parameters for the sub-alliance model Continue until the model meets the convergence condition; after convergence, the sub-coalition model is obtained.

[0169] 4. The federated server uses a shared test suite. ,use right Perform predictions and output prediction results (positive class probability for binary classification, class probability for multi-class classification, and continuous values ​​for regression), and map the indicators uniformly to... Interval:

[0170] For classification tasks (AUC / Macro-F1): it is already in... Within the specified range, retain directly. For regression tasks (standardized RMSE): ensure that a larger value indicates better performance (consistent with the direction of the classification metric).

[0171] 5. The federated server will With Sub-Alliance Logo The data is associated and stored in the value function library for subsequent value calculation by participating parties.

[0172] Step S3: Determine the reference contribution value for each participant. .

[0173] Among them, reference contribution value Used to characterize the value of each parameter. Reference contribution value. Participants can be adopted The approximate Shapley value is used for subsequent bias minimization. Considering that the number of participants in inter-agency horizontal federations is typically small (in agency-oriented scenarios), The Shapley value is calculated using a method of "precise calculation + pruning optimization". The specific steps are as follows:

[0174] Sub-alliance enumeration: Generates all alliances that do not include participating parties. Sub-alliance ,common Individual; for those with a scale greater than Sub-alliances utilize "complementarity" pruning (i.e. ), reducing the amount of computation;

[0175] Marginal value calculation: for each sub-alliance Calculated according to the algorithm in 5.2.2 and To obtain the marginal value increment ;

[0176] Weighting calculation: Sub-consortiums are calculated using the Shapley value formula. weight ;

[0177] Sum of Shapley values: Participants The value of the participants is the weighted sum of the marginal value increments of all sub-alliances:

[0178]

[0179] Step S4: Determine the target contribution vector for each participant.

[0180] Specifically, through a two-stage linear programming approach, a unique and stable contribution vector for each participant is obtained. Specifically,

[0181] 1. Construct the first linear programming model.

[0182] Determine the variables corresponding to the first linear programming model: the contribution values ​​of the participants. Maximum loss Determine the objective function for the first linear programming model: minimize the maximum loss. ,Right now Determine the constraints for the first linear programming model: rationality of the entire coalition (equality constraint): (Total contribution equals total value); Sub-alliance stability (inequality constraint): for all non-empty sub-alliances ,have (Total contribution of sub-alliances + Maximum loss ≥ Value of sub-alliances); Variable boundary: (The contribution can be any real number). (Loss, not negative).

[0183] 2. Call the first linear programming solver to solve the first linear programming model and obtain the first solution result. Based on the first solution result, obtain the maximum loss value after minimization.

[0184] Specifically, the above linear programming problem is solved using linear programming to obtain the minimum maximum loss e^* and the set of contribution vectors that satisfy the constraints (at this point, the solution is not unique).

[0185] 3. Construct the second linear programming model.

[0186] Objective: In Under the constraints, find the value with the participants. The unique contribution vector with the smallest deviation is determined through the following steps: First, determine the variables corresponding to the second linear programming model: participant contributions. Auxiliary variables of deviation , (Used to convert L1 bias into linear constraint).

[0187] Determine the objective function for the second linear programming model: minimize the total deviation, i.e.

[0188] Determine the constraints for the second linear programming model: Inherit the constraints from the first stage: And for all , And determine the deviation definition constraints corresponding to the second linear programming model: for each participant , Determine the variable boundaries for the second linear programming model: , , .

[0189] 4. Call the second linear programming solver to solve the second linear programming model and obtain the second solution result; based on the second solution result, determine the target contribution vector corresponding to each of the multiple edge computing devices.

[0190] The unique contribution vector is obtained through a linear programming solver. This vector satisfies both stability constraints and minimizes the deviation from the value of the participants.

[0191] Step S5: Contribution distribution and feedback iteration.

[0192] After the federation model converges, the federation server allocates incentives once based on the final determined contribution vector, and guides the participants to continuously optimize through a feedback mechanism, forming a closed loop. The specific operation is as follows:

[0193] 1. One-time incentive allocation: based on the final contribution vector of the participants. Based solely on this, one or more of the following incentive methods may be combined (the participants may negotiate and determine this themselves).

[0194] Dynamic aggregation weight fixing: This directly links the final weight of each participant in the model aggregation to their contribution value. (Contribution percentage) The higher the contribution, the higher the weight percentage of its local model parameters in the global model, and this weight is permanently effective within this federation cycle.

[0195] Differentiated model permission granting: Based on contribution vector ranking, high-contribution participants (such as...) The top 30% will receive priority access to the global model (e.g., priority access to model API calls in medical scenarios).

[0196] One-time revenue sharing: If the federated model generates commercial revenue (such as service fees for financial risk control models or advertising revenue sharing for recommendation systems), it will be distributed according to the proportion of contribution. A one-time allocation is made, that is, by the participating parties. Revenue share = Total revenue × .

[0197] 2. Feedback and Iteration Guidance: The federated server provides feedback on the final contribution value to the participating parties. Deviation from one's own expectations (e.g.) If the data quality is lower than expected, it indicates insufficient data quality or sample size. Please also include an explanation of the contribution calculation logic (e.g., the value of the sub-alliance). The impact weights); participants can optimize local data based on feedback (such as cleaning noisy samples and supplementing high-value labeled data) to increase their potential contribution to the next round of federated learning (if launched).

[0198] 3. Cycle closed-loop setting: After the current federated learning task ends, if the next iteration is started (e.g., the model needs to be updated to adapt to the new data distribution), then after the new round of model training converges (i.e. after S1 is completed), repeat steps S2-S5, recalculate the contribution vector and perform a one-time allocation to ensure that the incentive matches the dynamic contribution of the participants.

[0199] The federated learning contribution evaluation apparatus of one or more embodiments of this application will be described in detail below. Those skilled in the art will understand that these apparatuses can all be configured using commercially available hardware components through the steps taught in this solution.

[0200] Figure 5 This is a schematic diagram of a federated learning contribution evaluation device provided in an embodiment of this application, applied to a federated server in a distributed training system. The distributed training system further includes multiple edge computing devices, which collaboratively train a global model, such as... Figure 5 As shown, the device includes: a receiving module 11, a grouping module 12, an aggregation module 13, an optimization module 14, and a determination module 15.

[0201] The receiving module 11 is used to receive local model update information uploaded by the multiple edge computing devices.

[0202] Grouping module 12 is used to group the multiple edge computing devices to obtain multiple sub-federated learning sets.

[0203] The aggregation module 13 is used to aggregate the local model update information corresponding to each edge computing device in the target sub-federated learning set for the target sub-federated learning set, so as to generate an updated first global model, and determine the collaborative contribution value corresponding to the target sub-federated learning set based on the performance index of the first global model on the public test set; the target sub-federated learning set is any one of the multiple sub-federated learning sets.

[0204] The optimization module 14 is used to obtain the maximum loss value corresponding to all sub-federated learning sets by using the first linear programming solver, based on the collaborative contribution value corresponding to each of the multiple sub-federated learning sets, and to optimize the maximum loss value to obtain the minimized maximum loss value.

[0205] The determination module 15 is used to minimize the error between the contribution allocation of the multiple edge computing devices and the reference contribution value of the multiple edge computing devices based on the minimized maximum loss value corresponding to all sub-federated learning sets by the second linear programming solver, so as to determine the target contribution vector corresponding to each of the multiple edge computing devices, and send the target contribution vector corresponding to the multiple edge computing devices to their respective edge computing devices.

[0206] Wherein, the collaborative contribution value is used to characterize the model performance level achieved by the target sub-federated learning set through joint training of the global model; the maximum loss value is used to characterize the difference between the collaborative contribution value of the sub-federated learning set and the sum of the contribution values ​​obtained by the edge computing devices it contains under the current global contribution allocation scheme; the reference contribution value is used to characterize the performance level achieved by the edge computing devices training the global model independently.

[0207] Optionally, the local model update information is the model gradient, and the determining module 15 is further configured to: determine the first aggregation weight corresponding to each of the multiple edge computing devices during the aggregation process based on the target contribution vectors corresponding to the multiple edge computing devices; perform a weighted average of the model gradients corresponding to the multiple edge computing devices according to the first aggregation weights corresponding to the multiple edge computing devices to obtain a first global gradient; apply the first global gradient to the parameter update of the current global model to generate an updated second global model; and send the model parameters corresponding to the second global model to the multiple edge computing devices so that the multiple edge computing devices update their local models based on the model parameters corresponding to the second global model.

[0208] Optionally, before obtaining the maximum loss value corresponding to all sub-federated learning sets and optimizing the maximum loss value to obtain the minimized maximum loss value through the first linear programming solver based on the collaborative contribution value corresponding to each of the multiple sub-federated learning sets, the optimization module 14 is further configured to: aggregate the local model update information corresponding to all edge computing devices to generate an updated third global model; determine the global collaborative contribution value corresponding to all edge computing devices based on the performance index of the updated third global model on a public test set; the global collaborative contribution value is used to characterize the performance level achieved by all edge computing devices after jointly training the global model; take the individual contribution value corresponding to each of the multiple edge computing devices and the maximum loss value corresponding to all sub-federated learning sets as first decision variables; take minimizing the maximum loss value as the first objective function; determine the first constraint condition based on the first decision variable, the collaborative contribution values ​​corresponding to the multiple sub-federated learning sets, and the global collaborative contribution value; and construct a first linear programming model based on the first decision variable, the first objective function, and the first constraint condition.

[0209] Optionally, the first constraint includes equality constraints, inequality constraints, and a first variable boundary; the equality constraint is that the sum of the individual contribution values ​​of all edge computing devices equals the global collaborative contribution value; the inequality constraint is that, for the target sub-federated learning set, the sum of the individual contribution values ​​of the edge computing devices included in the target sub-federated learning set and the sum of the maximum loss values ​​corresponding to all sub-federated learning sets is not less than the collaborative contribution value of the target sub-federated learning set; the first variable boundary is that the individual contribution value is any real number, and the maximum loss value is a non-negative value.

[0210] Optionally, the optimization module 14 is specifically used to: call a first linear programming solver to solve the first linear programming model and obtain a first solution result; based on the first solution result, obtain the minimized maximum loss value; wherein the first linear programming solver is configured to output the optimal solution of the first objective function under all first constraints.

[0211] Optionally, before determining the target contribution vector for each of the multiple edge computing devices by minimizing the error between the actual contribution value and the reference contribution value of the multiple edge computing devices based on the minimized maximum loss value corresponding to all sub-federated learning sets using the second linear programming solver, the determining module 15 is further configured to: construct a first auxiliary variable and a second auxiliary variable corresponding to each of the multiple edge computing devices based on the reference contribution value corresponding to the multiple edge computing devices; determine the contribution deviation corresponding to each of the multiple edge computing devices based on the first auxiliary variable and the second auxiliary variable corresponding to each of the multiple edge computing devices; use the individual contribution value, the first auxiliary variable and the second auxiliary variable corresponding to each of the multiple edge computing devices as a second decision variable; use the sum of the absolute values ​​of the contribution deviations of all edge computing devices as a second objective function; determine a second constraint based on the second decision variable, the reference contribution value corresponding to the multiple edge computing devices and the minimized maximum loss value of all sub-federated learning sets; and construct a second linear programming model based on the second decision variable, the second objective function and the second constraint.

[0212] Optionally, the second constraint includes a minimum loss constraint, a deviation definition constraint, and a second variable boundary; the minimum loss constraint is the first constraint; the deviation definition constraint is that, for the target edge computing device, the difference between the individual contribution value of the target edge computing device and the reference contribution value of the target edge computing device is equal to the difference between the first auxiliary value and the second auxiliary value of the target edge computing device; wherein, the target edge computing device is any one of the plurality of edge computing devices; the second variable boundary is that the individual contribution value is any real number, and both the first auxiliary value and the second auxiliary value are non-negative values.

[0213] Optionally, the determining module 15 is specifically used to: call the second linear programming solver to solve the second linear programming model and obtain a second solution result; and determine the target contribution vector corresponding to each of the plurality of edge computing devices based on the second solution result; wherein the second linear programming solver is configured to output the optimal solution of the second objective function under all second constraints.

[0214] Optionally, the performance metrics include task performance metrics and the test uncertainty metrics; the aggregation module 13 is specifically used for: inputting the common test set into the first global model to determine the prediction results corresponding to the common test set; determining the task performance metrics corresponding to the target sub-federated learning set based on the prediction results and the reference results corresponding to the common test set; determining the test uncertainty metrics corresponding to the target sub-federated learning set; and determining the collaborative contribution value corresponding to the target sub-federated learning set based on the task performance metrics and the test uncertainty metrics.

[0215] Optionally, before determining the target contribution vector corresponding to each of the multiple edge computing devices by minimizing the error between the contribution allocation of the multiple edge computing devices and the reference contribution value corresponding to the multiple edge computing devices based on the maximum loss value after minimizing all sub-federated learning sets using the second linear programming solver, the determining module 15 is further configured to: randomly sample all possible participation orders of the multiple edge computing devices to generate multiple device permutation sequences; for each permutation sequence, calculate the marginal performance gain brought by each edge computing device when joining the current federated learning set; and determine the average value of the marginal performance gain corresponding to each edge computing device in all sampled permutations as the reference contribution value corresponding to each edge computing device.

[0216] Optionally, the model update information is the model gradient; wherein, the aggregation module 13 is specifically used to: obtain the local training data volume of each edge computing device in the target sub-federated learning set; determine the second aggregation weight corresponding to each edge computing device in the target sub-federated learning set according to the proportion of the local training data volume of each edge computing device in the target sub-federated learning set to the total training data volume of the target sub-federated learning set; perform a weighted average of the model gradients corresponding to each edge computing device in the target sub-federated learning set according to the second aggregation weight to obtain the second global gradient; apply the second global gradient to the parameter update of the current global model to generate the updated first global model.

[0217] Figure 5 The device shown can perform the steps described in the foregoing embodiments. For detailed execution process and technical effects, please refer to the description in the foregoing embodiments, which will not be repeated here.

[0218] In one possible design, the above Figure 5 The structure of the federated learning contribution assessment device shown can be implemented as an electronic device, such as... Figure 6 As shown, the electronic device may include: a memory 21, a processor 22, and a communication interface 23. The memory 21 stores executable code, which, when executed by the processor 22, enables the processor 22 to at least implement the federated learning contribution evaluation method provided in the foregoing embodiments.

[0219] Furthermore, this application embodiment also provides a non-transitory machine-readable storage medium storing executable code, which, when executed by a processor of an electronic device, enables the processor to at least implement the federated learning contribution evaluation method provided in the foregoing embodiments.

[0220] Furthermore, this application also provides a computer program product, which includes a computer program or instructions that, when executed by a processor, cause the processor to implement the steps in the above-described method embodiments. It should be understood that each step or combination of steps in the above-described method flow can be implemented by a computer program or instructions. Additionally, these computer programs or instructions can be applied to the processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing device, enabling the processor of such a device to function as an apparatus for implementing the corresponding functions in the above-described method embodiments.

[0221] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separate. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0222] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of a necessary general-purpose hardware platform, or by a combination of hardware and software. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a computer product. This application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0223] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A federated learning contribution assessment method, characterized in that, A federated server applied to a distributed training system, the distributed training system further comprising a plurality of edge computing devices, the plurality of edge computing devices cooperatively training a global model, the method comprising: receiving local model update information uploaded by the plurality of edge computing devices; grouping the plurality of edge computing devices to obtain a plurality of sub-federated learning sets; for a target sub-federated learning set, aggregating local model update information corresponding to each edge computing device in the target sub-federated learning set to generate an updated first global model, and determining a cooperative contribution value corresponding to the target sub-federated learning set based on a performance indicator of the first global model on a public test set; the target sub-federated learning set being any one of the plurality of sub-federated learning sets; obtaining a maximum loss value corresponding to all sub-federated learning sets and optimizing the maximum loss value by a first linear programming solver according to the cooperative contribution values corresponding to the plurality of sub-federated learning sets, to obtain a minimized maximum loss value; minimizing an error between a contribution allocation of the plurality of edge computing devices and a reference contribution value corresponding to the plurality of edge computing devices according to the minimized maximum loss value corresponding to all sub-federated learning sets by a second linear programming solver, to determine a target contribution vector corresponding to each of the plurality of edge computing devices, and sending the target contribution vector corresponding to each of the plurality of edge computing devices to the corresponding edge computing device; wherein the cooperative contribution value represents a model performance level achieved by the target sub-federated learning set through joint training of the global model; the maximum loss value represents a difference between the cooperative contribution value of the sub-federated learning set and the sum of the contribution values obtained by the edge computing devices included in the sub-federated learning set under the current global contribution allocation scheme; and the reference contribution value represents a performance level achieved by the edge computing device when training the global model alone.

2. The method of claim 1, wherein, The local model update information is a model gradient, and the method further comprises: determining a first aggregation weight corresponding to each of the plurality of edge computing devices in the aggregation process according to the target contribution vector corresponding to each of the plurality of edge computing devices; performing weighted averaging on the model gradients corresponding to the plurality of edge computing devices according to the first aggregation weight corresponding to each of the plurality of edge computing devices to obtain a first global gradient; applying the first global gradient to parameter update of a current global model to generate an updated second global model; downloading model parameters corresponding to the second global model to the plurality of edge computing devices, so that the plurality of edge computing devices update local models based on the model parameters corresponding to the second global model.

3. The method of claim 1, wherein, Before the step of obtaining a maximum loss value corresponding to all sub-federated learning sets and optimizing the maximum loss value by a first linear programming solver according to the cooperative contribution values corresponding to the plurality of sub-federated learning sets, the method further comprises: aggregating local model update information corresponding to all edge computing devices to generate an updated third global model; determine a global collaborative contribution value corresponding to the all edge computing devices based on the performance indicator of the updated third global model on the public test set; the global collaborative contribution value is used to represent a performance level reached after the all edge computing devices jointly train the global model; take the individual contribution value corresponding to each of the plurality of edge computing devices and the maximum loss value corresponding to the all sub-federated learning sets as first decision variables; take minimizing the maximum loss value as a first objective function; determine a first constraint condition according to the first decision variables, the collaborative contribution values corresponding to the plurality of sub-federated learning sets, and the global collaborative contribution value; construct a first linear programming model based on the first decision variables, the first objective function, and the first constraint condition.

4. The method of claim 3, wherein, The first constraint condition includes an equality constraint and an inequality constraint, and a first variable boundary; The equality constraint is that the sum of the individual contribution values of the all edge computing devices is equal to the global collaborative contribution value; The inequality constraint is that, for the target sub-federated learning set, the sum of the individual contribution values of the edge computing devices included in the target sub-federated learning set and the maximum loss value corresponding to the all sub-federated learning sets is not less than the collaborative contribution value of the target sub-federated learning set; The first variable boundary is that the individual contribution value is any real number, and the maximum loss value is a non-negative value.

5. The method of claim 4, wherein, The first linear programming solver is configured to output the optimal solution of the first objective function under the condition that all the first constraint conditions are met. Before minimizing the error between the actual contribution values of the plurality of edge computing devices and the reference contribution values corresponding to the plurality of edge computing devices to determine the target contribution vector corresponding to each of the plurality of edge computing devices according to the maximum loss value corresponding to the all sub-federated learning sets, the method further includes: construct first auxiliary variables and second auxiliary variables corresponding to each of the plurality of edge computing devices according to the reference contribution values corresponding to the plurality of edge computing devices; determine the contribution deviation corresponding to each of the plurality of edge computing devices according to the first auxiliary variables and the second auxiliary variables corresponding to each of the plurality of edge computing devices; 6. The method of claim 1, wherein, take the individual contribution value corresponding to each of the plurality of edge computing devices, the first auxiliary variables and the second auxiliary variables corresponding to each of the plurality of edge computing devices as second decision variables; take the sum of the absolute values of the contribution deviations of the all edge computing devices as a second objective function; ​ ​ ​ determine a second constraint condition according to the second decision variable, reference contribution values corresponding to the plurality of edge computing devices, and the minimized maximum loss value of the all sub-federated learning sets; construct a second linear programming model based on the second decision variable, the second objective function, and the second constraint condition.

7. The method of claim 6, wherein, The second constraint condition includes a minimum loss constraint and a bias definition constraint, a second variable boundary; The minimum loss constraint is the first constraint condition; The bias definition constraint is that, for a target edge computing device, a difference between an individual contribution value of the target edge computing device and a reference contribution value of the target edge computing device is equal to a difference between a first auxiliary value of the target edge computing device and a second auxiliary value of the target edge computing device; and the target edge computing device is any one of the plurality of edge computing devices; The second variable boundary is that the individual contribution value is any real number, and the first auxiliary value and the second auxiliary value are both non-negative values.

8. The method of claim 7, wherein, The minimizing, by the second linear programming solver, of the error between the contribution allocation of the plurality of edge computing devices and the reference contribution values corresponding to the plurality of edge computing devices according to the minimized maximum loss value of the all sub-federated learning sets to determine the target contribution vector corresponding to each of the plurality of edge computing devices includes: calling the second linear programming solver to solve the second linear programming model to obtain a second solution result; determining the target contribution vector corresponding to each of the plurality of edge computing devices based on the second solution result; The performance indicator includes a task performance indicator and a test uncertainty indicator.

9. The method of claim 1, wherein, The determining of the collaborative contribution value corresponding to the target sub-federated learning set based on the performance indicator of the updated first global model on the public test set includes: inputting the public test set into the first global model to determine a prediction result corresponding to the public test set; determining a task performance indicator corresponding to the target sub-federated learning set according to the prediction result and a reference result corresponding to the public test set; determining a test uncertainty indicator corresponding to the target sub-federated learning set; determining a collaborative contribution value corresponding to the target sub-federated learning set according to the task performance indicator and the test uncertainty indicator. The method further includes, before the minimizing, by the second linear programming solver, of the error between the contribution allocation of the plurality of edge computing devices and the reference contribution values corresponding to the plurality of edge computing according to the minimized maximum loss value of the all sub-federated learning sets to determine the target contribution vector corresponding to each of the plurality of edge computing devices:

10. The method of claim 1, wherein, randomly sampling all possible participation sequences of the plurality of edge computing devices to generate a plurality of device arrangement sequences; for each arrangement sequence, calculating a marginal performance gain brought by each edge computing device when joining the current federated learning set; ​ An average value of marginal performance gains of the edge computing devices in all sampling arrangements is determined as a reference contribution value corresponding to the edge computing devices.

11. The method of claim 1, wherein, The model update information is a model gradient; The aggregation of the local model update information of the edge computing devices in the target sub-federated learning set to generate the updated first global model comprises: Obtaining the local training data amount of each edge computing device in the target sub-federated learning set; According to the proportion of the local training data amount of each edge computing device in the target sub-federated learning set in the total training data amount of the target sub-federated learning set, determining a second aggregation weight corresponding to each edge computing device in the target sub-federated learning set; According to the second aggregation weight, the model gradient corresponding to each edge computing device in the target sub-federated learning set is weighted and averaged to obtain a second global gradient; The second global gradient is applied to the parameter update of the current global model to generate an updated first global model.

12. An electronic device, comprising: Comprise: A memory, a processor, a communication interface; wherein the memory stores executable code, when the executable code is executed by the processor, the processor executes the federated learning contribution evaluation method in any one of claims 1 to 11.

Images