Federated learning secure aggregation method, medium and device based on QoS gain
Through the federated learning security aggregation method based on QoS gain and the cooperative game alliance mechanism, global model aggregation is accelerated, and the client legitimacy identification problem and computational overhead problems are solved, and the rapid convergence and high accuracy of the global model are achieved.
Patent Information
- Application Number
- CN202410209452.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-02-26
- Publication Date
- 2024-11-01
- Estimated Expiration
- 2044-02-26
AI Technical Summary
The existing technology cannot effectively distinguish client legitimacy, resulting in problems such as large computing overhead and slow global model convergence speed and low accuracy.
Using the federated learning secure aggregation method based on QoS gain, local training is carried out through the data and computing power of illegal clients, global model aggregation is accelerated, and a client alliance mechanism for cooperative games is introduced to provide clients with unmet accuracy gains to join federated learning. Opportunity, train the D3QN model to filter the model parameters, reducing the overhead of verification calculation.
Fuzz the boundaries between legal and illegal clients, use illegal client resources to accelerate global model aggregation, improve the accuracy and robustness of the global model, reduce the intention of illegal client attack behavior, and achieve rapid convergence and efficient verification.
Smart Images

Figure CN117973565B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of data protection, and in particular to a QoS gain-based federated learning security aggregation method, medium and device. Background Art
[0002] Relying on big data, various deep learning technologies are constantly emerging. The traditional centralized machine learning model training model concentrates all data on a central server for training, which has caused worrying privacy and security issues because the data may be leaked during the concentration and processing process.
[0003] Federated learning is a promising machine learning paradigm that leverages distributed, personalized datasets from many clients while providing data privacy protection for the clients participating in the training. In each iteration of federated learning, a central server aggregates local updates from clients and uses them to train the global model. This also makes it easy for illegal clients with attack behaviors such as data poisoning to interfere with the normal global model aggregation.
[0004] Traditional methods are committed to quantifying indicators such as the client's reputation value to determine the legitimacy of the client in order to ensure reliable federated learning. However, the legitimacy of the client is ambiguous. In order to disguise themselves from being discovered, malicious clients will work normally during certain periods of time and appear to be legitimate clients to the outside world. However, legitimate clients will mislead the global model due to their own poor data quality, behave like illegal clients, and increase computing overhead, resulting in reduced accuracy of the global model. Summary of the invention
[0005] To this end, the technical problem to be solved by the present invention is to overcome the problems in the prior art that the legitimacy of the client cannot be effectively identified, the computational overhead is large, the global model convergence speed is slow, and the accuracy is low. To solve the above technical problems, the present invention provides a federated learning security aggregation method, medium, and device based on QoS gain, which uses the data and computing power of illegal clients to carry out local training, accelerate global model aggregation, provide multi-client aggregation accuracy, and further provide robustness.
[0006] In order to achieve the above object, the main technical solutions adopted by the present invention include:
[0007] In a first aspect, the present invention provides a federated learning security aggregation method based on QoS gain, comprising:
[0008] The client performs local training to obtain client model parameters, and uploads the client model parameters to the server; the client includes a third-party auxiliary client and a large trusted client; the client model parameters include third-party auxiliary client model parameters and large trusted client model parameters; the server tests the third-party auxiliary client model parameters through a public verification set of the task target to obtain the third-party auxiliary client model parameter accuracy, and selects the third-party auxiliary client that meets the preset accuracy gain verification conditions as a qualified client according to the third-party auxiliary client model parameter accuracy, and the third-party auxiliary client that does not meet the preset accuracy gain verification conditions is a non-qualified client;
[0009] The non-compliant clients are organized in the form of an alliance to obtain a client alliance; the client alliance performs local training to obtain alliance model parameters, and the alliance model parameters are uploaded to the server; the server tests the alliance model parameters through the public verification set of the task target to obtain the accuracy of the alliance model parameters, and selects the client alliance that meets the preset accuracy gain verification condition as the compliant alliance according to the alliance model parameter accuracy;
[0010] A target client is selected through a Top-K mechanism according to the qualified alliance and the qualified client, and a D3QN model is trained according to the client alliance and the qualified client. After the training is completed to obtain the D3QN model, the D3QN model is used to replace the Top-K mechanism to select the target client; the alliance model parameters of the target client or the third-party auxiliary client model parameters of the target client and the large trusted client model parameters are aggregated to obtain a global model;
[0011] If the accuracy of the global model does not reach the preset convergence accuracy, the client performs local training to obtain client model parameters, and uploads the client model parameters to the server until the accuracy of the global model reaches the preset convergence accuracy.
[0012] In one embodiment of the present invention, the client performs local training to obtain client model parameters, and uploads the client model parameters to the server; the client includes a third-party auxiliary client and a large trusted client; the client model parameters include third-party auxiliary client model parameters and large trusted client model parameters; the server tests the third-party auxiliary client model parameters through a public verification set of the task target to obtain the third-party auxiliary client model parameter accuracy, and selects the third-party auxiliary client that meets the preset accuracy gain verification condition as a qualified client according to the third-party auxiliary client model parameter accuracy, and the third-party auxiliary client that does not meet the preset accuracy gain verification condition as a non-qualified client. The specific steps include:
[0013] The server initializes the initial global model and obtains the initialization model parameters, and the client obtains the initialization model parameters;
[0014] Each of the clients performs local training according to the initialization model parameters to obtain the client model parameters, and uploads the client model parameters to the server;
[0015] The server tests the third-party auxiliary client model parameters through the public verification set of the task target to obtain the accuracy of the third-party auxiliary client model parameters;
[0016] Determining whether the accuracy of the third-party auxiliary client model parameters meets the preset accuracy gain verification condition according to the accuracy of the global model;
[0017] If yes, then the third-party auxiliary client is a qualified client;
[0018] If not, the third-party auxiliary client is a substandard client.
[0019] In one embodiment of the present invention, the preset accuracy gain verification condition is expressed as:
[0020]
[0021] Wherein, Bi is the third-party auxiliary client, is the third-party auxiliary client model parameter accuracy of the third-party auxiliary client Bi, Acc t-1 For the accuracy of the global model, λ is set to exponential decay, λ = N0e -αt >0, N0 and α are both preset constant values, and t is the number of iterations.
[0022] In one embodiment of the present invention, the server stores historical global model parameters. After receiving the client model parameters, the server compares the historical global model parameters with the client model parameters. If the client model parameters are the same as the historical global model parameters, the client corresponding to the client model parameters is an illegal client, and regardless of whether the illegal client is a qualified client, it will be eliminated.
[0023] In one embodiment of the present invention, the non-compliant clients are organized into a coalition to obtain a client coalition, and the specific steps include:
[0024] The non-compliant client set is formed according to the non-compliant clients, and the initial contribution value of each non-compliant client to the training accuracy after the alliance is established is obtained by calculating the Shapley value in cooperative game theory;
[0025] Penalize each of the non-compliant clients according to the amount of data shared by each of the non-compliant clients in the cooperative game to obtain a penalty value for each of the non-compliant clients; and obtain a final contribution value for each of the non-compliant clients according to the initial contribution value and the penalty value;
[0026] Initializing the non-compliant client set to obtain a non-compliant client set replica according to the final contribution value and the third-party auxiliary client model parameter accuracy;
[0027] Selecting the non-compliant client with the highest final contribution value from the non-compliant clients that have not yet formed an alliance as the cluster center of the alliance;
[0028] Calculate the cosine similarity of other clients that fail to meet the criteria in the replica with respect to the cluster center, and use the cosine similarity as the similarity between the cluster center and other clients that fail to meet the criteria;
[0029] Adding the unqualified clients whose similarity is greater than a preset similarity threshold to the alliance of the cluster center to obtain an alliance set, and removing the alliance set from the copy; wherein the clients in the same alliance set form one client alliance;
[0030] If the non-compliant client exists in the replica, the non-compliant client set is formed based on the non-compliant client, and the initial contribution value of each non-compliant client to the training accuracy after the alliance is established is calculated by the Shapley value in cooperative game theory until the replica is an empty set.
[0031] In one embodiment of the present invention, the final contribution value is expressed as:
[0032]
[0033] Where, j is the number of the client that does not meet the standard, S j is the set formed by all subsets of the non-compliant client set that include the non-compliant client j, |s| is the number of elements in the alliance s, ω(|s|) is the weighting factor, [υ(s)-v(s{j})] is the marginal contribution of the non-compliant client j, ρ is the weight coefficient and ρ>0, Mount j The amount of data contributed by the currently participating clients that do not meet the standards to the alliance. max The amount of data contributed by the non-compliant client that shares the largest amount of data during the cooperative game.
[0034] In one embodiment of the present invention, a target client is selected through a Top-K mechanism according to the qualified alliance and the qualified client, and a D3QN model is trained according to the client alliance and the qualified client. After the training is completed to obtain the D3QN model, the D3QN model is used to replace the Top-K mechanism to select the target client; the specific steps of aggregating the alliance model parameters of the target client or the third-party auxiliary client model parameters of the target client and the large trusted client model parameters to obtain a global model include:
[0035] According to the accuracy of the alliance model parameters of the qualified alliance and the accuracy of the third-party auxiliary client model parameters of the qualified client, a preset number of qualified alliances or qualified clients ranked top in accuracy are selected as the target clients through a Top-K mechanism;
[0036] Establishing an initial D3QN model, training the initial D3QN model according to the alliance model parameter accuracy of the client alliance and the third-party auxiliary client model parameter accuracy of the qualified client, until the accuracy of the initial D3QN model reaches the preset model accuracy, and obtaining a D3QN model; after the training is completed and the D3QN model is obtained, using the D3QN model to replace the Top-K mechanism to select the target client;
[0037] The alliance model parameters of the target client or the third-party auxiliary client model parameters of the target client and the large trusted client model parameters are aggregated to obtain a global model.
[0038] In one embodiment of the present invention, the state space in the D3QN model is set as:
[0039]
[0040] Wherein, M is the number of the third-party auxiliary clients, Z is the number of the qualified clients, MZ is the number of the unqualified clients, R is the number of the client alliances, s t is the state space of the tth iteration, is the client model parameter of the cth qualified client, is the alliance model parameter of the dth client alliance, w t is the parameter of the global model; the parameter of the alliance model is The alliance model parameter of the client alliance after the client alliance is preset to 0, and when selecting the target client, the client alliance with the alliance model parameter of 0 is excluded;
[0041] The action space in the D3QN model is set as:
[0042]
[0043] in, The action of selecting the client model parameters of the qualified client is as follows: An action for selecting the alliance model parameters of the target-achieving alliance; When , it indicates that the cth client meeting the criteria is selected as the target client. d represents the selection of the dth alliance that meets the standard as the target client, and t represents the number of iterations;
[0044] The D3QN model is set as follows:
[0045]
[0046] Among them, r t Use action a for the tth iteration t The reward received, ε represents a constant greater than 1, is the accuracy of the global model, and θ is the target convergence accuracy.
[0047] In a second aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed, implements the QoS gain-based federated learning security aggregation method described in any one of the first aspects above.
[0048] In a third aspect, the present invention provides a storage device comprising a storage medium and a processor, wherein the storage medium stores a computer program, and when the program is executed by the processor, the federated learning security aggregation method based on QoS gain described in any one of the first aspects above is implemented.
[0049] The above technical solution of the present invention has the following beneficial effects compared with the prior art:
[0050] The QoS gain-based federated learning security aggregation method, medium and device described in the present invention blur the boundaries between legal and illegal clients, and use illegal client data and computing power to conduct local training in order to hide the intention of illegal clients to accelerate global model aggregation by using illegal client data and computing power resources. A client alliance mechanism based on cooperative game is introduced to provide clients with unsatisfactory accuracy gain with the opportunity to join federated learning, and maximize the speed increment of mining unsatisfactory accuracy gain clients to promote global model aggregation. At the same time, the intention of illegal clients to conduct data poisoning and other attack behaviors through the alliance is restricted, and a reinforcement learning network D3QN is trained for screening model parameters to reduce the computational overhead of verification while enabling federated learning to complete convergence as quickly as possible. The present application takes the QoS (Quality of Service) of global model aggregation as the sole guide, selects multiple large trusted clients and third-party auxiliary clients for model aggregation, and further improves robustness. Multi-client aggregation can further improve accuracy, and shows good robustness against poisoning attacks. BRIEF DESCRIPTION OF THE DRAWINGS
[0051] In order to make the content of the present invention more clearly understood, the present invention is further described in detail below according to specific embodiments of the present invention in conjunction with the accompanying drawings, wherein
[0052] Figure 1 The overall flow chart of the federated learning security aggregation method based on QoS gain provided by the preferred embodiment of the present invention;
[0053] Figure 2 for Figure 1 A flowchart of obtaining qualified clients and non-qualified clients of the QoS gain-based federated learning security aggregation method;
[0054] Figure 3 for Figure 1 A flowchart of obtaining a client alliance by the QoS gain-based federated learning security aggregation method;
[0055] Figure 4 for Figure 1 The flowchart of the alliance that obtains the standard of the federated learning security aggregation method based on QoS gain. DETAILED DESCRIPTION
[0056] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments so that those skilled in the art can better understand the present invention and implement it, but the embodiments are not intended to limit the present invention.
[0057] First, refer to Figures 1 to 4As shown, a preferred embodiment of the present invention provides a federated learning security aggregation method based on QoS gain, comprising:
[0058] S1, the client performs local training to obtain client model parameters, and uploads the client model parameters to the server; the server tests the third-party assisted client model parameters through the public verification set of the task target to obtain the third-party assisted client model parameter accuracy, and selects the third-party assisted client that meets the preset accuracy gain verification conditions as the qualified client according to the third-party assisted client model parameter accuracy, and the third-party assisted client that does not meet the preset accuracy gain verification conditions is the unqualified client;
[0059] The client includes a third-party auxiliary client and a large trusted client; the client model parameters include a third-party auxiliary client model parameter and a large trusted client model parameter;
[0060] Specifically, in the federated learning of the present application, a server and a client are included, each client is connected to the server respectively, and the client includes a third-party auxiliary client and a large trusted client, and each client has its own client data set; if only a legitimate client (large trusted client) is used to train the global model, it will be difficult for the entire framework to have sufficient training data, and the quality of the final global model cannot be guaranteed, resulting in reduced accuracy of the global model. Therefore, the present application introduces a third-party auxiliary client on the premise that a large trusted client already exists to increase the training data, but the third-party auxiliary client may have illegal behavior or weak capabilities that affect the training of the global model, so the present application takes the QoS (Quality of Service) of global model aggregation as the only guide, screens out qualified clients that meet the accuracy gain threshold, introduces a client alliance mechanism based on cooperative game, and provides non-qualified clients that do not meet the accuracy gain threshold with the opportunity to join federated learning, maximizes the mining of non-qualified clients' resources, promotes global model aggregation, and utilizes illegal client data and computing power resources while restricting the intention of illegal clients to attack through the alliance, such as data poisoning, accelerates the convergence of the global model under the condition of the presence of illegal clients, and exhibits good robustness against poisoning attacks.
[0061] Specifically, all third-party auxiliary clients and large trusted clients that want to participate in federated learning upload client model parameters to the server after local training; client model parameters include third-party auxiliary client model parameters and large trusted client model parameters; the accuracy of the third-party auxiliary client model parameters in this round is tested through the public verification set of the task target inside the server, and by comparing the accuracy of the third-party auxiliary client model parameters with the accuracy of the global model in the previous round, the third-party auxiliary clients with a difference greater than a certain threshold pass the threshold accuracy setting consideration, and the threshold is set to exponential decay. At the same time, in order to save training costs, some illegal clients do not perform local training and directly upload historical global model parameters for speculation, the server compares the client model parameters uploaded by the client by storing the historical global model parameters, and directly eliminates these illegal clients in this iteration.
[0062] S1 specific steps include:
[0063] S11, the server initializes the initial global model and obtains the initialization model parameters, and the client obtains the initialization model parameters; each client performs local training according to the initialization model parameters, obtains the client model parameters, and uploads the client model parameters to the server;
[0064] Specifically, the server initializes the initial global model and obtains the initial model parameters W ini All third-party auxiliary clients and large trusted clients who want to participate in federated learning download the initialization model parameters W from the server ini ; After local training on the client, the server will collect U large trusted clients (A=A1,A2,…,A U ) and M third-party auxiliary clients (B=B1,B2,…,B M )Uploaded client model parameters;
[0065] S12, the server tests the third-party auxiliary client model parameters through the public verification set of the task target to obtain the third-party auxiliary client model parameter accuracy;
[0066] Specifically, through the public verification set (Val T ), the server tests the third-party auxiliary client B in this iteration i (i=1,2,…,M) model parameter accuracy, and obtain the third-party auxiliary client model parameter accuracy This application is different from the validation set used for local training on the client. The public validation set in the server is independent and will be updated over time to prevent illegal participants from deliberately overfitting on the fixed validation set. In addition, since the evaluation time of the public validation set will directly affect the final convergence time of the global model, this validation set is designed in the form of a small validation set to reduce the time consumed by accuracy verification.
[0067] S13, judging whether the accuracy of the model parameters of the third-party auxiliary client meets the preset accuracy gain verification conditions according to the accuracy of the global model;
[0068] If yes, then the third-party auxiliary client is a qualified client;
[0069] If not, the third-party auxiliary client is a substandard client.
[0070] Specifically, the preset accuracy gain verification condition is expressed as:
[0071]
[0072] Among them, Bi is a third-party auxiliary client. is the third-party auxiliary client model parameter accuracy of the third-party auxiliary client Bi, Acc t-1 is the accuracy of the global model, λ is set to exponential decay, λ=N0e -αt >0, N0 and α are both preset constant values, and t is the number of iterations.
[0073] Specifically, the third-party auxiliary client model parameter accuracy Compare the accuracy Acc of the global model obtained in the previous iteration t-1 ,when When the third-party auxiliary client passes the threshold accuracy setting, it is considered as a qualified client. The qualified client set is denoted as B = (B1, B2, ..., B Z ). During the federated learning process, the accuracy of the global model will gradually increase as the training progresses. In the later stages of training, the increase in accuracy will gradually slow down or even approach 0. This application sets λ to exponential decay.
[0074] In addition, during the local training stage of the client, each round of training will incur a certain computing cost. If the illegal client does not perform local training and directly uploads the historical global model parameters, the training cost can be saved. And because this application is guided by accuracy gain, and directly copying the global model parameters of round t-1 for public verification set testing in round t, there will also be a relatively high probability of passing the above-mentioned accuracy gain verification. In order to avoid the above-mentioned speculative behavior, the server compares the model parameters uploaded by the client by storing the historical global model parameters. Even if these third-party auxiliary clients meet the accuracy threshold requirements because they record the previous global model, they will be eliminated. Among them, the rationality of the above mechanism is based on the assertion that the probability that the two sets of neural network model parameters are completely consistent is almost zero.
[0075] S2: organize the clients that do not meet the standards into an alliance to obtain a client alliance. The client alliance performs local training to obtain alliance model parameters, and uploads the alliance model parameters to the server. The server tests the alliance model parameters through the public verification set of the task target to obtain the alliance model parameter accuracy. According to the alliance model parameter accuracy, the client alliance that meets the preset accuracy gain verification conditions is selected as the qualified alliance.
[0076] This application provides an opportunity for weaker clients that fail to pass the preset accuracy gain verification conditions to join federated learning, forming multiple client alliances with the non-compliant clients to improve computing power and integrate the data resources of the client alliance for training.
[0077] The specific steps of S2 include:
[0078] S21, organizing the clients that do not meet the standards into an alliance to obtain a client alliance;
[0079] S21 specifically includes:
[0080] S211, forming a non-standard client set according to the non-standard clients, and calculating the initial contribution value of each non-standard client to the training accuracy after the alliance is established by using the Shapley value in cooperative game theory;
[0081] Specifically, this application transforms the alliance behavior between non-compliant clients into a cooperative game, and uses the Shapley Value (SV) in the cooperative game to calculate the contribution of each auxiliary client to the training accuracy after the alliance. Let I = {1, 2, 3...MZ} be a set of MZ non-compliant clients, and the initial contribution value of each non-compliant client is expressed as:
[0082]
[0083] Where j is the number of the client that does not meet the standard, S jis the set of all subsets of the non-compliant client set that includes the non-compliant client j, |s| is the number of elements in the alliance s, [υ(s)-v(s{j})] is the marginal contribution of the non-compliant client j, ω(|s|) is the weighting factor, and ω(|s|) is expressed as:
[0084]
[0085] υ(s) is the overall benefit generated by the cooperation of all elements in alliance s (the accuracy that alliance s can achieve).
[0086] S212, punishing each non-compliant client according to the amount of data shared by each non-compliant client in the cooperative game to obtain a penalty value for each non-compliant client; and obtaining a final contribution value for each non-compliant client according to the initial contribution value and the penalty value;
[0087] Specifically, considering that the way in which each non-compliant client shares data within the alliance violates the principle of federated learning not to directly exchange data to avoid privacy leakage to a certain extent, this application punishes the member according to the amount of data shared by each participant in the cooperative game. Participants gain more marginal contributions by sharing more local data, so a proportional penalty value is given; the final contribution value of each non-compliant client is obtained based on the initial contribution value and the penalty value, and the final contribution value is expressed as:
[0088]
[0089] Where ρ>0, set as weight coefficient, Mount j The amount of data contributed by the currently participating clients that do not meet the standards to the alliance. max The amount of data contributed by the non-compliant clients that share the most data during cooperative games.
[0090] S213, according to the final contribution value and the third-party auxiliary client model parameter accuracy, initialize the non-compliant client set to obtain a non-compliant client set replica; select the non-compliant client with the highest final contribution value from the non-compliant clients that have not yet formed an alliance as the cluster center of the alliance; calculate the cosine similarity of other non-compliant clients in the replica with respect to the cluster center, and use the cosine similarity as the similarity between the cluster center and other non-compliant clients; add the non-compliant clients whose similarity is greater than a preset similarity threshold to the alliance of the cluster center to obtain an alliance set, and remove the alliance set in the replica; wherein the clients in the same alliance set form a client alliance;
[0091] Specifically, after calculating the final contribution value of each non-compliant client that wants to form an alliance, combined with the test accuracy of the round recorded by the server, initialize the non-compliant client set replica Q and the empty set P about SV and accuracy, first select the non-compliant client with the highest final contribution value among the non-compliant clients that have not yet formed an alliance as the cluster center of the first alliance; secondly, calculate the cosine similarity of each non-compliant client node in the replica about the center point as the similarity between the center point and the node, and form a client alliance about the center point according to the preset similarity threshold (that is, non-compliant clients with a value greater than the preset similarity threshold join the client alliance), and output it as the client alliance set P, each output client alliance set P is represented as an independent subset, and the alliance set P is removed from the replica Q at the same time;
[0092] S214, if there are non-compliant clients in the replica, a non-compliant client set is formed based on the non-compliant clients, and the initial contribution value of each non-compliant client to the training accuracy after the alliance is established is calculated by the Shapley value in cooperative game theory until the replica is an empty set.
[0093] Specifically, the final contribution values of the remaining clients that have not formed an alliance and have not reached the target are recalculated and the above steps are repeated until Q is an empty set. The present application ensures the dispersion of each alliance center point and the similarity of clients within the alliance by dynamically updating the alliance cluster center.
[0094] S22, the client alliance performs local training to obtain alliance model parameters, and uploads the alliance model parameters to the server;
[0095] S23, the server tests the alliance model parameters through the public verification set of the task target to obtain the alliance model parameter accuracy, and selects the client alliance that meets the preset accuracy gain verification conditions as the qualified alliance according to the alliance model parameter accuracy.
[0096] S3: Select the target client through the Top-K mechanism based on the qualified alliance and qualified clients, and train the D3QN model based on the client alliance and qualified clients. After the training is completed and the D3QN model is obtained, the D3QN model is used to replace the Top-K mechanism to select the target client; the alliance model parameters of the target client or the third-party auxiliary client model parameters of the target client and the large trusted client model parameters are aggregated to obtain the global model;
[0097] Specifically, this application selects the Top-K clients with accuracy performance from the set of qualified clients and qualified alliances on the public auxiliary verification set as target clients, and participates in the aggregation of the global model together with large trusted clients; to address the problem of high time cost of the Top-K mechanism, this application trains a reinforcement learning network D3QN for screening model parameters. As the training process progresses, D3QN converges and replaces the Top-K mechanism.
[0098] The specific steps of S3 include:
[0099] The alliance model parameter accuracy of the qualified alliance is obtained based on the qualified alliance, and the third-party auxiliary client model parameter accuracy of the qualified client is obtained based on the qualified client;
[0100] The Top-K mechanism selects a preset number of qualified alliances or qualified clients in the top accuracy ranking as target clients, and aggregates the alliance model parameters of the target clients or the third-party auxiliary client model parameters of the target clients and the large trusted client model parameters to obtain a global model;
[0101] Establish an initial D3QN model, and train the initial D3QN model according to the alliance model parameter accuracy of the client alliance and the third-party assisted client model parameter accuracy of the qualified client, until the accuracy of the initial D3QN model reaches the preset model accuracy, and obtain the D3QN model. After the training is completed and the D3QN model is obtained, the D3QN model is used to replace the Top-K mechanism to select the target client, and the alliance model parameters of the target client or the third-party assisted client model parameters of the target client and the large trusted client model parameters are aggregated to obtain the global model of this round.
[0102] Among them, this application screens out qualified clients whose accuracy gain meets the threshold from the additionally introduced third-party auxiliary terminals, forms multiple alliances with clients that do not meet the standards, integrates computing power, and screens out qualified alliances. Finally, the target clients with higher accuracy are screened out again from the qualified alliances and qualified clients for training together with large trusted clients to obtain a global model. The whole process integrates the client's data and computing power resources multiple times to maximize the speed increment of mining the client to promote the aggregation of the global model and accelerate the convergence of the global model.
[0103] This application first uses the Top-K mechanism to screen out the target clients, and at the same time trains a reinforcement learning network D3QN model for screening model parameters. As the training process progresses, D3QN converges and replaces the Top-K mechanism. The D3QN model is used to select the highest quality from R+Z model parameters to enter the global model aggregation stage; the specific settings of the D3QN model are as follows:
[0104] State space: Each round of updates in federated learning will update the parameters of the model. The state of the tth round is:
[0105]
[0106] Among them, M is the number of third-party assisted clients, Z is the number of qualified clients, MZ is the number of unqualified clients, and R is the number of client alliances. is the client model parameter of the cth qualified client, is the alliance model parameter of the d-th client alliance, w t is the parameter of the global model; it is worth noting that when the number of client alliances R is not equal to MZ, the alliance model parameter is The alliance model parameters of the client alliance after R will be set to 0, that is, Set it to 0. When selecting actions, exclude the state that is set to 0.
[0107] Action space: After receiving the state s of round t t After that, an action a needs to be performed t To select which non-zero model parameters to enter the training of the global model in federated learning, a t Defined as:
[0108]
[0109] in, is a 0, 1 binary variable representing the selection action of the client model parameters for the cth qualified client: If it is 1, it means that the client model parameters of the cth qualified client are selected to participate in the training of the global model, that is, the cth qualified client is selected as the target client, a t If c is 0, the cth qualified client is not selected; is a 0, 1 binary variable representing the alliance model parameter selection action for the dth qualified alliance. If it is 1, it means that the alliance model parameters of the dth qualified alliance are selected to participate in the training of the global model, that is, the dth qualified alliance is selected as the target client. If it is 0, it means that it is not selected. The action space in this application can select the model parameters of the most appropriate scale from the client model parameters of the qualified client and the alliance model parameters of the qualified alliance, thereby avoiding the training overhead in the test accuracy stage.
[0110] Reward: In state s t Next, use action a t After the selection, you will receive a corresponding reward to indicate the quality of the selected action, and set the reward function to the global model accuracy The difference between the target convergence accuracy θ and the current federated learning round number t is matched:
[0111]
[0112] ε represents a constant greater than 1, The setting can enable the D3QN model to complete training in fewer federated learning training rounds, thereby achieving the ideal global model preset convergence accuracy.
[0113] S4, if the accuracy of the global model does not reach the preset convergence accuracy, the client performs local training to obtain the client model parameters, and uploads the client model parameters to the server until the accuracy of the global model reaches the preset convergence accuracy.
[0114] Specifically, after steps S1 to S3, the present application selects a batch of high-quality target clients from the initial third-party auxiliary clients, and uses the target clients together with large trusted clients to participate in federated learning and train the global model. Since each third-party auxiliary client will not become an illegal client in every iteration, the present application blurs the boundary between legal and illegal clients. In each step S1 to S3, different actively participating high-quality target clients will be re-screened to make full use of the resources of each client, and the target clients will be used together with large trusted clients to train the global model. The iteration is continued until the accuracy of the global model reaches the preset convergence accuracy, and the iteration is terminated to obtain the final global model.
[0115] In the case of insufficient training data, this application introduces a third-party auxiliary client, and uses QoS (Quality of Service) as the sole guide to screen out high-quality clients that meet the accuracy standards. Through an alliance method based on cooperative game, low-quality clients that do not meet the accuracy standards are formed into multiple client alliances, computing power and data are integrated, and the qualified alliance is screened out again. Finally, the target client with the highest accuracy is screened out from the high-quality qualified clients and the qualified alliance, and participates in federated learning with large trusted clients to aggregate the global model. This application continuously iterates through the above steps, so that each client has the opportunity to join the federated learning in each iteration. This application continuously screens out batches of high-quality clients from the third-party auxiliary clients to participate in the training of the global model, further improving the robustness and accuracy, and accelerating the convergence of the global model.
[0116] In a second aspect, an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed, implements the QoS gain-based federated learning security aggregation method described in any one of the first aspects above.
[0117] In a third aspect, an embodiment of the present invention provides a storage device, comprising a storage medium and a processor, wherein the storage medium stores a computer program, and when the program is executed by the processor, the federated learning security aggregation method based on QoS gain described in any one of the above-mentioned first aspects is implemented.
[0118] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.
[0119] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0120] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0121] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0122] Obviously, the above embodiments are merely examples for clear explanation and are not intended to limit the implementation methods. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation methods here. The obvious changes or modifications derived from these are still within the protection scope of the invention.
Claims
1. A method for secure aggregation of federated learning, characterized in that: include: The client performs local training to obtain client model parameters, and uploads the client model parameters to the server; the client includes a third-party auxiliary client and a large trusted client; The client model parameters include third-party assisted client model parameters and large trusted client model parameters; the server tests the third-party assisted client model parameters through a public verification set of the task target to obtain the third-party assisted client model parameter accuracy, and selects the third-party assisted client that meets the preset accuracy gain verification conditions as a qualified client according to the third-party assisted client model parameter accuracy, and the third-party assisted client that does not meet the preset accuracy gain verification conditions is a non-qualified client; Organizing the non-compliant clients into an alliance to obtain a client alliance; The client alliance performs local training to obtain alliance model parameters, and uploads the alliance model parameters to the server. The server tests the alliance model parameters through the public verification set of the task target to obtain the accuracy of the alliance model parameters. According to the accuracy of the alliance model parameters, the client alliance that meets the preset accuracy gain verification condition is selected as a qualified alliance. Select a target client through a Top-K mechanism according to the qualified alliance and the qualified client, and train a D3QN model according to the client alliance and the qualified client. After the training is completed to obtain the D3QN model, use the D3QN model to replace the Top-K mechanism to select the target client; Aggregating the alliance model parameters of the target client or the third-party auxiliary client model parameters of the target client and the large trusted client model parameters to obtain a global model; If the accuracy of the global model does not reach the preset convergence accuracy, the client performs local training to obtain client model parameters, and uploads the client model parameters to the server until the accuracy of the global model reaches the preset convergence accuracy.
2. The method for secure aggregation of federated learning according to claim 1, characterized in that: The client performs local training to obtain client model parameters, and uploads the client model parameters to the server; the client includes a third-party auxiliary client and a large trusted client; The client model parameters include third-party assisted client model parameters and large trusted client model parameters; the server tests the third-party assisted client model parameters through a public verification set of the task target to obtain the third-party assisted client model parameter accuracy, and selects the third-party assisted client that meets the preset accuracy gain verification condition as a qualified client according to the third-party assisted client model parameter accuracy, and the third-party assisted client that does not meet the preset accuracy gain verification condition as a non-qualified client. The specific steps include: The server initializes the initial global model and obtains the initialization model parameters, and the client obtains the initialization model parameters; Each of the clients performs local training according to the initialization model parameters to obtain the client model parameters, and uploads the client model parameters to the server; The server tests the third-party auxiliary client model parameters through the public verification set of the task target to obtain the accuracy of the third-party auxiliary client model parameters; Determining whether the accuracy of the third-party auxiliary client model parameters meets the preset accuracy gain verification condition according to the accuracy of the global model; If yes, then the third-party auxiliary client is a qualified client; If not, the third-party auxiliary client is a substandard client.
3. The method for secure aggregation of federated learning according to claim 2, characterized in that: The preset accuracy gain verification condition is expressed as: Wherein, Bi is the third-party auxiliary client, is the third-party auxiliary client model parameter accuracy of the third-party auxiliary client Bi, Acc t-1 For the accuracy of the global model, λ is set to exponential decay, λ = N0e -αt >0, N0 and α are both preset constant values, and t is the number of iterations.
4. The method for secure aggregation of federated learning according to claim 2, characterized in that: The server stores historical global model parameters. After receiving the client model parameters, the server compares the historical global model parameters with the client model parameters. If the client model parameters are the same as the historical global model parameters, the client corresponding to the client model parameters is an illegal client, and regardless of whether the illegal client is a qualified client, it will be eliminated.
5. The method for secure aggregation of federated learning according to claim 1, characterized in that: The specific steps of organizing the non-compliant clients into an alliance to obtain the client alliance include: The non-compliant client set is formed according to the non-compliant clients, and the initial contribution value of each non-compliant client to the training accuracy after the alliance is established is obtained by calculating the Shapley value in cooperative game theory; Penalize each of the non-compliant clients according to the amount of data shared by each of the non-compliant clients in the cooperative game to obtain a penalty value for each of the non-compliant clients; and obtain a final contribution value for each of the non-compliant clients according to the initial contribution value and the penalty value; Initializing the non-compliant client set to obtain a non-compliant client set replica according to the final contribution value and the third-party auxiliary client model parameter accuracy; Selecting the non-compliant client with the highest final contribution value from the non-compliant clients that have not yet formed an alliance as the cluster center of the alliance; Calculate the cosine similarity of other clients that fail to meet the criteria in the replica with respect to the cluster center, and use the cosine similarity as the similarity between the cluster center and other clients that fail to meet the criteria; Adding the unqualified clients whose similarity is greater than a preset similarity threshold to the alliance of the cluster center to obtain an alliance set, and removing the alliance set from the copy; wherein the clients in the same alliance set form one client alliance; If the non-compliant client exists in the replica, the non-compliant client set is formed based on the non-compliant client, and the initial contribution value of each non-compliant client to the training accuracy after the alliance is established is calculated by the Shapley value in cooperative game theory until the replica is an empty set.
6. The method for secure aggregation of federated learning according to claim 5, characterized in that: The final contribution value is expressed as: Where, j is the number of the client that does not meet the standard, S j is the set formed by all subsets of the non-compliant client set that include the non-compliant client j, |s| is the number of elements in the alliance s, ω(|s|) is the weighting factor, [v(s)-v(s{j})] is the marginal contribution of the non-compliant client j, ρ is the weight coefficient and ρ>0, Mount j The amount of data contributed by the currently participating clients that do not meet the standards to the alliance. max The amount of data contributed by the non-compliant client that shares the largest amount of data during the cooperative game.
7. The method for secure aggregation of federated learning according to claim 1, characterized in that: Selecting a target client through a Top-K mechanism according to the qualified alliance and the qualified client, and training a D3QN model according to the client alliance and the qualified client. After the training is completed to obtain the D3QN model, the D3QN model is used to replace the Top-K mechanism to select the target client;The specific steps of aggregating the alliance model parameters of the target client or the third-party auxiliary client model parameters of the target client and the large trusted client model parameters to obtain a global model include: According to the accuracy of the alliance model parameters of the qualified alliance and the accuracy of the third-party auxiliary client model parameters of the qualified client, a preset number of qualified alliances or qualified clients ranked top in accuracy are selected as the target clients through a Top-K mechanism; Establishing an initial D3QN model, training the initial D3QN model according to the alliance model parameter accuracy of the client alliance and the third-party auxiliary client model parameter accuracy of the qualified client, until the accuracy of the initial D3QN model reaches the preset model accuracy, and obtaining a D3QN model; after the training is completed and the D3QN model is obtained, using the D3QN model to replace the Top-K mechanism to select the target client; The alliance model parameters of the target client or the third-party auxiliary client model parameters of the target client and the large trusted client model parameters are aggregated to obtain a global model.
8. The method for secure aggregation of federated learning according to claim 7, characterized in that: The state space in the D3QN model is set as: Wherein, M is the number of the third-party auxiliary clients, Z is the number of the qualified clients, MZ is the number of the unqualified clients, R is the number of the client alliances, s t is the state space of the tth iteration, is the client model parameter of the cth qualified client, is the alliance model parameter of the dth client alliance, w t is the parameter of the global model; the parameter of the alliance model is The alliance model parameter of the client alliance after the client alliance is preset to 0, and when selecting the target client, the client alliance with the alliance model parameter of 0 is excluded; The action space in the D3QN model is set as: in, The action of selecting the client model parameters of the qualified client is as follows: An action for selecting the alliance model parameters of the target-achieving alliance; When , it indicates that the cth client meeting the criteria is selected as the target client. d represents the selection of the dth alliance that meets the standard as the target client, and t represents the number of iterations; The D3QN model is set as follows: Among them, r t Use action a for the tth iteration t The reward received, ε represents a constant greater than 1, is the accuracy of the global model, and θ is the target convergence accuracy.
9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the federated learning security aggregation method described in any one of claims 1 to 8 is implemented.
10. A storage device comprising a storage medium and a processor, wherein the storage medium stores a computer program, wherein: When the processor executes the computer program, the federated learning security aggregation method described in any one of claims 1 to 8 is implemented.
