Federal learning fair model aggregation method based on security data synthesis

Abstract

The invention discloses a federal learning fair model aggregation method based on security data synthesis, which comprises the following steps: training a student model to guide the generation of a generator model, segmenting the data of a participant into a plurality of disjoint subsets and training a plurality of teacher models, and then performing classification and voting by using a data sample generated by a generator, and carrying out noise addition aggregation on the voting result to protect privacy, then labeling a data set by using an aggregation result with noise, training a student model, and further optimizing a generator. After a certain round of optimization, the generators are distributed to the participants, and finally, the aggregated model is sent to the participants for continuous training, so that the fair optimization of the model is realized while the data privacy is protected.

Description

Technical Field

[0001] The present invention relates to the field of federated learning and privacy protection, and in particular to a federated learning fairness model aggregation method based on secure data synthesis. Background Art

[0002] Federated learning does not need to collect all the data of the participants during training like common machine learning. It only optimizes the model through model sharing, which can effectively solve the problem of data islands caused by data incompatibility. When the local data quality of some participants is poor, the models trained by these participants will seriously affect the performance of the global model. In real scenarios, the poor data quality of these participants does not mean that the data they have is worthless. For example, in financial institutions, the data owned by banks and insurance companies in economically developed and underdeveloped regions are indeed different. It is unfair to arbitrarily exclude some participants with poor data quality from training. Expanding the local sample size through data synthesis can help participants with poor data quality solve this problem.

[0003] The local data features of the participants are needed in the process of synthesizing data, but federated learning does not allow the participants to communicate with each other. If data synthesis is performed locally on the participant, the quality of the synthesized data will be difficult to ensure; if the local data features of each participant are shared with the server for unified generation, the risk of privacy leakage is very likely to occur during the sharing process.

[0004] There are many data synthesis technologies under the framework of federated learning to solve the problem of poor data quality of participants, but most of them do not pay attention to the risk of data leakage in the process of synthesizing data. At the same time, these methods do not evaluate the data quality of participants, that is, after synthesizing data, they do not distinguish between participants when selecting participants for training, which is unfair to participants with good data quality.

[0005] Therefore, the problems with the existing technology are: 1) The existing data synthesis method does not consider the risk of data leakage, that is, when the participants share data features with the server, if it is attacked, the original data information is very likely to be leaked; 2) The existing method randomly selects the participants after data synthesis. This selection method is unfair to the participants who have made great contributions.

[0006] Therefore, a fair model aggregation method for federated learning based on secure data synthesis is desired. Summary of the invention

[0007] In order to solve the above technical problems, this application is proposed. The embodiment of this application provides a federated learning model optimization method based on secure data synthesis, which is used to improve the model quality under the premise of protecting the privacy of the participants when the data quality of the participants in the federated learning is poor.

[0008] According to one aspect of the present application, a server-side federated learning fair model aggregation method based on secure data synthesis is provided, which includes: S101. In the server, the generator model is trained by the student model S; S102. The server sends the data samples generated by the generator to all participants; S103. The server receives the noisy voting results after aggregation from the participants, and uses them to annotate a public data set; S104. The student model S is trained using the data set, and the generator is trained by the classification results of the student model S; S105. The server sends the optimized trained generator to the participants; S106. The server receives the data expansion amount and the KL divergence between the expanded data set and the original data set from each participant, and assigns each participant a different training participation probability according to the KL divergence; S107. The server assigns different aggregation weights to the uploaded model updates according to the model updates uploaded by each participant and the data expansion amount of each participant, aggregates the model updates of each participant, and forms a global model; S108. The stability of the global model is achieved through multiple iterations of the participants and the server.

[0009] The federated learning fair model aggregation method based on secure data synthesis provided in the embodiment of the present application realizes secure data expansion of participants and fair server model aggregation, which not only protects the privacy of participants when synthesizing data, but also optimizes the performance of the global model by reasonably allocating aggregation weights according to data quality and expansion amount, ensuring fairness to all participants, thereby providing long-term and high-quality federated learning services.

[0010] Preferably, the step S101 includes: in the server, training the generator model through the student model S, and the student model optimization function L G for:

[0011]

[0012] Among them, m is the number of samples generated by the generator in the pre-training stage, j∈m, S(G(z j θ G )) is the classification probability of the student model for the samples generated by the generator, G(z j θ G ) uses random noise z for the generator j Generated samples;

[0013] Generator parameter θ G The update formula is as follows:

[0014]

[0015] Among them, η G is the learning rate, YesLG For θ G gradient.

[0016] Preferably, in step S102, the server sends the m data samples generated by the generator to all participants, and the teacher model of each participant For data sample g from the server j Classify and count the categories f(f∈c i ) The number of votes received, the number of votes The calculation formula is as follows:

[0017]

[0018] Among them, d is the number of disjoint subsets divided into the original data set, is the indicator function. When the k-th teacher model of participant i converts sample g j When the classification is category f, the indicator function takes the value of 1, otherwise it is 0.

[0019] Preferably, in step S104, the server uses the aggregated noisy voting results to annotate a public data set, the data set is composed of samples generated by the generator, a student model S is trained using the annotated data, the generator is trained using the classification results of the student model S, and the function L is optimized. S for:

[0020]

[0021] Among them, g j Generate samples for the generator, is the noisy label aggregated by the teacher model;

[0022] Student model parameters θ S The update formula is as follows:

[0023]

[0024] Among them, η S is the learning rate, YesL s For θ S gradient.

[0025] Preferably, the step S106 is used for: the server assigns a selection probability p to each participant according to the KL divergence uploaded by the participant. i , and set a training number threshold τ. Each time a participant participates in a training, the threshold τ of the participant is increased. i Minus one, when τ i If it is still selected when it decreases to 0, it will not participate in the training in this round, and the server will reselect, τ i Add one, and the formula is as follows:

[0026]

[0027] Among them, p i is the selection probability of the ith participant, KL i D is the original data distribution of participant i i (x) and the expanded data distribution , R is the set of participants selected to upload model updates in each round, τ is the threshold of the number of times a participant participates in training, τ ≥ 0.

[0028] Preferably, in step S107, after receiving the model update uploaded by the participant, the server aggregates the model update, and the aggregation parameter is based on the data expansion amount ΔD of the participant. i To distribute, the global model aggregation formula and aggregation weight calculation formula are:

[0029]

[0030] Among them, h i is the aggregate weight of the i-th selected participant, w t+1 is the global model for the t+1th round, is the model updated by the i-th participant in round t+1.

[0031] According to one aspect of the present application, a method for aggregating a fair model of federated learning based on secure data synthesis is also provided, which includes: S201. Dividing local data into data subsets and training a teacher model; S202. The teacher model of each participant classifies the data samples from the server and counts the votes obtained by the category; S203. When aggregating the voting results of the teacher model, the votes for each category are noised; S204. Each participant sends the aggregated noisy voting results to the server; S205. Each participant receives the generator model sent from the server, synthesizes data samples according to their own data distribution, and sends the data expansion amount and the KL divergence of the expanded data set and the original data set to the server; S206. Each participant uses the expanded data set to train the initialization model from the server.

[0032] Preferably, step S201 includes: participant i (i∈I) divides local data into d disjoint data subsets, each data subset Train a teacher model Teacher model optimization function L T for:

[0033]

[0034] Where I is the set of participants, i is the i-th participant, i∈I, d is the number of disjoint subsets of the original data set, The kth data subset divided by the i-th participant, For data subset Sample size, x h is the data subset d k The data sample in y h For x h The corresponding label;

[0035] Teacher model parameters The update formula is as follows:

[0036]

[0037]

[0038] The votes are processed by adding noise, which satisfies differential privacy and generates the voting results of the samples.

[0039]

[0040] in, is the Laplace noise added to the voting result of the f-th category of participant i. λ is the scale parameter. The larger λ is, the greater the noise intensity is.

[0041] Preferably, the step S205 is used to: reduce the data volume from the original n i Expand to The enlarged dataset is represented as Each participant uses KL divergence to calculate the difference between the original data distribution and the expanded data distribution, and calculates the data expansion amount ΔD by taking the difference i , then KL divergence and ΔD i Upload to the server, the formula is as follows:

[0042]

[0043] Among them, n i is the original data volume of the ith participant, is the amount of new data after participant i adds the synthetic data, D i is the original local dataset of the ith participant, The new dataset after adding the generator’s synthetic data for the i-th participant, ΔD i is the data expansion amount of the i-th participant, ΔD i ≥0, x represents the original data sample of participant i, represents the data sample of participant i after expansion, and X represents all data samples, that is,

[0044] The present invention is used to improve the model quality while protecting the privacy of the participants in federated learning when the data quality of the participants is poor. In addition, the present invention does not ignore the issue of fairness. When aggregating the model updates uploaded by the participants, a higher aggregation weight is given to the participants with good local data, which also reduces the negative impact of synthetic data on model updates to a certain extent. Specifically, the beneficial technical effects brought about by the present invention are:

[0045] Safe data expansion effect for participants: The present invention avoids the problem of privacy leakage when participants synthesize data. It takes the scenario of federated learning data being stationary and the model being dynamic. The PATE-GAN mechanism is used before federated learning training. The privacy of the participants is protected by the joint training of the generator by the participants and the server. The participants use the generator to synthesize data to expand the local data scale.

[0046] Fair server aggregation model effect: This invention effectively avoids the problem of poor global model effect caused by participants with poor data quality. It takes the federated learning server aggregation update model as the scenario, assigns the participant selection probability according to the original data quality, and assigns the update model aggregation weight according to the data expansion amount, so as to train the best global model and provide long-term high-quality services to the participants. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] By describing the embodiments of the present application in more detail in conjunction with the accompanying drawings, the above and other purposes, features and advantages of the present application will become more apparent. The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0048] Figure 1 The present invention is a flowchart of a method for aggregating a fair model of federated learning based on secure data synthesis according to an embodiment of the present application. DETAILED DESCRIPTION

[0049] Various exemplary embodiments, features and aspects of the present application will be described in detail below with reference to the accompanying drawings. The same reference numerals in the accompanying drawings represent elements with the same or similar functions. Although various aspects of the embodiments are shown in the accompanying drawings, the drawings are not necessarily drawn to scale unless otherwise specified.

[0050] The word “exemplary” is used exclusively herein to mean “serving as an example, example, or illustration.” Any embodiment described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments.

[0051] In addition, in order to better illustrate the present application, numerous specific details are given in the following specific embodiments. It should be understood by those skilled in the art that the present application can also be implemented without certain specific details. In some examples, methods, means, components and circuits well known to those skilled in the art are not described in detail in order to highlight the subject matter of the present application.

[0052] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of this application, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0053] Each participant in the federated learning service is of limited rationality and cannot own the data of other participants. That is, they have a sense of privacy protection and cannot effectively evaluate the data quality of the participants. As the training time increases, the local model trained by the participant with poor data quality will affect the performance of the global model. At the same time, if the server randomly selects the participants for training, it is obviously unfair to the participants with good data quality.

[0054] Therefore, before starting federated learning training, the server and participants work together to train a generative adversarial network (GAN), and synthesize data through the generator in GAN to make up for the problem of participants with poor data quality. Participants synthesize new data through the generator to expand the local data scale, which will reduce the adverse effects on participants with poor data quality, and participants with good data quality will expand the data volume, achieving a win-win situation. After starting federated learning training, the server does not randomly select participants for training, but will give each participant a different selection probability based on the initial data quality. At the same time, participants with good data quality cannot be selected blindly. The server sets a unified training threshold for each participant. When the number of training times exceeds the threshold, it is prohibited to participate in this round of training. Participants with less than the threshold are selected to avoid model overfitting.

[0055] During the training process, participants in the federated learning service need to send model updates after training with local data to the server. When the server aggregates the model updates, it assigns different weights to each participant's model update, making the training process more fair.

[0056] For each participant, the PATE-GAN mechanism helps the server train the generator without sharing data distribution. After that, each participant generates a certain number of data samples through the generator, which can greatly improve the global model performance. At the same time, the PATE-GAN mechanism satisfies differential privacy and will not leak privacy during the data synthesis stage. For the server, each participant will be given a different selection probability based on the quality of the original data, and a unified training threshold will be set for the participants to prevent some participants from participating in training too much. When aggregating model updates, different aggregation weights will be assigned according to the amount of data of each participant.

[0057] Therefore, this solution is designed based on the scenario where there are participants with poor data quality. Specifically, some parameters are defined as follows:

[0058] I: the set of participants;

[0059] T: number of training rounds;

[0060] R: the set of participants selected to upload model updates in each round;

[0061] N: total number of participants;

[0062] i: the i-th participant, i∈I;

[0063] p i : The selection probability of the i-th participant;

[0064] w0: initialize the global model;

[0065] w t : The global model of round t;

[0066] The model updated by the i-th participant in round t+1;

[0067] D i : The original local dataset of the i-th participant;

[0068] D i (x): D i Data distribution;

[0069] n i : The amount of original data of the i-th participant;

[0070] d: the number of disjoint subsets into which the original data set is divided;

[0071] The kth data subset divided by the i-th participant;

[0072] c i : Local data category of participant i;

[0073] m: the number of samples generated by the generator in the pre-training phase;

[0074] g j : training samples generated by the generator in the pre-training phase, j∈m;

[0075] G(z j θ G ): The generator uses random noise z j The generated sample, i.e. g j ;

[0076] S(G(z j θ G )): The classification probability of the student model for the samples generated by the generator;

[0077] S(g j ): Student model for sample g j The classification probability of

[0078] T k (g j ): The teacher model is used for sample g j The classification results;

[0079] The Laplace noise added to the f-th voting result of participant i;

[0080] The new data set after the i-th participant synthesizes data through the generator;

[0081] Data distribution;

[0082] The amount of new data after participant i adds the synthetic data;

[0083] ΔD i : The amount of data expansion of the i-th participant, ΔD i ≥0;

[0084] KL i :Participant i original data distribution D i (x) and the expanded data distribution KL divergence of

[0085] τ: threshold of the number of times a participant participates in training, τ≥0;

[0086] h i : Aggregate weight of the i-th selected participant;

[0087] Therefore, a data synthesis-based federated learning fair model aggregation method, such as Figure 1As shown, its basic process is as follows. According to the data synthesis-based federated learning fair model aggregation method of the embodiment of the present application, the server side includes: S101. In the server, the generator model is trained by the student model S; S102. The server sends the data samples generated by the generator to all participants; S103. The server receives the noisy voting results after aggregation from the participants and uses them to annotate a public data set; S104. The student model S is trained using the data set, and the generator is trained by the classification results of the student model S; S105. The server sends the optimized trained generator to the participants; S106. The server receives the data expansion amount from each participant and the KL divergence between the expanded data set and the original data set, and assigns each participant a different training participation probability according to the KL divergence S107. The server assigns different aggregation weights to the uploaded model updates according to the model updates uploaded by each participant and the data expansion amount of each participant, aggregates the model updates of each participant, and forms a global model; S108. Through multiple iterations of the participants and the server, the stability of the global model is achieved.

[0088] In the embodiment of the present application, step S101 includes: in the server, training the generator model through the student model S, and the student model optimization function L G for:

[0089]

[0090] Among them, m is the number of samples generated by the generator in the pre-training stage, j∈m, S(G(z j θ G )) is the classification probability of the student model for the samples generated by the generator, G(z j θ G ) uses random noise z for the generator j Generated samples;

[0091] Generator parameter θ G The update formula is as follows:

[0092]

[0093] Among them, η G is the learning rate, YesL G For θ G gradient.

[0094] In the embodiment of the present application, in step S102, the server sends the m data samples generated by the generator to all participants, and the teacher model of each participant For data sample g from the server j Classify and count the categories f(f∈c i) The number of votes received, the number of votes The calculation formula is as follows:

[0095]

[0096] Among them, d is the number of disjoint subsets divided into the original data set, is the indicator function. When the k-th teacher model of participant i converts sample g j When the classification is category f, the indicator function takes the value of 1, otherwise it is 0.

[0097] In an embodiment of the present application, in step S103, the server receives the noisy voting results aggregated from the participants and uses them to annotate a public data set. It should be understood that in federated learning, it is very important to protect the data privacy of the participants. By adding noise to the voting results, it is possible to prevent any party from identifying the original data, thereby protecting the data of the participants from being leaked. This method complies with the principle of differential privacy and allows the collective statistical properties of data to be utilized without leaking personal data. The annotated public data set can be used to train the student model S. The classification results of the student model S will be fed back to the generator G to help the generator G learn to generate more realistic data samples. This process is iterative, and as the generator G is continuously optimized, the quality of the generated data samples will improve, thereby improving the performance of the entire federated learning system.

[0098] In the embodiment of the present application, in step S104, the server uses the aggregated noisy voting results to annotate a public data set, the data set is composed of samples generated by the generator, a student model S is trained using the annotated data, the generator is trained using the classification results of the student model S, and the optimization function L is S for:

[0099]

[0100] Among them, g j Generate samples for the generator, is the noisy label aggregated by the teacher model;

[0101] Student model parameters θ S The update formula is as follows:

[0102]

[0103] Among them, η S is the learning rate, YesL S For θ S gradient.

[0104] In an embodiment of the present application, in step S105, the server sends the optimized trained generator to the participants. It should be understood that in federated learning, it is very important to ensure that all participants can contribute fairly to the training of the global model. By sending the optimized generator, it can be ensured that all participants can generate high-quality synthetic data, thereby playing a greater role in model training. Federated learning is an iterative process, and the optimization of the generator is also iterative. By sending the optimized generator to the participants, it can be ensured that the latest generator is used in the next round of training, thereby continuously improving the quality of the synthetic data.

[0105] In the embodiment of the present application, step S106 is used to: the server assigns a selection probability p to each participant according to the KL divergence uploaded by the participant. i , and set a training number threshold τ. Each time a participant participates in a training, the threshold τ of the participant is increased. i Minus one, when τ i If it is still selected when it decreases to 0, it will not participate in the training in this round, and the server will reselect, τ i Add one, and the formula is as follows:

[0106]

[0107] Among them, p i is the selection probability of the ith participant, KL i D is the original data distribution of participant i i (x) and the expanded data distribution , R is the set of participants selected to upload model updates in each round, τ is the threshold of the number of times a participant participates in training, τ ≥ 0.

[0108] In the embodiment of the present application, in step S107, after receiving the model update uploaded by the participant, the server aggregates the model update, and the aggregation parameter is based on the data expansion amount ΔD of the participant. i To distribute, the global model aggregation formula and aggregation weight calculation formula are:

[0109]

[0110] Among them, h i is the aggregate weight of the i-th selected participant, w t+1 is the global model for the t+1th round, is the model updated by the i-th participant in round t+1.

[0111] In the embodiment of the present application, the step S108 achieves the stability of the global model through multiple iterations of the participants and the server. It should be understood that in federated learning, protecting the data privacy of the participants is one of the core principles. Through the iterative process, the global model can be gradually optimized without directly sharing the data, thereby improving the model performance while protecting privacy. Through multiple iterations, it can be ensured that the data of all participants have the opportunity to affect the global model, which helps to ensure the fairness and representativeness of the model and avoid the excessive impact of the data of some participants on the model. In federated learning, since the data of the participants may be biased or noisy, multiple iterations provide opportunities to correct these errors. Through continuous model updates and aggregation, the impact of single participant data problems on the global model can be reduced. After each iteration, the model performance can be tested on the validation set to ensure that the model is not overfitted and has good generalization ability. If the performance does not meet expectations, the model parameters or learning rate can be adjusted, and then the iteration continues. In federated learning, the global model is an aggregation of the models of all participants. Multiple iterations help to find a balance point so that the global model can reflect the data characteristics of all participants and achieve global consensus.

[0112] Furthermore, the participants include: S201. Divide local data into data subsets and train a teacher model; S202. The teacher model of each participant classifies the data samples from the server and counts the votes obtained by the category; S203. When aggregating the voting results of the teacher model, the votes for each category are noised; S204. Each participant sends the aggregated noisy voting results to the server; S205. Each participant receives the generator model sent from the server, synthesizes data samples according to their own data distribution, and sends the data expansion amount and the KL divergence of the expanded data set and the original data set to the server; S206. Each participant uses the expanded data set to train the initialization model from the server.

[0113] In the embodiment of the present application, step S201 includes: participant i (i∈I) divides local data into d disjoint data subsets, each data subset Train a teacher model Teacher model optimization function L T for:

[0114]

[0115] Where I is the set of participants, i is the i-th participant, i∈I, d is the number of disjoint subsets of the original data set, The kth data subset divided by the i-th participant, For data subset Sample size, x his the data subset d k The data sample in y h For x h The corresponding label;

[0116] Teacher model parameters The update formula is as follows:

[0117]

[0118] Among them, η T is the learning rate, For L T for gradient.

[0119] In an embodiment of the present application, in step S202, the teacher model of each participant classifies the data samples from the server and counts the votes obtained by the category. It should be understood that adding noise when counting the votes can further protect the privacy of the participants and ensure that the classification results of a single participant will not have too much impact on the global results, thereby meeting the requirements of differential privacy. By allowing all participants to classify the synthetic data samples, it can be ensured that each participant contributes to the training of the global model, which helps to achieve fairness in the model training process. The teacher models of different participants may have different characteristics and advantages. By aggregating the classification results of multiple models, the robustness of the global model can be improved and the reliance on the weaknesses of a single model can be reduced. By counting the votes, it is possible to identify which categories of samples are prone to misclassification, so that these categories can be optimized in subsequent training to improve the overall performance of the model.

[0120] In the embodiment of the present application, step S203 includes: when aggregating the voting results of the teacher model, the votes of each category are subjected to noise processing, and the process satisfies differential privacy and generates the voting results of the sample

[0121]

[0122] in, is the Laplace noise added to the voting result of the f-th category of participant i. λ is the scale parameter. The larger λ is, the greater the noise intensity is.

[0123] In the embodiment of the present application, in step S204, each participant sends the aggregated noisy voting results to the server. It should be understood that by adding noise to the voting results, the privacy of the participants can be protected. This technology is often related to differential privacy, which protects the data of individuals or small groups from being identified by adding sufficient randomness to ensure that the impact of a single data point on the overall result is minimized. In federated learning, security is a core consideration. Noisy voting results reduce the possibility of malicious participants or external attackers inferring sensitive information by analyzing the voting results.

[0124] In the embodiment of the present application, step S205 is used to: i Expand to The enlarged dataset is represented as Each participant uses KL divergence to calculate the difference between the original data distribution and the expanded data distribution, and calculates the data expansion amount ΔD by taking the difference i , then KL divergence and ΔD i Upload to the server, the formula is as follows:

[0125]

[0126] Among them, n i is the original data volume of the ith participant, is the amount of new data after participant i adds the synthetic data, D i is the original local dataset of the ith participant, The new dataset after adding the generator’s synthetic data for the i-th participant, ΔD i is the data expansion amount of the i-th participant, ΔD i ≥0, x represents the original data sample of participant i, represents the data sample of participant i after expansion, and X represents all data samples, that is,

[0127] In the embodiment of the present application, in step S206, each participant uses an expanded data set to train the initialized model from the server side. It should be understood that in federated learning, it is very important to protect the original data of the participants from being leaked. By using an expanded data set (including synthetic data and original data), the model can be trained without directly sharing the original data, thereby protecting data privacy. The expanded data set can increase the diversity of the data set and help the model learn a wider range of features and patterns. This is crucial to improving the generalization ability of the model and its robustness on different data distributions. The expanded data set can provide more training samples, which helps to improve the performance of the model, especially when the amount of data is small. More data can reduce overfitting and improve the stability and accuracy of the model. Using an expanded data set can ensure that the data of all participants have the opportunity to affect the global model, which helps to ensure the fairness and representativeness of the model and avoid the excessive impact of the data of some participants on the model.

[0128] Specifically, in the scenario of financial risk assessment, the data owned by third- and fourth-tier cities or county-level banks are different from those of first- and second-tier cities or provincial-level banks in terms of scale and distribution. In the absence of mutual data sharing, a financial risk assessment system suitable for cities at all levels can be trained through federated learning to achieve risk assessment of potential customers. The specific implementation process is as follows:

[0129] In the first step, the head office (trusted third party) is responsible for initializing a generator model G. The initial version of the generator generates some synthetic data samples through random noise.

[0130] In the second step, county-level banks and provincial / municipal-level banks randomly select a portion of their own data to form a training subset, and each bank independently trains its own teacher model T on the local data subset.

[0131] In the third step, the server's generator G generates a batch of financial data samples, which are sent to the teacher models of county-level banks and provincial / municipal banks. The teacher models of each bank classify these synthetic samples, for example, classifying the samples as low-risk or high-risk customers. To ensure privacy protection, noise is added to the voting results of the teacher model. The county-level banks and provincial / municipal banks then send the voting results back to the head office server.

[0132] In the fourth step, the server uses the noisy voting results to label the data synthesized by the generator. The server uses the labeled data to train the student model S. The generator continuously adjusts its parameters based on the classification results of the student model to generate more realistic synthetic financial data.

[0133] Step 5: After the generator is able to synthesize data stably, the head office will optimize the generator G *Send to each bank. County-level banks and provincial / municipal-level banks use the generator to generate more synthetic financial data based on their own data distribution. Calculate the number of synthetic data samples for each bank, the KL divergence between the expanded data set and the original data set, and send them to the head office.

[0134] In the sixth step, the head office sends the initialized model to each bank, and each bank uses the expanded data set to train the model. The head office assigns each bank a different probability of participating in the training based on the KL divergence uploaded by each bank. The smaller the KL divergence, the higher the selection probability, which means that banks with more accurate synthetic data will participate more in model training. Each bank has a training threshold, and the threshold is reduced by 1 each time it participates in training. When the threshold is 0, the bank's participation will be suspended and wait for the next round of reallocation.

[0135] In the seventh step, the head office updates the models uploaded by the banks based on their data expansion, assigns different aggregation weights to the uploaded model updates, aggregates the model updates of each bank, and forms a global model.

[0136] In the eighth step, through multiple iterations of subordinate banks and the head office server, the global model finally becomes stable, thus ensuring the service quality of the financial risk assessment system.

[0137] The embodiments of the present disclosure have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A server-side federated learning fair model aggregation method based on secure data synthesis, characterized in that: include: S101. In the server, the generator model is trained by the student model S; S102. The server sends the data sample generated by the generator to all participants; S103. The server receives the aggregated noisy voting results from the participants and uses them to annotate a public data set; S104. Use the data set to train the student model S, and train the generator through the classification results of the student model S; S105. The server sends the optimized trained generator to the participants; S106. The server receives the data expansion amount and the KL divergence between the expanded data set and the original data set from each participant, and assigns each participant a different training participation probability according to the KL divergence; S107. The server assigns different aggregation weights to the uploaded model updates based on the model updates uploaded by each participant and the data expansion amount of each participant, aggregates the model updates of each participant, and forms a global model; S108. The global model is stabilized through multiple iterations of participants and the server.

2. The method for aggregating a fair model of federated learning based on secure data synthesis on the server side according to claim 1, characterized in that: The step S101 includes: in the server, training the generator model through the student model S, and the student model optimization function L G for: Among them, m is the number of samples generated by the generator in the pre-training stage, j∈m, S(G(z j θ G )) is the classification probability of the student model for the samples generated by the generator, G(z j θ G ) uses random noise z for the generator j Generated samples; Generator parameter θ G The update formula is as follows: Among them, η G is the learning rate, YesL G For θ G gradient.

3. The method for aggregating a fair model of federated learning based on secure data synthesis on the server side according to claim 2, characterized in that: In step S102, the server sends the m data samples generated by the generator to all participants, and the teacher model of each participant For data sample g from the server j Classify and count the categories f(f∈c i ) The number of votes received, the number of votes The calculation formula is as follows: Among them, d is the number of disjoint subsets divided into the original data set, is the indicator function. When the k-th teacher model of participant i converts sample g j When the classification is category f, the indicator function takes the value of 1, otherwise it is 0.

4. The method for aggregating a fair model of federated learning based on secure data synthesis at the server end according to claim 3 is characterized in that: In step S104, the server uses the aggregated noisy voting results to annotate a public data set, where the data set consists of samples generated by the generator, uses the annotated data to train a student model S, and uses the classification results of the student model S to train the generator to optimize the function L S for: Among them, g j Generate samples for the generator, is the noisy label aggregated by the teacher model; Student model parameters θ S The update formula is as follows: Among them, η S is the learning rate, YesL S For θ S gradient.

5. The method for aggregating a fair model of federated learning based on secure data synthesis at the server end according to claim 4, characterized in that: The step S106 is used for: the server assigns a selection probability p to each participant according to the KL divergence uploaded by the participant. i , and set a training number threshold ∈. Each time a participant participates in a training, the threshold τ i Minus one, when τ i If it is still selected when it decreases to 0, it will not participate in the training in this round, and the server will reselect, τ i Add one, and the formula is as follows: Among them, p i is the selection probability of the ith participant, KL i D is the original data distribution of participant i i (x) and the expanded data distribution , R is the set of participants selected to upload model updates in each round, τ is the threshold of the number of times a participant participates in training, τ ≥ 0.

6. The method for aggregating a fair model of federated learning based on secure data synthesis at the server end according to claim 5, characterized in that: In step S107, after receiving the model update uploaded by the participant, the server aggregates the model update, and the aggregation parameter is based on the data expansion amount ΔD of the participant. i To distribute, the global model aggregation formula and aggregation weight calculation formula are: Among them, h i is the aggregate weight of the i-th selected participant, w t+1 is the global model for the t+1th round, is the model updated by the i-th participant in round t+1.

7. A method for aggregating a fair model of federated learning based on secure data synthesis by participants, characterized in that: include: S201. Divide the local data into data subsets and train the teacher model; S202. The teacher model of each participant classifies the data samples from the server and counts the votes obtained by the categories; S203. When aggregating the voting results of the teacher model, the number of votes for each category is subjected to noise processing; S204. Each participant sends the aggregated noisy voting results to the server; S205. Each participant receives the generator model sent from the server, synthesizes data samples according to their respective data distributions, and sends the data expansion amount and the KL divergence between the expanded data set and the original data set to the server; S206. Each participant uses the expanded data set to train the initialization model from the server.

8. The method for aggregating a fair model of federated learning based on secure data synthesis according to claim 7, characterized in that: The step S201 includes: the participant i (i∈I) divides the local data into d disjoint data subsets, each data subset Train a teacher model Teacher model optimization function L T for: Where I is the set of participants, i is the i-th participant, i∈I, d is the number of disjoint subsets of the original data set, The kth data subset divided by the i-th participant, For data subset Sample size, x h is the data subset d k The data sample in y h For x h The corresponding label; Teacher model parameters The update formula is as follows: Among them, η T is the learning rate, For L T for gradient.

9. The method for aggregating a fair model of federated learning based on secure data synthesis according to claim 8, characterized in that: The step S203 includes: when aggregating the voting results of the teacher model, performing noise processing on the votes of each category, which satisfies differential privacy and generates the voting results of the sample in, is the Laplace noise added to the voting result of the f-th category of participant i. λ is the scale parameter. The larger λ is, the greater the noise intensity is.

10. The method for aggregating a fair model of federated learning based on secure data synthesis according to claim 9, characterized in that: The step S205 is used to: expand the data volume from the original to The enlarged dataset is represented as Each participant uses KL divergence to calculate the difference between the original data distribution and the expanded data distribution, and calculates the data expansion amount ΔD by taking the difference i , then KL divergence and ΔD i Upload to the server, the formula is as follows: Among them, n i is the original data volume of the ith participant, is the amount of new data after participant i adds the synthetic data, D i is the original local dataset of the ith participant, The new dataset after adding the generator’s synthetic data for the i-th participant, ΔD i is the data expansion amount of the i-th participant, ΔD i ≥0, x represents the original data sample of participant i, represents the data sample of participant i after expansion, and X represents all data samples, that is,

Images