A federated learning fair model aggregation method based on secure data synthesis

Abstract

The application discloses a federated learning fair model aggregation method based on secure data synthesis. A student model is trained to guide the generation of a generator model. The data of participants is divided into multiple disjoint subsets and multiple teacher models are trained. Then, the data samples generated by the generator are classified and voted. The voting results are aggregated with noise to protect privacy. Then, the noisy aggregation results are used to label a data set and train a student model to further optimize the generator. After a certain number of optimization rounds, the generator is distributed to each participant. Finally, the aggregated model is sent to each participant for further training, thereby achieving fair optimization of the model while protecting data privacy.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of federated learning and privacy protection, and particularly to a federated learning fair model aggregation method based on secure data synthesis. BACKGROUND

[0002] Federated learning does not need to collect all the data of participants when training, unlike common machine learning, and only shares models to realize model optimization, which can well solve the data island problem caused by data inaccessibility. When some participants have poor data quality, the models trained by these participants will seriously affect the performance of the global model. In real-world scenarios, these participants with poor data quality do not mean that the data they have is not valuable. For example, in financial institutions, the data owned by banks and insurance companies in developed and underdeveloped regions are indeed different. If some participants with poor data quality are arbitrarily excluded from training, it is unfair. Expanding the local sample size through data synthesis can help participants with poor data quality solve this problem.

[0003] In the process of synthesizing data, the local data features of participants are needed, but federated learning does not allow participants to exchange information. If data synthesis is performed locally at the participants, the quality of the synthesized data will be difficult to guarantee. If the local data features of all participants are shared with the server to generate uniformly, there is a high risk of privacy leakage in the sharing process.

[0004] At present, there are many data synthesis techniques under the framework of federated learning to solve the problem of poor data quality of participants, but most of them do not pay attention to the risk of data leakage in the process of synthesizing data. At the same time, these methods do not evaluate the data quality of participants, that is, after synthesizing data, there is no distinction between participants when selecting participants for training, which is unfair to participants with good data quality.

[0005] Therefore, the problems of the prior art are that: 1) the existing data synthesis method does not consider the risk of data leakage, that is, if attacked when sharing data features with the server, the original data information is easily leaked; 2) the existing method randomly selects participants after data synthesis, which is unfair to participants who make great contributions.

[0006] Therefore, it is desirable to provide a federated learning fair model aggregation method based on secure data synthesis. SUMMARY

[0007] In order to solve the above technical problems, the present application is proposed. The embodiments of the present application provide a federated learning model optimization method based on secure data synthesis, which is used to improve the model quality under the premise of protecting the privacy of participants when there are participants with poor data quality in federated learning.

[0008] According to an aspect of the present application, a server-based federated learning fair model aggregation method based on secure data synthesis is provided, which comprises: S101. In the server, a generator model is trained by a student model S; S102. The server sends the data samples generated by the generator to all participants; S103. The server receives the aggregated noisy voting results from the participants, which are used to label a public data set; S104. The student model S is trained using the data set, and the generator is trained through the classification results of the student model S; S105. The server sends the optimized generator to the participants; S106. The server receives the data expansion amount and the KL divergence of the expanded data set and the original data set from each participant, and gives each participant different training participation probability according to the KL divergence; S107. The server gives different aggregation weights to the uploaded model updates of each participant according to the model updates uploaded by each participant, aggregates the model updates of each participant, and forms a global model; S108. Through multiple iterations of the participants and the server, the stability of the global model is realized.

[0009] The federated learning fair model aggregation method based on secure data synthesis provided by the embodiments of the present application realizes secure participant data expansion and fair server model aggregation, which not only protects the privacy of the participants when synthesizing data, but also optimizes the performance of the global model by reasonably allocating aggregation weights according to data quality and expansion amount, ensures the fairness to all participants, and thus provides long-term and high-quality federated learning services.

[0010] Preferably, the step S101 comprises: training a generator model in the server by a student model S, and the student model optimization function L G is:

[0011]

[0012] wherein m is the number of samples generated by the generator in the pre-training stage, j e m, S(G(z j ; θ G )) is the classification probability of the student model for the sample generated by the generator, G(z j ; θ G ) is the sample generated by the generator using random noise z j ;

[0013] The generator parameter θ G is updated according to the following formula:

[0014]

[0015] wherein η G is the learning rate, is LG For θ G , the gradient of the loss function.

[0016] Preferably, the step S102, the server sends the m data samples generated by the generator to all participants, and each participant's teacher model classifies the data samples g j from the server, and counts the votes of the categories f (f∈c i ) obtained, and the votes are calculated as follows:

[0017]

[0018] Where d is the number of disjoint subsets of the original data set, is an indicator function, which takes the value 1 when the kth teacher model of the participant i classifies the sample g j into category f, otherwise 0.

[0019] Preferably, the step S104, the server uses the aggregated noisy voting results to label a public data set, which is composed of samples generated by the generator, and trains a student model S using the labeled data, and trains the generator through the classification results of the student model S, and the optimization function L S is:

[0020]

[0021] Where g j is the sample generated by the generator, is the noisy label aggregated by the teacher model;

[0022] The update formula of the student model parameter θ S is as follows:

[0023]

[0024] Where η S is the learning rate, is L s for θ S , the gradient of the loss function.

[0025] Preferably, the step S106, for: the server gives each participant a selection probability p i according to the KL divergence uploaded by the participant, while setting a participation threshold τ, and each time the participant participates in training, the threshold τ i of the participant is reduced by one, and when τ i is reduced to 0, if it is still selected, it does not participate in this round of training, the server reselects, and τ i is increased by one, and the formula is as follows:

[0026]

[0027] where p i is the selection probability of the i-th participant, KL i is the KL divergence between the original data distribution D i (x) of the participant i and the expanded data distribution , R is the set of participants selected to upload model updates in each round, and τ is the threshold of the number of times of participation in training of the participant, τ ≥ 0.

[0028] Preferably, the step S107, after receiving the model updates uploaded by the participants, the server aggregates the model updates, and the aggregation parameters are allocated according to the data expansion amount ΔD i of the participants, and the global model aggregation formula and the aggregation weight calculation formula are as follows:

[0029]

[0030] where h i is the aggregation weight of the i-th selected participant, w t+1 is the global model in the t+1th round, is the model updated in the t+1th round by the i-th participant.

[0031] According to an aspect of the present application, a federated learning fair model aggregation method based on secure data synthesis of a participant is also provided, which comprises the following steps: S201. dividing local data into data subsets and training a teacher model; S202. the teacher model of each participant classifies data samples from a server and counts the votes obtained by each class; S203. when the voting results of the teacher model are aggregated, the votes of each class are processed with noise; S204. each participant sends the aggregated noisy voting results to the server; S205. each participant receives a generator model sent from the server, synthesizes data samples according to the respective data distribution, and sends the KL divergence between the data expansion amount and the expanded data set and the original data set to the server; S206. each participant trains an initialization model from the server end using the expanded data set.

[0032] Preferably, the step S201 comprises: dividing, by a participant i (i ∈ I), local data into d disjoint data subsets, and training a teacher model for each data subset The teacher model optimization function L T is as follows:

[0033]

[0034] Wherein, I is a set of participants, i is the ith participant, i∈I, d is the number of original data set partition disjoint subsets, is the kth data subset partitioned for the ith participant, is the data subset sample size, x h is the data subset d k data sample in, y h is x h corresponding label;

[0035] teacher model parameters The update formula is as follows:

[0036]

[0037] The number of votes is added to the noise, and the process meets the differential privacy, and the voting result of the sample is generated

[0038]

[0039] Wherein, is the Laplace noise added by the ith participant's fth voting result, λ is the scale parameter, and the larger the noise intensity is.

[0040] Preferably, the step S205 is used for: expanding the data size from the original n i to The expanded data set is represented as Each participant calculates the difference between the original data distribution and the expanded data distribution by KL divergence, and calculates the data expansion ΔD by difference i , then upload the KL divergence and ΔD i to the server, the formula is as follows:

[0041]

[0042] Wherein, n i is the original data size of the ith participant, is the new data size of the participant i after adding synthetic data, D i is the original local data set of the ith participant, is the new data set of the ith participant after adding generator synthetic data, ΔD i is the data expansion of the ith participant, ΔD i ≥0, x represents the original data sample of the participant i, represents the expanded data sample of the participant i, X represents all data samples, that is

[0043] The application is used for improving model quality under the premise of protecting participant privacy in the case that there is poor participant data quality in federated learning. In addition, the application does not ignore the problem of fairness, and gives higher aggregation weight to participants with good local data when aggregating the model updates uploaded by the participants, which also reduces the negative impact of synthetic data on model updates to a certain extent. Specifically, the beneficial technical effects brought by the application are:

[0044] Safe participant data expansion effect: the application avoids the problem of privacy leakage when participants synthesize data, and uses the PATE-GAN mechanism before federated learning training in the scene of federated learning data without model movement. The generator is trained by the participants and the server to protect the privacy of the participants, and the participants use the generator to synthesize data to expand the size of the local data.

[0045] Fair server aggregated model effect: the application effectively avoids the problem of poor global model effect caused by participants with poor data quality, and trains the best global model by giving participants selection probability according to original data quality and giving update model aggregation weight according to data expansion amount in the scene of federated learning server aggregated update model, so as to provide long-term high-quality service for participants. BRIEF DESCRIPTION OF DRAWINGS

[0046] The above and other objects, features and advantages of the present application will become more apparent from the following detailed description of embodiments of the present application, taken in conjunction with the accompanying drawings. The drawings provided in the specification and the embodiments of the present application together serve to explain the present application and, therefore, should be considered a part of this specification. In the drawings, like reference numerals refer to like elements or steps throughout.

[0047] Figure 1 A flowchart of the federated learning fair model aggregation method based on safe data synthesis according to the embodiments of the present application. DETAILED DESCRIPTION

[0048] Various exemplary embodiments, features and aspects of the present application will be described in detail below with reference to the accompanying drawings. The same reference numerals in the drawings represent functionally the same or similar elements. Although various aspects of the embodiments are shown in the drawings, the drawings are not necessarily drawn to scale unless specifically indicated.

[0049] The word "exemplary" is used herein in the sense of being an example, instance, or illustration. Any implementation described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other implementations.

[0050] In addition, for the purpose of better illustrating the present application, numerous specific details are set forth in the following detailed description. One skilled in the art will appreciate that the application can be practiced without the specific details, or with an equivalent alternative. In some instances, well-known methods, structures, elements, and circuits have not been described in detail in order to avoid obscuring the application.

[0051] Furthermore, the terms "first", "second", etc. are used herein only to describe the different instances of an element, and do not imply or suggest relative importance of the elements or a number of the elements indicated. Thus, the features defined with "first", "second", etc. can include one or more of the features explicitly or implicitly. In the description of the present application, the meaning of "a plurality" is two or more, unless otherwise specifically limited.

[0052] Each participant in the federated learning service is bounded rationality and cannot have the data of other participants, that is, they have privacy protection awareness and cannot effectively evaluate the quality of participant data. As the training time increases, the local model trained by the participant with poor data quality will affect the performance of the global model. At the same time, if the server randomly selects the participants to train, it is obviously unfair to the participants with good data quality.

[0053] Therefore, before starting the federated learning training, the server and the participants cooperate to train a generative adversarial network (GAN), and the data synthesis is performed through the generator in the GAN to make up for the problem of the existence of the participant with poor data quality. The participant expands the local data scale by synthesizing new data through the generator, which reduces the adverse effects for the participant with poor data quality, and expands the data quantity for the participant with good data quality, achieving a win-win situation. After starting the federated learning training, the server does not randomly select the participants to train, but gives different selection probabilities to each participant according to the initial data quality. At the same time, the server cannot blindly select the participants with good data quality, but sets a unified training threshold for each participant, and prohibits the participant from participating in the current training when the training times exceed the threshold, selects the participants whose training times are less than the threshold, and avoids the phenomenon of model overfitting.

[0054] During the training process, the participants in the federated learning service need to send the model updated after the local data training to the server. The server gives different weights to each participant's model update when aggregating the model updates, so that the training process is more fair.

[0055] For each participant, the PATE-GAN mechanism helps the server train the generator without sharing the data distribution, and then each participant generates a certain number of data samples through the generator, which can greatly improve the global model performance. At the same time, the PATE-GAN mechanism meets differential privacy and does not leak privacy in the data synthesis stage. For the server, each participant will be given different selection probabilities according to the quality of the original data, and a unified training threshold will be set for the participants to prevent some participants from participating in training too much. When aggregating model updates, different aggregation weights will be given according to the data volume of each participant.

[0056] Therefore, this scheme is designed based on the scenario that there are participants with poor data quality. Specifically, some parameters are defined as follows:

[0057] I: set of participants;

[0058] T: number of training rounds;

[0059] R: set of participants selected to upload model updates in each round;

[0060] N: total number of participants;

[0061] i: the ith participant, i∈I;

[0062] p i : selection probability of the ith participant;

[0063] w0: initial global model;

[0064] w t : global model in the tth round;

[0065] updated model of the ith participant in the t+1th round;

[0066] D i : original local data set of the ith participant;

[0067] D i (x): data distribution of D i ;

[0068] n i : original data volume of the ith participant;

[0069] d: number of disjoint subsets of the original data set;

[0070] kth data subset divided by the ith participant;

[0071] c i : local data category of participant i;

[0072] m: The number of samples generated by the generator during the pre-training phase;

[0073] g j : Training samples generated by the generator during the pre-training phase, j∈m;

[0074] G(z j ;θ G The generator uses random noise z. j The generated sample, i.e., g j ;

[0075] S(G(z j ;θ G ): The classification probability of the student model for the samples generated by the generator;

[0076] S(g j ): Student model on sample g j The classification probability;

[0077] T k (g j Teacher model for sample g j The classification results;

[0078] Laplace noise was added to the voting results of participant i in class f.

[0079] The new dataset after the i-th participant synthesizes data through the generator;

[0080] Data distribution;

[0081] The amount of new data after participant i adds the synthetic data;

[0082] ΔD i : The amount of data augmentation for the i-th participant, ΔD i ≥0;

[0083] KL i : Original data distribution of participant i D i (x) and the expanded data distribution KL divergence;

[0084] τ: Threshold for the number of times a participant participates in training, τ≥0;

[0085] h i : The aggregate weight of the i-th selected participant;

[0086] Therefore, a federated learning fairness model aggregation method based on data synthesis, such as... Figure 1As shown, the basic flow is as follows. According to the federated learning fairness model aggregation method based on data synthesis of the embodiment of the present application, the server side comprises: S101. In the server, a generator model is trained by a student model S; S102. The server sends the data samples generated by the generator to all participants; S103. The server receives the aggregated noisy voting results from the participants, which are used to label a public data set; S104. The student model S is trained using the data set, and the generator is trained through the classification result of the student model S; S105. The server sends the optimized generator to the participants; S106. The server receives the data expansion amount and the KL divergence of the expanded data set and the original data set from each participant, and gives each participant different training participation probability according to the KL divergence; S107. The server combines the model updates uploaded by each participant according to the model updates uploaded by each participant, and gives different aggregation weights to the uploaded model updates, aggregates the model updates of each participant, and forms a global model; S108. Through multiple iterations of the participants and the server, the stability of the global model is realized.

[0087] In the embodiment of the present application, the step S101 comprises: in the server, training a generator model by a student model S, and the student model optimization function L G is:

[0088]

[0089] wherein m is the number of samples generated by the generator in the pre-training stage, j e m, S(G(z j ; θ G ) is the classification probability of the student model for the sample generated by the generator, G(z j ; θ G ) is the sample generated by the generator using random noise z j .

[0090] The generator parameter θ G is updated as follows:

[0091]

[0092] wherein η G is the learning rate, is the gradient of L G for θ G .

[0093] In the embodiment of the present application, the step S102, the server sends m data samples generated by the generator to all participants, and the teacher model of each participant classifies the data samples g j from the server, and counts the categories f (f e c iThe obtained vote number, vote number The calculation formula is as follows:

[0094]

[0095] Wherein, d is the number of disjoint subsets of the original data set, is an indicator function, when the kth teacher model of the participant i classifies the sample g j into category f, the indicator function takes the value 1, otherwise 0.

[0096] In the embodiment of the application, the step S103, the server receives the aggregated noisy vote result from the participants, which is used to label a public data set. It should be understood that in federated learning, it is very important to protect the data privacy of the participants. By adding noise to the voting result, it can prevent any party from identifying the original data, so as to protect the data of the participants from being leaked. This method conforms to the principle of differential privacy, which allows the collective statistical characteristics of data to be utilized without leaking individual data. The labeled public data set can be used to train a student model S. The classification result of the student model S will be fed back to the generator G to help the generator G learn to generate more realistic data samples. This process is iterative, and as the generator G is continuously optimized, the quality of the generated data samples will improve, thereby improving the performance of the entire federated learning system.

[0097] In the embodiment of the application, the step S104, the server uses the aggregated noisy vote result to label a public data set, the data set is composed of samples generated by the generator, a student model S is trained using the labeled data, the generator is trained through the classification result of the student model S, and the optimization function L S is:

[0098]

[0099] Wherein, g j is the sample generated by the generator, is the noisy label aggregated by the teacher model;

[0100] The student model parameter θ S is updated as follows:

[0101]

[0102] Wherein, η S is the learning rate, is the gradient of L S with respect to θ S .

[0103] In this embodiment, in step S105, the server sends the optimized generator to the participants. It should be understood that in federated learning, ensuring all participants contribute fairly to the training of the global model is crucial. By sending the optimized generator, it is ensured that all participants can generate high-quality synthetic data, thus playing a greater role in model training. Federated learning is an iterative process, and generator optimization is also iterative. By sending the optimized generator to the participants, it is ensured that the latest generator is used in the next round of training, thereby continuously improving the quality of the synthetic data.

[0104] In this embodiment of the application, step S106 is used to: assign a selection probability p to each participant based on the KL divergence uploaded by the participants. i At the same time, a threshold τ is set for the number of times a participant can participate in training. Each time a participant participates in training, the threshold τ is reduced. i Decrease by one, when τ i If a value is reduced to 0 and the selected element is still chosen, it will not participate in training in this round, and the server will reselect. i Add one, the formula is as follows:

[0105]

[0106] Where, p i Let KL be the selection probability of the i-th participant. i For participant i, the original data distribution D i (x) and the expanded data distribution The KL divergence is given by R, where R is the set of participants selected to upload model updates in each round, and τ is the threshold for the number of times a participant participates in training, where τ ≥ 0.

[0107] In this embodiment of the application, in step S107, after receiving the model update uploaded by the participant, the server aggregates the model update, and the aggregation parameter is based on the data expansion amount ΔD of the participant. i The allocation is performed using the global model aggregation formula and the aggregation weight calculation formula:

[0108]

[0109] Among them, h i For the aggregate weight of the i-th selected participant, w t+1 For the (t+1)th round global model, This is the model updated by the i-th participant in round t+1.

[0110] In the embodiments of the present application, the step S108 realizes the stability of the global model through multiple iterations of the participants and the server. It should be understood that in federated learning, protecting the data privacy of participants is one of the core principles. Through the iterative process, the global model can be gradually optimized without directly sharing data, thereby improving the model performance while protecting privacy. Through multiple iterations, the data of all participants has the opportunity to influence the global model, which helps to ensure the fairness and representativeness of the model and avoid the data of some participants having too much influence on the model. In federated learning, due to the bias or noise of the data of the participants, multiple iterations provide an opportunity to correct these errors, and through continuous model updating and aggregation, the influence of the data problems of a single participant on the global model can be reduced. After each iteration, the model performance can be tested on the validation set to ensure that the model has not overfitting and has good generalization ability. If the performance does not meet the expectation, the model parameters or learning rate can be adjusted and then the iteration is continued. In federated learning, the global model is the aggregation of the models of all participants. Multiple iterations help to find a balance point so that the global model can reflect the data characteristics of all participants and achieve global consensus.

[0111] Further, the participants include: S201. dividing the local data into data subsets and training a teacher model; S202. the teacher model of each participant classifies the data samples from the server and counts the votes obtained by each class; S203. when aggregating the voting results of the teacher model, the votes of each class are subjected to noise processing; S204. each participant sends the aggregated noisy voting results to the server; S205. each participant receives the generator model sent from the server and synthesizes data samples according to the respective data distribution, and sends the KL divergence of the data expansion amount, the expanded data set and the original data set to the server; S206. each participant trains the initialization model from the server end using the expanded data set.

[0112] In the embodiments of the present application, the step S201 includes: a participant i (i∈I) divides the local data into d disjoint data subsets, each data subset Training a teacher model Teacher model optimization function L T is:

[0113]

[0114] wherein I is a participant set, i is the i th participant, i∈I, d is the number of disjoint subsets of the original data set, is the k th data subset divided by the i th participant, is the data subset sample size, x hFor data samples in data subset d k h For x h Corresponding labels.

[0115] Teacher model parameters The update formula is as follows:

[0116]

[0117] Wherein, η T is the learning rate, is L T For the gradient of

[0118] In the embodiments of the present application, in the step S202, the teacher model of each participant classifies the data samples from the server and counts the votes of the categories obtained. It should be understood that noise is added when counting the votes, which can further protect the privacy of the participants and ensure that the classification results of a single participant do not have a large impact on the global results, thereby meeting the requirements of differential privacy. By allowing all participants to classify the synthetic data samples, it can be ensured that each participant contributes to the training of the global model, which helps to achieve fairness in the model training process. The teacher models of different participants may have different characteristics and advantages. By aggregating the classification results of multiple models, the robustness of the global model can be improved, and the dependence on the weaknesses of a single model can be reduced. By counting the votes, it can be identified which categories of samples are prone to misclassification, so that these categories can be optimized in subsequent training to improve the overall performance of the model.

[0119] In the embodiments of the present application, in the step S203, the votes of each category are added with noise when the voting results of the teacher model are aggregated, and the process meets the differential privacy, to generate the voting result of the sample

[0120]

[0121] Wherein, is the Laplace noise added to the voting result of the i-th participant in the f-th category, and λ is the scale parameter. The larger the λ, the stronger the noise.

[0122] ​​In the embodiment of the present application, in the step S204, each participant sends the aggregated noisy voting result to the server. It can be understood that by adding noise to the voting result, the privacy of the participants can be protected. This technique is often related to differential privacy, which adds enough randomness to ensure that the influence of a single data point on the overall result is minimized, thereby protecting the data of individuals or small groups from being identified. In federated learning, security is a core consideration. Noisy voting results reduce the possibility of malicious participants or external attackers inferring sensitive information by analyzing the voting results.

[0123] In the embodiment of the present application, in the step S205, the data volume is expanded from the original n i to n The expanded data set is represented as Each participant calculates the difference between the original data distribution and the expanded data distribution by KL divergence, and calculates the data expansion amount ΔD i by difference, and then uploads the KL divergence and ΔD i to the server, as follows:

[0124]

[0125] Where n i is the original data volume of the i-th participant, is the new data volume of the participant i after adding synthetic data, D i is the original local data set of the i-th participant, is the new data set of the i-th participant after adding synthetic data generated by the generator, ΔD i is the data expansion amount of the i-th participant, ΔD i ≥ 0, x represents the original data sample of the participant i, represents the expanded data sample of the participant i, and X represents all data samples, i.e.

[0126] In the embodiments of the present application, in the step S206, each participant trains the initial model from the server side using the augmented dataset. It should be understood that in federated learning, it is very important to protect the original data of the participants from being leaked. By using the augmented dataset (including synthetic data and original data), the model can be trained without directly sharing the original data, thereby protecting data privacy. The augmented dataset can increase the diversity of the dataset, helping the model to learn more extensive features and patterns. This is crucial for improving the generalization ability of the model and the robustness on different data distributions. The augmented dataset can provide more training samples, which helps to improve the performance of the model, especially in the case of small amount of data. More data can reduce overfitting, improve the stability and accuracy of the model. Using the augmented dataset can ensure that the data of all participants has the opportunity to influence the global model, which helps to ensure the fairness and representativeness of the model, and avoids the data of some participants having too much influence on the model.

[0127] Specifically, in the scenario of financial risk assessment, the data owned by the third and fourth-tier city banks or county-level banks has a gap in scale and distribution compared to the first and second-tier city banks or provincial banks. In the case of not sharing data with each other, a financial risk assessment system suitable for banks at different levels of cities is trained through federated learning to realize potential customer risk assessment. The specific implementation process is as follows:

[0128] First, the head office (trusted third party) is responsible for initializing a generator model G. The initial version of the generator generates some synthetic data samples through random noise.

[0129] Second, the county-level banks and provincial / city-level banks randomly select a part of their data to form a training subset. Each bank independently trains its own teacher model T on the local data subset.

[0130] Third, the generator G of the server generates a batch of financial data samples, which are sent to the teacher models of the county-level banks and provincial / city-level banks. The teacher models of each bank classify these synthetic samples, for example, classifying the samples into low-risk, high-risk customers, etc. In order to ensure privacy protection, noise is added to the voting results of the teacher models. Then the county-level banks and provincial / city-level banks send the voting results back to the head office server.

[0131] Fourth, the server uses the noisy voting results to label the data generated by the generator. The server trains a student model S using the labeled data, and the generator adjusts its parameters according to the classification results of the student model to generate more realistic synthetic financial data.

[0132] Fifth, after the generator can stably generate data, the head office will optimize the generator G *The synthetic financial data is sent to each bank. The county-level banks and the provincial / city-level banks respectively generate more synthetic financial data using the generator according to their own data distribution. The number of synthetic data samples of each bank, the KL divergence between the expanded data set and the original data set are calculated respectively, and are sent to the head office.

[0133] In the sixth step, the head office sends the initialized model to each bank, and each bank uses the expanded data set to train the model. The head office gives each bank a different training participation probability according to the KL divergence uploaded by each bank. The smaller the KL divergence, the higher the selection probability, which means that the banks with more accurate synthetic data will participate more in the model training. Each bank has a training frequency threshold, and the threshold decreases by 1 each time it participates in the training. When the threshold is 0, the participation of the bank will be suspended, and the next round of redistribution will be waited.

[0134] In the seventh step, the head office gives different aggregation weights to the uploaded model updates according to the data expansion of each bank, aggregates the model updates of each bank, and forms a global model.

[0135] In the eighth step, through multiple iterations between the subordinate banks and the head office server, the global model finally reaches stability, which ensures the service quality of the financial risk assessment system.

[0136] The above has described various embodiments of the present disclosure, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes are obvious to those skilled in the art without departing from the scope and spirit of the described embodiments. The choice of terms used herein is intended to best explain the principles, practical applications, or improvements to the technology in the market of the embodiments, or to enable other ordinary skilled persons in the art to understand the embodiments disclosed herein.

Claims

1. A federated learning fair model aggregation method based on secure data synthesis of a service end, characterized in that, Comprising: S101. In the server, a generator model is trained by a student model S; S102. The server sends the data samples generated by the generator to all participants; S103. The server receives the aggregated noisy voting results from the participants, which are used to label a public data set; S104. The student model S is trained using the data set, and the generator is trained through the classification results of the student model S; S105. The server sends the generator after optimization training to the participants; S106. The server receives the data expansion amount and the KL divergence of the expansion data set and the original data set from each participant, and gives each participant different training participation probability according to the KL divergence; S107. The server gives different aggregation weights to the uploaded model updates according to the data expansion amount of each participant, aggregates the model updates of each participant, and forms a global model; S108. Through multiple iterations between the participants and the server, the stability of the global model is realized.

2. The federated learning fair model aggregation method based on secure data synthesis of the server according to claim 1, characterized in that, The step S101 comprises: in the server, training a student model by a student model optimization function training the generator model, the student model optimization function is: wherein, a number of samples generated for the pre-training phase generator, , a classification probability of the student model for the samples generated by the generator, for the generator to use the generated samples; Generator parameters The update formula is as follows: wherein, is the learning rate, is for the gradient.

3. The federated learning fair model aggregation method based on secure data synthesis of the server according to claim 2, characterized in that, In step S102, the server generates the generator. A data sample is sent to all participants, and each participant's teacher model... Data samples from the server Classify and count the categories. The number of votes received , As a participant Local data categories, votes The calculation formula is as follows: wherein, is the number of disjoint subsets of the original dataset, is an indicator function, when a participant the first teacher model classifies a sample into class , the indicator function takes the value 1, otherwise 0.

4. The federated learning fair model aggregation method based on secure data synthesis of the server according to claim 3, characterized in that, The step S104 is that the server uses the aggregated noisy voting result to label a public data set, the data set is composed of sample groups generated by the generator, and a student model is trained using the labeled data The generator is trained through a classification result of the student model , and the optimization function is wherein, generating samples for the generator, is the aggregated noisy label by the teacher model; Student model parameters The update formula is as follows: wherein, is the learning rate, is for the gradient.

5. The federated learning fair model aggregation method based on secure data synthesis of the server according to claim 4, characterized in that, The step S106 is used for giving each participant a selection probability according to the KL divergence uploaded by the participant. Meanwhile, a threshold of the number of training times is set, and the threshold of the number of training times of the participant is reduced by one each time the participant participates in the training. When the threshold of the number of training times is reduced to zero, if the participant is still selected, the participant does not participate in the training in this round, and the server reselects, When the threshold of the number of training times is reduced to zero, if the participant is still selected, the participant does not participate in the training in this round, and the server reselects, and the threshold of the number of training times of the participant is increased by one, and the formula is as follows: wherein, is the selection probability of the th participant is the number of participants is the original data distribution is the data distribution after expansion is the divergence of the is the set of participants selected to upload model updates in each round, is the number of participants is the threshold of the number of participable training times of the th participant.

6. The federated learning fair model aggregation method based on secure data synthesis of the server according to claim 5, characterized in that, The step S107, after receiving the model update uploaded by the participant, the server aggregates the model update, and the aggregation parameter is according to the data expansion amount of the participant The global model aggregation formula and the aggregation weight calculation formula are as follows: wherein, is the aggregated weight of the th selected participant, is the aggregated weight of the th participant, is the global model after the th round of updates.

7. A method for federated learning fair model aggregation based on secure data synthesis of participants, characterized in that, Comprising: S201. Divide the local data into data subsets and train a teacher model; S202. The teacher model of each participant classifies the data samples from the server and counts the votes of each class; S203. When aggregating the voting results of the teacher model, the votes of each class are added with noise; S204. Each participant sends the aggregated noisy voting results to the server; the server receives the aggregated noisy voting results from the participants, which are used to label a public data set, trains a student model S using the data set, and trains a generator through the classification results of the student model S, and sends the generator after optimization training to the participants; S205. Each participant receives the generator model sent by the server, synthesizes data samples according to their own data distribution, sends the data expansion amount, the KL divergence of the expansion data set and the original data set to the server; the server gives each participant different training participation probability according to the KL divergence; S206. Each participant trains the initialization model from the server using the expansion data set, the server gives different aggregation weights to the uploaded model updates according to the data expansion amount of each participant, aggregates the model updates of each participant, forms a global model, and realizes the stability of the global model through multiple iterations between the participants and the server.

8. The method of claim 7, wherein the federated learning fair model aggregation method based on security data synthesis of the participant is characterized in that, The step S201 comprises: dividing the local data into data subsets, each data subset training a teacher model , the teacher model optimization function is: in, For the first Each participating party The number of disjoint subsets to divide the original dataset. For the gathering of participants, For the first The first division of the participants A subset of data, | | for data subsets Sample size For data subset Data samples in for Corresponding tags; Teacher model parameters The update formula is as follows: wherein, is the learning rate, is for the gradient.

9. The method of claim 8, wherein the federated learning fair model aggregation method based on secure data synthesis of the participant is characterized in that, The step S203 comprises: in the process of aggregating the voting results of the teacher model, adding noise to the votes of each category, and the process satisfies differential privacy, to generate the voting result of the sample : wherein, parties The The Laplacian noise added to the vote results of the class, is a scale parameter, The greater the noise intensity is the greater, is a statistical class The number of votes obtained, , parties The local data class of the 10. The method of claim 9, wherein the federated learning fair model aggregation method based on security data synthesis of the participant is characterized in that, The step S205 is used for: expanding the data volume from the original to , and the expanded data set is expressed as Each participant calculates the difference between the original data distribution and the expanded data distribution through KL divergence, and calculates the data expansion amount by difference , and then uploads the KL divergence and to the server, and the formula is as follows: in, For the first The amount of original data from each participant As a participant The new data volume after adding synthetic data, For the first The original local datasets of each participant, For the first The new dataset is generated by each participant adding a generator to synthesize the data. For the first Data expansion for each participant Indicates the participating parties Original data sample, Indicates the expanded participants Data samples, This represents all data samples, i.e. .

Images