Class fair federated learning aggregation method based on bulldozer distance and dynamic attenuation

Abstract

The invention discloses a category fair federated learning aggregation method based on bulldozer distance and dynamic attenuation. Aiming at the problem of poor model fairness of existing federated learning under non-independent identically distributed data, in a category fair federated learning scene, the bulldozer distance between data distribution and preset uniform ideal distribution is locally calculated through a client, and the data distribution quality is evaluated; a relative distribution quality scoring mechanism is introduced, and the aggregation weight is dynamically adjusted in combination with the data volume weight and the time-varying attenuation coefficient decreasing progressively along with the training round. According to the method, client updating with more balanced distribution is preferentially adopted in the initial training stage, the fairness of the model is improved, the distribution difference influence is gradually weakened in the later stage, and stable convergence is achieved. The method does not need a server to provide dynamic reference distribution, protects privacy, and is suitable for federal learning tasks with high fairness requirements, such as education and medical treatment.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and distributed machine learning technology, specifically to a model aggregation method in federated learning, and more particularly to a class-fair federated learning client weight adjustment method based on bulldozer distance and dynamic decay mechanism. It is applicable to multi-client collaborative learning systems in non-independent identically distributed (Non-IID) data environments, such as smart healthcare, online education, fintech, and other application scenarios with high requirements for model fairness. Background Technology

[0002] Federated learning, as a distributed machine learning paradigm that protects data privacy, allows multiple clients to collaboratively train a global model without sharing the original data, and has been widely used in fields such as healthcare, finance, and smart terminals. However, the local data of each client usually exhibits non-independent and identically distributed (Non-IID) characteristics, meaning that the data distribution varies significantly. This causes traditional aggregation methods (such as FedAvg) to allocate aggregation weights based solely on the amount of data from each client, which can easily lead to model bias towards clients with large amounts of data or concentrated categories, compromising the fairness of the model to clients with minority classes or small numbers of clients.

[0003] In existing technologies, some studies have attempted to adjust aggregation weights by introducing data distribution similarity metrics, but these often rely on servers collecting client distribution information, posing a risk of privacy breaches; others employ static weight mechanisms, which are difficult to adapt to dynamic changes during training. Furthermore, the lack of a reasonable model for the "ideal data distribution" results in a lack of a unified benchmark for distribution quality assessment.

[0004] To address this, this invention proposes a class-based fair federated learning aggregation method based on bulldozer distance and dynamic decay. By setting a uniform distribution as the ideal distribution, the client can independently calculate its distance from the bulldozers in the ideal distribution and evaluate the distribution quality. Combining a relative scoring mechanism and a decay control strategy with training rounds, the aggregation weights are dynamically optimized while ensuring privacy, thereby improving the fairness and convergence stability of the global model. Summary of the Invention

[0005] The purpose of this invention is to overcome the problems of poor model fairness, static aggregation strategies, and lack of benchmarks for distribution evaluation in existing federated learning methods under non-independent identically distributed (Non-IID) data environments. It provides a category-fair federated learning aggregation method, system, and medium based on bulldozer distance and dynamic decay. This method, while protecting client data privacy, improves the fairness of the global model for each category and participant by introducing an ideal distribution benchmark and a dynamic weight adjustment mechanism. It is suitable for applications with high fairness requirements, such as education, healthcare, and finance.

[0006] To achieve the above objectives, this invention provides a class-based fair federated learning aggregation method based on bulldozer distance and dynamic decay, comprising the following steps:

[0007] S1. Obtain local data information from multiple clients participating in federated learning. The local data information includes at least the data sample size of the client and the bulldozer distance between the local data distribution of the client and a preset ideal distribution.

[0008] S2. Based on the data sample size, calculate the data volume weight for each client;

[0009] S3. Based on the bulldozer distance, calculate the distributed quality weight for each client;

[0010] S3.1. Standardize the bulldozer distances of all clients to generate a relative distribution quality score relative to the group of clients participating in this round, and adjust the distribution quality weights based on this score; wherein, the standardization process includes: calculating the mean μ of the bulldozer distances of all clients. d With standard deviation σ d And for each client i, calculate its relative distribution quality score as follows:

[0011]

[0012] Where, d i The bulldozer distance for client i is given, and a higher score indicates that the data distribution of that client is closer to the ideal distribution.

[0013] S3.2. Generate distributed quality weights for each client:

[0014]

[0015] in, z is the distribution quality weight adjusted based on the relative distribution quality score. i The relative distribution quality score is given, and γ is a hyperparameter for modulating sensitivity.

[0016] S4. Multiply the data volume weight of each client by the distribution quality weight adjusted by the time-varying decay coefficient to obtain the initial aggregate weight of the client, wherein the time-varying decay coefficient decreases monotonically with the increase of training rounds.

[0017] S5. Normalize the initial aggregation weights of all clients to obtain the final weights used for model aggregation.

[0018] Furthermore, the preset ideal distribution is a uniform distribution with equal probabilities for each category, enabling each client to independently calculate the bulldozer distance between its local data distribution and the ideal distribution without relying on a server to provide a dynamic reference distribution.

[0019] Furthermore, the distribution quality weight is positively correlated with the relative distribution quality score. The higher the score, the greater the distribution quality weight, which is used to prioritize the client contributions with more balanced data distribution in the early stages of training, thereby improving the fairness of the global model.

[0020] Furthermore, the time-varying decay coefficient is a function of the training round t, and its value monotonically decreases from the initial value to a non-negative minimum value. The decay method adopts one of the following: exponential decay, linear decay, or step decay function.

[0021] The time-varying decay is used to emphasize the quality of data distribution in the early stage of training and gradually weaken the impact of distribution differences on aggregation weights in the later stage of training, so as to achieve a smooth transition from "correction" to "fusion".

[0022] Furthermore, the data volume weight is the proportion of the data sample size of this client to the total sample size of all clients, that is:

[0023]

[0024] Where, n i Let N be the number of data samples for client i, and N be the total number of clients participating in the aggregation in this round.

[0025] Furthermore, the step of "multiplying the data volume weight by the time-varying decayed distribution quality weight" is specifically achieved through the following formula:

[0026]

[0027] in, Let α(t) be the initial aggregation weight of client i, and let α(t) be the time-varying decay coefficient in round t.

[0028] Furthermore, the "normalization process" refers to dividing the initial aggregate weight of each client by the sum of the initial aggregate weights of all clients to obtain the final aggregate weight, i.e.:

[0029]

[0030] in, Let be the final aggregate weight for client i.

[0031] Furthermore, the bulldozer distance is replaced by one of Jensen-Shannon divergence, maximum mean difference (MMD), or label entropy, or Gini coefficient, to adapt to fairness assessments under different computing resources or privacy protection requirements.

[0032] The present invention also provides an aggregation system for category-fair federated learning, comprising a central server and multiple clients, wherein the central server is configured to perform the above-described method to perform weighted aggregation of local models uploaded by the clients.

[0033] The present invention also provides a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, implement the above-described method.

[0034] The present invention has the following advantages and effects compared with the prior art:

[0035] (1) Improve model fairness: By setting uniform distribution as the ideal distribution and combining bulldozer distance to evaluate the quality of client data distribution, the updates of clients with more balanced distribution are adopted first, which effectively alleviates the model bias problem caused by Non-IID data;

[0036] (2) Dynamically adapting to the training process: Introducing a time-varying decay mechanism, emphasizing distribution quality in the early stage of training to "correct bias" and weakening differences in the later stage to "integrate", taking into account both fairness and convergence stability;

[0037] (3) Protect data privacy: The client only needs to calculate the distance from the preset ideal distribution, without uploading the real distribution information, to prevent distribution-level privacy leakage;

[0038] (4) Computationally efficient and scalable: The relative scoring mechanism is simple to implement, supports alternative indicators such as MMD and tag entropy, and adapts to different resource and scenario requirements;

[0039] (5) High applicability: It is particularly suitable for federated learning tasks with high requirements for fairness and privacy, such as new user cold start, active learning, and medical joint modeling. Attached Figure Description

[0040] Figure 1 This is a schematic diagram of the architecture of a category-based fair federated learning aggregation system provided in an embodiment of the present invention;

[0041] Figure 2 This is a flowchart of a category-based fair federated learning aggregation method based on bulldozer distance and dynamic decay provided in an embodiment of the present invention;

[0042] Figure 3 This is a schematic diagram of the time-varying decay coefficient α(t) changing with the training round t, provided by an embodiment of the present invention. Detailed Implementation

[0043] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0044] Example

[0045] This embodiment uses the collaborative training of a disease diagnosis model by multiple hospitals in the field of smart healthcare as an example to explain in detail the specific implementation process of the "Class-Based Fair Federated Learning Aggregation Method Based on Bulldozer Distance and Dynamic Decay" described in this invention. The system architecture is as follows: Figure 1 As shown, the overall process follows Figure 2 The steps are shown.

[0046] 1. System Architecture

[0047] like Figure 1 As shown, this system includes a central server and N participating hospital clients, forming a typical horizontal federated learning architecture:

[0048] • Clients: Representing five hospitals located in different regions, each possessing electronic medical record data of local patients, used to train the diabetes prediction model. The data category distribution of each hospital exhibits significant Non-IID characteristics (e.g., Client 1 mainly consists of elderly patients, while Client 2 mainly consists of obese young patients).

[0049] • Central server: responsible for coordinating the training process, distributing the global model, collecting local updates, and executing the aggregation algorithm proposed in this invention to generate a new global model.

[0050] The communication method is a secure encrypted channel (such as TLS), and the raw data does not leave the local machine. Only model parameters and metadata (sample size, bulldozer distance) are exchanged.

[0051] 2. Method Execution Flow

[0052] The method execution flow is as follows: Figure 2 As shown:

[0053] The entire training process is divided into several rounds, with each round consisting of the following steps:

[0054] S1: Obtain local data information

[0055] Before each training round begins, each client performs the following calculations based on its local data:

[0056] • Count the total local sample size n i (For example, client 1 has 1000 records);

[0057] · Calculate the local label distribution p i (e.g., the ratio of diabetic to non-diabetic);

[0058] • Set the preset ideal distribution as a uniform distribution q = [0.5, 0.5] (binary classification task);

[0059] · Calculate the local data distribution p i The distance between the bulldozer and the ideal distribution q is denoted as d. i .

[0060] This step is completed locally on the client side, without needing to upload the actual distribution, thus protecting privacy.

[0061] S2: Calculate the weight of the data volume

[0062] The server collects the sample size n of all clients participating in this round. i Calculate its data volume weight:

[0063]

[0064] Assuming the total sample size of the five clients in this round is 5000, then the weight of client 1 is 0.2.

[0065] S3: Calculate the data distribution weights

[0066] First, the client's bulldozer distance is standardized. The server then standardizes the received client bulldozer distance d. i Calculate the mean μ d With variance σ d :

[0067]

[0068] The relative distribution quality score of client i is obtained:

[0069]

[0070] Among them, z i The higher the value, the closer the client's data distribution is to the ideal distribution.

[0071] Secondly, generate relative distribution quality scores and adjust the weights.

[0072] Generate distributed quality weights for each client:

[0073]

[0074] in, z is the distribution quality weight adjusted based on the relative distribution quality score. i The relative distribution quality score is given, and γ is a hyperparameter for adjusting sensitivity (e.g., set to 1.0).

[0075] This mechanism allows a client to obtain a high weight even if the sample size is small, as long as the distribution is balanced.

[0076] S4: Calculate the initial aggregation weights

[0077] Weighting the amount of data Multiplying the distributed mass weights adjusted by the time-varying decay coefficient α(t) yields the initial aggregate weights:

[0078]

[0079] in, Let α(t) be the initial aggregate weight for client i, and let α(t) be a monotonically decreasing function of training epoch t, ​​such as exponential decay.

[0080] α(t)=α0·e -βt (8)

[0081] like Figure 3 As shown, the curve of α(t) changing with the number of rounds t when the initial value α0 = 1 and the decay rate β = 0.05 is shown. In the early stage of training (e.g., t < 5), α ≈ 1, the distribution quality has a significant impact, and the system prioritizes client updates with a balanced distribution to achieve "bias correction"; in the later stage of training (e.g., t > 40), α → 0.1, the impact of distribution differences is weakened, and the system shifts to "fusion" dominated by the amount of data to ensure convergence stability.

[0082] S5: Normalization yields the final aggregate weights.

[0083] Sum and normalize the initial weights for all clients:

[0084]

[0085] in, Let be the final aggregate weight for client i. The server uses the final weights for each client to calculate the global model and updates the global model.

[0086] 3. Extensibility Description

[0087] If the client has limited computing resources or higher privacy protection requirements, the bulldozer distance can be replaced by Jensen-Shannon divergence, maximum mean difference (MMD), label entropy, or Gini coefficient.

[0088] This method is applicable to multi-class classification tasks; only the ideal distribution needs to be adjusted accordingly.

[0089] The above embodiments are preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the above embodiments. Any changes, modifications, substitutions, combinations, or simplifications made without departing from the spirit and principle of the present invention shall be considered equivalent substitutions and shall be included within the protection scope of the present invention.

Claims

1. A class-based fair federated learning aggregation method based on bulldozer distance and dynamic decay, characterized in that, The category-based fair federated learning requires the model to have balanced representativeness across categories, and the method includes the following steps: S1. Obtain local data information from multiple clients participating in federated learning. The local data information includes at least the data sample size of the client and the bulldozer distance between the local data distribution of the client and a preset ideal distribution. S2. Based on the data sample size, calculate the data volume weight for each client; S3. Based on the bulldozer distance, calculate the distributed quality weight for each client; S3.

1. Standardize the bulldozer distances of all clients to generate a relative distribution quality score relative to the group of clients participating in this round, and adjust the distribution quality weights based on this score; wherein, the standardization process includes: calculating the mean μ of the bulldozer distances of all clients. d With standard deviation σ d And for each client i, calculate its relative distribution quality score as follows: Where, d i The bulldozer distance for client i is given, and a higher score indicates that the data distribution of that client is closer to the ideal distribution. S3.

2. Generate distribution quality weights for each client: in, z is the distribution quality weight adjusted based on the relative distribution quality score. i The relative distribution quality score is given, and γ is a hyperparameter for modulating sensitivity. S4. Multiply the data volume weight of each client by the distribution quality weight adjusted by the time-varying decay coefficient to obtain the initial aggregate weight of the client, wherein the time-varying decay coefficient decreases monotonically with the increase of training rounds. S5. Normalize the initial aggregation weights of all clients to obtain the final weights used for model aggregation.

2. The category-based fair federated learning aggregation method based on bulldozer distance and dynamic decay according to claim 1, characterized in that, The preset ideal distribution is a uniform distribution with equal probabilities for each category, enabling each client to independently calculate the bulldozer distance between its local data distribution and the ideal distribution without relying on a server to provide a dynamic reference distribution.

3. The category-based fair federated learning aggregation method based on bulldozer distance and dynamic decay according to claim 1, characterized in that, The distribution quality weight is positively correlated with the relative distribution quality score. The higher the score, the greater the distribution quality weight. This is used to prioritize the client contributions with more balanced data distribution in the early stages of training, thereby improving the fairness of the global model.

4. The category-based fair federated learning aggregation method based on bulldozer distance and dynamic decay according to claim 1, characterized in that, The time-varying decay coefficient is a function of the training round t, and its value decreases monotonically from the initial value to a non-negative minimum value. The decay method adopts one of the following: exponential decay, linear decay, or step decay function. The time-varying decay is used to emphasize the quality of data distribution in the early stage of training and gradually weaken the impact of distribution differences on aggregation weights in the later stage of training, so as to achieve a smooth transition from "correction" to "fusion".

5. The category-based fair federated learning aggregation method based on bulldozer distance and dynamic decay according to claim 1, characterized in that, The data volume weight is the proportion of the data sample size of this client to the total sample size of all clients, that is: Where, n i Let N be the number of data samples for client i, and N be the total number of clients participating in the aggregation in this round.

6. The category-based fair federated learning aggregation method based on bulldozer distance and dynamic decay according to claim 1, characterized in that, The step of "multiplying the data volume weight by the time-varying decayed distribution quality weight" is specifically achieved through the following formula: in, Let α(t) be the initial aggregation weight of client i, and let α(t) be the time-varying decay coefficient in round t.

7. The category-based fair federated learning aggregation method based on bulldozer distance and dynamic decay according to claim 1, characterized in that, The "normalization process" refers to dividing the initial aggregate weight of each client by the sum of the initial aggregate weights of all clients to obtain the final aggregate weight, i.e.: in, Let be the final aggregate weight for client i.

8. The fair federated learning aggregation method based on bulldozer distance and dynamic decay according to claim 1, characterized in that, The bulldozer distance was replaced with one of Jensen-Shannon divergence, maximum mean difference (MMD), or label entropy, or Gini coefficient, to adapt to fairness assessments under different computing resources or privacy protection requirements.

9. An aggregation system for category-based fair federated learning, characterized in that, It includes a central server and multiple clients, wherein the central server is configured to perform a class-fair federated learning aggregation method based on bulldozer distance and dynamic decay as described in any one of claims 1 to 7, and to perform weighted aggregation of local model parameters uploaded by each client to update the global model.

10. A computer-readable storage medium storing computer program instructions thereon, characterized in that, When the computer program instructions are executed by the processor, they implement the category-fair federated learning aggregation method based on bulldozer distance and dynamic decay as described in any one of claims 1 to 7.

Images