Federal learning global model training method based on fuzzy weighting and dynamic clustering

Abstract

The invention belongs to the technical field of model training, and particularly relates to a federated learning global model training method based on fuzzy weighting and dynamic clustering. The method comprises the steps of client local training and information reporting, frequency domain feature extraction, feature normalization and joint vector construction, dynamic clustering number determination and division, intra-group fuzzy weighted aggregation, inter-group fuzzy weighted aggregation, weight lower limit protection, learning rate scheduling and global model evaluation and termination. According to the method, multi-modal joint representation based on training loss, model parameter frequency domain features and category entropy is introduced into federated learning, and a dynamic clustering and double-layer fuzzy weighted aggregation mechanism is constructed, so that the global model obtains remarkably higher stability, precision and generalization ability in a non-independent identically distributed environment. By jointly using the loss, the frequency domain statistics and the category entropy, the heterogeneity between the clients in the aspects of data distribution, training process and model updating structure can be described more accurately.

Description

Technical Field

[0001] This invention belongs to the field of model training technology, specifically relating to a federated learning global model training method based on fuzzy weighting and dynamic clustering. Background Technology

[0002] Federated Learning (FL) is a distributed learning framework that allows multiple clients to collaboratively train machine learning models without sharing the original data. In practical applications, the data from participating parties often exhibit significant non-independent and identically distributed (Non-IID) characteristics, such as class imbalance, differences in feature distribution, and uneven distribution of sample numbers, making it difficult for traditional FedAvg-like methods to converge to a well-performing global model.

[0003] However, existing methods, such as introducing proximal terms to alleviate some of the drift problem or using fixed clustering based on model parameters, have the following shortcomings: relying solely on model parameters for clustering fails to capture loss features and training dynamics; the number of clusters is fixed and cannot be dynamically adjusted according to the training process, leading to over- or under-clustering; both within-group and between-group aggregations use fixed forms, lacking a flexible weight allocation mechanism; and the structural features of the model in the frequency domain are not considered, failing to reflect the higher-order statistical properties of the model parameters. Therefore, there is an urgent need for a federated learning method that can improve stability and convergence performance in highly heterogeneous environments. Summary of the Invention

[0004] In view of the shortcomings of the prior art, the purpose of this invention is to provide a federated learning global model training method based on fuzzy weighting and dynamic clustering, which can improve the stability, robustness and accuracy of model training in non-IID environments.

[0005] To achieve the above objectives, this invention provides a federated learning global model training method based on fuzzy weighting and dynamic clustering, comprising the following steps: S1. The server sends the current global model parameters to the selected client. The client performs stochastic gradient descent update with momentum, calculates the local average training loss and class entropy, and reports them to the server along with the updated model parameters. S2. The server flattens the received updated model parameters to obtain a one-dimensional vector, and performs a fast Fourier transform to obtain frequency domain statistical features. S3. The server performs maximum and minimum value normalization on the local average training loss and frequency domain statistical features of all clients, and constructs a joint feature vector reflecting client heterogeneity by combining class entropy. S4. Calculate the overall heterogeneity index based on the normalized local average training loss and frequency domain statistical characteristics. Dynamically calculate the number of clusters K based on the overall heterogeneity index and divide all clients into K groups based on the joint feature vector. S5. Calculate the intra-group weight of each client within the group, and perform weighted aggregation on the client models within the same cluster to obtain the intra-group aggregated model. S6. Calculate the inter-group weights of each group, perform secondary weighted aggregation on all intra-group aggregation models, and obtain the updated global model. S7. During training, threshold detection is performed on the weights within and between groups. Weights that are below the preset lower limit are reset to the lower limit threshold and renormalized. S8. The server adjusts the learning rate of the client's local training in the next round according to the current training round. The adjustment adopts a three-stage strategy that includes a warm-up period, a stabilization period, and a decay period. S9. Evaluate the global model performance on the public test set and determine whether the termination condition is met based on the accuracy or loss metric.

[0006] As a preferred embodiment of the present invention, in S1, the category entropy is the entropy calculated based on the client label distribution or the predicted distribution, and is used to characterize the degree of balance of the local data distribution. The client reports the local average training loss, class entropy, and updated model parameters to the server. In addition, the client also reports statistics that characterize its local data distribution, including the local gradient norm and the rate of local loss descent.

[0007] In a preferred embodiment of the present invention, in step S2, frequency domain energy, peak amplitude, mean, and standard deviation are extracted using Fast Fourier Transform, and frequency domain statistical characteristics are obtained: ; In the formula, For the frequency domain statistical characteristics of client k; For the frequency domain energy of client k; The peak value of the amplitude for client k; Let k be the frequency domain standard deviation of the client. Let k be the frequency domain mean of the client. , , , They are respectively , , , The weighting coefficients.

[0008] As a preferred embodiment of the present invention, in S3, the local average training loss for client k is... Frequency domain statistical characteristics Perform maximum and minimum value normalization processing to obtain the following results: and , After normalization , After normalization ,Will and Combined into a two-dimensional joint representation vector ; exist Introducing the class entropy of client k The joint feature vector is obtained. The superscript T indicates transpose.

[0009] As a preferred embodiment of the present invention, in S4, based on and Calculate the loss and the heterogeneity in the frequency domain: ; ; In the formula, Indicates loss heterogeneity; Indicates frequency domain characteristic heterogeneity; It is a function of standard deviation; The overall heterogeneity index is obtained by combining the results. : ; Using a hybrid linear-truncation strategy, the number of clusters K is calculated: ; ; ; In the formula, This indicates rounding to the nearest integer. Indicates the initial number of clusters; Indicates the number of intermediate clusters; This is the heterogeneity amplification factor in dynamic clustering; , , respectively, represent the preset maximum and minimum number of clusters; b is the basic bias term; For joint eigenvectors Perform K-Means clustering to divide all clients into K groups.

[0010] As a preferred embodiment of the present invention, in S5, calculating the intra-group weight of each client in the group specifically involves first calculating the initial weight, which is either an inverse intra-group weight or a Softmax intra-group weight. The inverse proportional group weights are expressed as follows: ; In the formula, This represents the inverse-proportional weight of client k within the g-th group; , They are respectively and Weighting coefficients; The softmax type within-group weights are represented as follows: ; ; ; In the formula, T1 represents the unnormalized Softmax weight of client k within the g-th group; T1 represents the temperature coefficient in the Softmax function, used to control the smoothness of the weight distribution. This represents the Softmax type in-group weight of client k within the g-th group; This represents the sum of the unnormalized Softmax weights of all clients within the g-th group; Let j represent the set of clients in the g-th group, and j represent one of the clients. , Let represent the normalized local average training loss and normalized frequency domain statistical characteristics of client j within the g-th group, respectively. , These represent the weight coefficients of the normalized local average training loss and the normalized frequency domain statistical features, respectively. choose , Any one of them is used as the initial weight of client k within the g-th group. ; Subsequently, category entropy was used for correction. : ; In the formula, This represents the weight of client k within the g-th group; This represents the entropy adjustment factor, used to adjust the client's aggregation weights based on category entropy; Ultimately, the intra-group aggregation model is obtained: ; In the formula, This represents the parameters of the within-group aggregation model for the g-th group; This represents the updated model parameters for client k.

[0011] As a preferred embodiment of the present invention, in S6, calculating the inter-group weights of each group specifically involves first calculating the group-level loss and frequency domain statistics: ; ; In the formula, This represents the average training loss within the g-th group; This represents the average frequency domain statistical characteristics within the g-th group; The between-group weights can be either inverse between-group weights or Softmax between-group weights. The inverse proportion between-group weights are expressed as follows: ; In the formula, This represents the inverse-proportional inter-group weights of the g-th group; , They are respectively , Weighting coefficients in inverse proportional intergroup weighting; The softmax type between-group weights are represented as follows: ; ; In the formula, This represents the Softmax type inter-group weights of the g-th group; , They are respectively , Weighting coefficients in Softmax type inter-group weights; This represents the unnormalized contribution index of the g-th group; This represents the unnormalized contribution index of the h-th group; choose , Any one of them is used as the inter-group weight of the g-th group. ; Finally, the updated global model is obtained: ; In the formula, This represents the final parameters of the global model when the training round is t+1.

[0012] As a preferred embodiment of the present invention, in S7, a lower threshold is set. ,for and : ; ; In the formula, This indicates an update to; Then renormalize to ensure the weight sum is 1: ; ; In the formula, This represents the inter-group weight of the h-th group; This represents the weight of client j within the g-th group.

[0013] As a preferred embodiment of the present invention, the three-stage strategy comprising a preheating period, a stabilization period, and a decay period in S8 is expressed as follows: ; In the formula, This represents the learning rate when there are t training epochs; Indicates the initial learning rate; This represents the learning rate decay coefficient; This represents the decay period constant; This indicates the end of the preset warm-up period; This indicates the start of the preset decay period.

[0014] In a preferred embodiment of the present invention, in S9, an evaluation index is calculated after each round of training, and training is terminated when a termination condition is met. The termination condition is... or ,in, This represents the test set accuracy when there are t training epochs. This represents the preset optimal accuracy, and t represents the number of training epochs. This is the preset maximum number of training rounds.

[0015] The beneficial effects of this invention are: This invention introduces a multimodal joint representation based on training loss, frequency domain features of model parameters, and class entropy into federated learning, and constructs a dynamic clustering and two-layer fuzzy weighted aggregation mechanism. This enables the global model to achieve significantly higher stability, accuracy, and generalization ability in a non-independent and identically distributed (Non-IID) environment. By jointly using loss, frequency domain statistics, and class entropy, this invention can more accurately characterize the heterogeneity among clients in data distribution, training process, and model update structure. Through the heterogeneity-driven dynamic clustering mechanism, clients can adaptively group according to training features, avoiding the structural rigidity problem caused by a fixed number of clusters. Based on this, this invention adopts a fuzzy weighting strategy both within and between groups, allowing the client contribution to be automatically adjusted according to its training performance and features, thereby reducing the interference of abnormal clients or extreme updates on the global model and improving training robustness.

[0016] This invention employs a weight lower bound protection mechanism to prevent information loss, and utilizes a preheating-constant-decaying learning rate scheduling to improve early stability and later convergence efficiency. Combined with engineering measures such as cluster failure fallback, small sample clustering processing, and BatchNorm buffer protection, this invention maintains high reliability and consistency even in complex real-world scenarios. In summary, this invention effectively improves the stability, convergence, and practicality of federated learning on highly heterogeneous data. Attached Figure Description

[0017] Figure 1 This is a flowchart illustrating the principle of this invention; Figure 2 This is a schematic diagram of the overall framework for image classification tasks in Embodiment 2 of the present invention. Detailed Implementation

[0018] The embodiments of the present invention will be further described below with reference to the accompanying drawings: Example 1: As Figure 1 As shown, the federated learning global model training method based on fuzzy weighting and dynamic clustering includes the following steps: S1. Client Local Training and Information Reporting: The server sends the current global model parameters to the selected client. The client performs stochastic gradient descent with momentum (SGD with Momentum) updates based on the local dataset, calculates the local average training loss (scalar) and class entropy, and reports them to the server along with the updated model parameters. S2. Frequency domain feature extraction: The server flattens the received updated model parameters to obtain a one-dimensional vector, and performs a fast Fourier transform to obtain frequency domain statistical features. S3. Feature Normalization and Joint Vector Construction: The server performs maximum and minimum value normalization on the local average training loss and frequency domain statistical features of all clients, and constructs a joint feature vector that reflects the heterogeneity of the clients by combining the class entropy. S4. Determination and division of the number of clusters in dynamic clusters: Calculate the overall heterogeneity index based on the normalized local average training loss and frequency domain statistical characteristics, dynamically calculate the number of clusters K based on the overall heterogeneity index, and divide all clients into K groups based on the joint feature vector. S5. Intra-group fuzzy weighted aggregation: Calculate the intra-group weight of each client within the group, and perform weighted aggregation on the client models within the same cluster to obtain the intra-group aggregated model; S6. Inter-group fuzzy weighted aggregation: Calculate the inter-group weights of each group, perform secondary weighted aggregation on all intra-group aggregation models, and obtain the updated global model. S7. Weight Lower Bound Protection: During training, threshold detection is performed on the weights within and between groups. Weights that are below the preset lower bound are reset to the lower bound threshold and renormalized. S8. Learning Rate Scheduling: The server adjusts the learning rate of the client's local training in the next round according to the current training round. The adjustment adopts a three-stage strategy that includes a warm-up period, a stabilization period, and a decay period. S9. Global Model Evaluation and Termination: Evaluate the performance of the global model on a common test set and determine whether the termination condition is met based on the accuracy or loss metric.

[0019] In S1, category entropy is the entropy calculated based on the client label distribution or predicted distribution, used to characterize the degree of balance in the local data distribution; The client reports the local average training loss, class entropy, and updated model parameters to the server. In addition, the client also reports statistics that characterize its local data distribution to enhance the server's ability to identify heterogeneity, including the local gradient norm and the rate of local loss descent.

[0020] For client k, the server selects the client set in training round t. and broadcast the current global model parameters to it. Client k receives Then, perform a local stochastic gradient descent update for E epochs: ; In the formula, This represents the training sample sampled by client k in the i-th sampling. This represents the learning rate when there are t training epochs; For gradient operators; For local training loss functions (such as cross-entropy loss); , , These represent the model parameters of client k during the (i-1), i, and i+1th local iterations, respectively. It is the momentum coefficient (the core hyperparameter of momentum SGD).

[0021] During each local iteration, the client randomly samples one training sample. Based on this sample, the loss gradient is calculated, and then the SGD formula with momentum is used to adjust the current parameters. Updated to .

[0022] The client returned the following values: Model update results: ; This represents the updated model parameters for client k. Let represent a d-dimensional real space, where d is the total dimension of the trainable parameters of the model; Average loss, used to characterize the training difficulty on the client side: ; In the formula, Let k be the local average training loss for client k; This represents a single sample in the local training dataset of client k, containing x: the input features of the sample (such as image pixels, text vectors, etc.) and y: the true label of the sample (such as the class label for a classification task or the true value for a regression task). For the local training dataset of client k; Category entropy (a measure of the uniformity of data distribution): ; In the formula, Let k be the class entropy of client k. C represents the proportion of category c in the client's data (i.e., the proportion of the number of samples of category c in the label distribution to the total number of samples in the client); C is the total number of categories in the task (e.g., in an image classification task, cat, dog, and car correspond to C=3). In S2, the server receives Then, it is flattened to obtain a one-dimensional vector. : ; In the formula, Indicates flattening process; right Calculate the one-dimensional fast Fourier transform.

[0023] This embodiment is based on the observation that the updated models of different clients exhibit stable and separable statistical differences in the frequency domain, reflecting non-IID characteristics such as local training noise, gradient oscillations, and smoothness. Therefore, from the frequency domain amplitude spectrum... Four core statistical features were extracted: ; ; ; ; In the formula, For the frequency domain energy of client k; The peak value of the amplitude for client k; Let k be the frequency domain standard deviation of the client. Let k be the frequency domain mean of the client. For the frequency domain vector of client k, each element For each frequency component (z is the index of the traversed frequency components), store the parameter characteristics (such as oscillation amplitude and phase) at that frequency. Frequency domain energy, peak amplitude, mean, and standard deviation are extracted using Fast Fourier Transform, and frequency domain statistical characteristics are obtained. ; In the formula, For the frequency domain statistical characteristics of client k; , , , They are respectively , , , The weighting coefficients are determined by experience or validation sets; the same applies to other weighting factors and coefficients.

[0024] Higher-order frequency domain features such as mid-frequency energy ratio, bandwidth, and spectral centroid can also be added to more comprehensively characterize the client training dynamics.

[0025] In S3, the local average training loss for client k Frequency domain statistical characteristics Perform maximum and minimum value normalization processing to obtain the following results: and , This is the result of normalization (subsequent normalization refers to maximum and minimum value normalization; a minimum constant can be introduced during normalization to prevent the denominator from being zero). , After normalization ,Will and Combined into a two-dimensional joint representation vector ; exist Introducing the class entropy of client k The joint feature vector is obtained. The superscript T denotes transpose. This three-dimensional vector is used to describe the dynamic differences in training for each client, and is then used for subsequent dynamic clustering.

[0026] In S4, based on and Calculate the loss and the heterogeneity in the frequency domain: ; ; In the formula, Indicates loss heterogeneity; Indicates frequency domain characteristic heterogeneity; It is a function of standard deviation; The overall heterogeneity index is obtained by combining the results. : ; Using a hybrid linear-truncation strategy, the number of clusters K is calculated: ; ; ; In the formula, This indicates rounding to the nearest integer. Indicates the initial number of clusters; Indicates the number of intermediate clusters; This is the heterogeneity amplification factor in dynamic clustering; , , respectively, represent the preset maximum and minimum number of clusters; b is the basic bias term; For joint eigenvectors Perform K-Means clustering to divide all clients into K groups.

[0027] To enhance robustness, multiple random initializations can be set, and the result with the lowest sum of squared errors (SSE) can be selected. That is, when the server performs K-Means clustering on the joint features of the client (such as normalized loss and frequency domain features), it will generate multiple clustering results by randomly initializing the centroids multiple times, and finally select the result with the lowest SSE as the final grouping for the client.

[0028] In S5, the calculation of the intra-group weight of each client in the group is as follows: First, calculate the initial weight, which is either an inverse intra-group weight or a Softmax intra-group weight. The inverse proportional group weights are expressed as follows: ; In the formula, This represents the inverse-proportional weight of client k within the g-th group; , They are respectively and Weighting coefficients; The softmax type within-group weights are represented as follows: ; ; ; In the formula, T1 represents the unnormalized Softmax weight of client k within the g-th group; T1 represents the temperature coefficient in the Softmax function, used to control the smoothness of the weight distribution. This represents the Softmax type in-group weight of client k within the g-th group; This represents the sum of the unnormalized Softmax weights of all clients within the g-th group; Let j represent the set of clients in the g-th group, and j represent one of the clients. , Let represent the normalized local average training loss and normalized frequency domain statistical characteristics of client j within the g-th group, respectively. , These represent the weight coefficients of the normalized local average training loss and the normalized frequency domain statistical features, respectively; the exp function represents the natural exponential function, that is, an exponential function with the real number e as its base.

[0029] choose , Any one of them is used as the initial weight of client k within the g-th group. ; Subsequently, category entropy was used for correction. : ; In the formula, This represents the weight of client k within the g-th group; This represents the entropy adjustment factor, used to adjust the client's aggregation weights based on category entropy; Ultimately, the intra-group aggregation model is obtained: ; In the formula, This represents the parameters of the intra-group aggregation model for the g-th group.

[0030] In S6, the calculation of the inter-group weights for each group is specifically as follows: First, the group-level loss and frequency domain statistics are calculated: ; ; In the formula, This represents the average training loss within the g-th group; This represents the average frequency domain statistical characteristics within the g-th group; The between-group weights can be either inverse between-group weights or Softmax between-group weights. The inverse proportion between-group weights are expressed as follows: ; In the formula, This represents the inverse-proportional inter-group weights of the g-th group; , They are respectively , Weighting coefficients in inverse proportional intergroup weighting; The softmax type between-group weights are represented as follows: ; ; In the formula, This represents the Softmax type inter-group weights of the g-th group; , They are respectively , Weighting coefficients in Softmax type inter-group weights; This represents the unnormalized contribution index of the g-th group; This represents the unnormalized contribution index of the h-th group; choose , Any one of them is used as the inter-group weight of the g-th group. ; Finally, the updated global model is obtained: ; In the formula, This represents the final parameters of the global model when the training round is t+1.

[0031] In intra-cluster fuzzy weighted aggregation, flexible weights are assigned to clients within each cluster, ensuring that high-quality clients contribute more to the model. Simultaneously, class entropy is used to prevent clients with overly balanced data from being ignored. In inter-cluster fuzzy weighted aggregation, to further enhance the robustness of the aggregation, weighting is performed again at the group level, allowing for flexible adjustment of cross-cluster differences. Softmax or inverse weight types can be selected as needed, and a preset threshold for switching weight strategies can be set. This is used to determine when to switch from inverse weights to Softmax weights.

[0032] In S7, a lower threshold is set. ,for and : ; ; In the formula, This indicates an update to; Then renormalize to ensure the weight sum is 1: ; ; In the formula, This represents the inter-group weight of the h-th group; This represents the weight of client j within the g-th group.

[0033] In S8, the three-stage strategy, comprising a warm-up period, a stabilization period, and a decay period, is represented as follows: ; In the formula, Indicates the initial learning rate; This represents the learning rate decay coefficient; This represents the decay period constant; This indicates the end of the preset warm-up period; This indicates the start of the preset decay period.

[0034] In S9, an evaluation metric is calculated after each training round, and training terminates when a termination condition is met. The termination condition is... or ,in, This represents the test set accuracy when there are t training epochs. This represents the preset optimal accuracy, and t represents the number of training epochs. This is the preset maximum number of training rounds.

[0035] During the inference phase, client-side clustering, fuzzy weight calculation, and learning rate updates are no longer performed. Instead, image classification or detection tasks are performed solely based on fixed final model parameters, enabling stable and reliable cross-device deep inference.

[0036] To ensure training stability, further settings can be implemented, such as: small sample clustering degradation fallback (merging clusters smaller than a threshold with the nearest cluster); automatic reinitialization upon K-Means clustering failure; replacement strategy for BatchNorm statistics; and strategy of running multiple times to select the optimal result.

[0037] Example 2: As Figure 2 As shown, this embodiment is a specific application example of the method in Embodiment 1 in an image classification task. The input image is... , Let H represent a 3×H×W dimensional real space, where H is the height of the input image and W is the width of the input image.

[0038] The local model is a CNN / Transformer for image classification, and its set of trainable parameters is denoted as w or ; S1. After receiving the global model from the server, the client performs several rounds of SGD training with momentum on the local image classification dataset and uploads the updated model parameters, local average training loss, and statistical information such as class entropy to the server. S2. The server flattens the local model parameters reported by the client and performs a Fast Fourier Transform (FFT) on the flattened vector to extract frequency domain statistical features such as energy, peak value, mean, and variance from the frequency domain amplitude information. S3. The server performs normalization processing on the training loss and frequency domain features of all clients, and combines the two to form a joint feature vector, which is used to represent the dynamic differences in training between different clients. S4. The server dynamically determines the number of clusters based on the overall difference of the joint feature vectors, and uses the K-Means method to divide all clients into several groups to capture the heterogeneous structure among clients. S5. Within each cluster, the server assigns intra-group fuzzy weights to each client based on the normalization loss and frequency domain features, and performs weighted aggregation of all client models within the cluster to obtain the group model. S6. The server further assigns inter-group fuzzy weights to each group based on the average loss within the group and the frequency domain statistics within the group, and performs a second weighted aggregation on all group models to obtain a global model update. S7. Apply lower bound protection to all intra-group and inter-group weights to ensure that no client or group is completely ignored during the aggregation process, thus avoiding information loss. S8. In each training round, the server sends a dynamically adjusted local learning rate to the client to improve stability in the early stages of training and enhance convergence performance in the later stages of training. S9. The server evaluates the updated global model on a test set and determines whether to end training based on preset termination conditions (such as stable performance or reaching the upper limit of training rounds).

[0039] In the above process, The calculation method is as follows ; Furthermore, in order to ensure numerical stability Alternatives are available , It is a very small constant.

[0040] The specific implementation process of each step is the same as in Example 1.

[0041] Based on the above process, after training is complete, the judgment process for any new image to be classified in this invention is as follows: First, the image is input into the already trained global model. The image will sequentially pass through the backbone network feature extraction module generated during the client training phase, automatically extracting multi-level spatial visual features. Subsequently, the frequency domain transformation operator stored inside the model will perform fast frequency domain statistical analysis on the classification parameters corresponding to the image features to obtain its structural response at the frequency domain level. During the inference phase, the loss recording, statistical normalization, or frequency domain feature recalculation corresponding to S1 to S3 will not be performed again. Instead, the final model parameters and corresponding converged statistical characteristics obtained by the server after training will be directly used for detection.

[0042] Next, the model will not perform the dynamic clustering operation corresponding to S4, nor will it perform client-side partitioning and group-level aggregation. Instead, it will directly use the final global model obtained during training through two-layer fuzzy weighted aggregation to make feature decisions. At this point, the two-layer fuzzy weighting strategy has converged to a stable global weight expression through training. Therefore, these weights remain fixed during the inference phase, are no longer updated, and no weight lower bound protection or normalization operations will be performed.

[0043] Subsequently, the depth spatial features of the input image I will be determined by the classifier head in the final aggregation model, and the classifier head will output a probability vector that the image belongs to a certain category. ,in This represents the probability that the input image belongs to class c. In binary classification scenarios, the probability value corresponding to the target class can be directly read. And compare it with a set threshold (e.g., 0.5); when If the image is in the target category, it is determined to be in the target category; otherwise, it is determined to be in the non-target category.

[0044] During the inference phase, the model parameters remain frozen, and no training-related optimization processes are performed, including loss calculation, fuzzy weight adjustment, clustering operations, learning rate scheduling, and group-level or intra-group weight updates; all of these modules are inactive. The final classification result is entirely given by the global model predictions obtained during training, thus ensuring high inference efficiency, strong stability, and good generalization ability.

[0045] Example 3: This example is a remote sensing image scene classification based on Example 1, including: In the field of remote sensing scene recognition, after training, it is directly used for automatic ground feature classification of satellite images. During the inference phase, the remote sensing image to be identified is input into the model. The backbone network extracts the texture, structure, and spatial scale features of ground features, while simultaneously utilizing the frequency domain patterns learned during training to improve the recognition ability of periodic textures (such as farmland and densely built-up areas). During inference, clustering, fuzzy weighting, and learning rate scheduling from the training phase are not performed; only the finally converged global classification head is used for inference. The model outputs probability vectors for different ground feature categories, and the final scene classification is performed based on the maximum probability or a set threshold, thereby achieving efficient and stable automatic remote sensing recognition.

[0046] Example 4: This example is based on Example 1, focusing on vehicle identification and type determination in traffic monitoring. In the field of traffic monitoring, it is used for vehicle type identification in monitoring images. During the inference phase, the model directly utilizes pre-trained spatial-frequency domain fusion features for classification, without performing clustering or fuzzy weighting, relying only on the frozen final model parameters. The classification head outputs the vehicle category probability, thus quickly completing vehicle identification and statistics.

[0047] Example 5: Based on Example 1, correction is performed using category entropy. At that time, entropy adjustment factor It is a time-varying parameter that is dynamically adjusted based on the local gradient change rate on the client side, rather than a fixed value; Calculate the rate of change of the local gradient of client k over t consecutive training rounds. : ; In the formula, , These represent the local gradient vectors of client k during training rounds t and t-1, respectively. Represents the L2 norm; according to Dynamically adjust the entropy adjustment factor : ; In the formula, This represents the initial entropy adjustment factor; The sensitivity coefficient representing the gradient rate of change is used to control the adjustment magnitude of the entropy adjustment factor as the gradient rate of change changes. Let be the entropy adjustment factor for client k during the t-th round of training. Substituting into the within-group weight correction formula, we obtain the time-varying within-group weights.

[0048] By introducing dynamic adjustment of gradient rate of change Decrease when gradient fluctuations are large To reduce the impact of class entropy on weights and avoid model training oscillations; to increase the impact of gradient fluctuations when they are small. This enhances the adaptability to heterogeneous data distribution, and compared to fixed-parameter methods, it can significantly improve the convergence speed and final accuracy of the global model.

[0049] Example 6: A global model training device for federated learning based on fuzzy weighting and dynamic clustering, comprising: One or more processors; Memory, used to store one or more computer programs; When one or more programs are executed by one or more processors, the one or more processors perform the method of any one of the embodiments in Examples 1-5.

[0050] Example 7: A computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, cause the processor to perform the method of any one of Examples 1-5.

Claims

1. A federated learning global model training method based on fuzzy weighting and dynamic clustering, characterized in that... Includes the following steps: S1. The server sends the current global model parameters to the selected client. The client performs stochastic gradient descent update with momentum, calculates the local average training loss and class entropy, and reports them to the server along with the updated model parameters. S2. The server flattens the received updated model parameters to obtain a one-dimensional vector, and performs a fast Fourier transform to obtain frequency domain statistical features. S3. The server performs maximum and minimum value normalization on the local average training loss and frequency domain statistical features of all clients, and constructs a joint feature vector reflecting client heterogeneity by combining class entropy. S4. Calculate the overall heterogeneity index based on the normalized local average training loss and frequency domain statistical characteristics. Dynamically calculate the number of clusters K based on the overall heterogeneity index and divide all clients into K groups based on the joint feature vector. S5. Calculate the intra-group weight of each client within the group, and perform weighted aggregation on the client models within the same cluster to obtain the intra-group aggregated model. S6. Calculate the inter-group weights of each group, perform secondary weighted aggregation on all intra-group aggregation models, and obtain the updated global model. S7. During training, threshold detection is performed on the weights within and between groups. Weights that are below the preset lower limit are reset to the lower limit threshold and renormalized. S8. The server adjusts the learning rate of the client's local training in the next round according to the current training round. The adjustment adopts a three-stage strategy that includes a warm-up period, a stabilization period, and a decay period. S9. Evaluate the global model performance on the public test set and determine whether the termination condition is met based on the accuracy or loss metric.

2. The federated learning global model training method based on fuzzy weighting and dynamic clustering according to claim 1, characterized in that, In S1, the category entropy is the entropy calculated based on the client label distribution or prediction distribution, and is used to characterize the degree of balance in the local data distribution. The client reports the local average training loss, class entropy, and updated model parameters to the server. In addition, the client also reports statistics that characterize its local data distribution, including the local gradient norm and the rate of local loss descent.

3. The federated learning global model training method based on fuzzy weighting and dynamic clustering according to claim 1, characterized in that, In S2, frequency domain energy, peak amplitude, mean, and standard deviation are extracted using Fast Fourier Transform, and frequency domain statistical characteristics are obtained. ； In the formula, For the frequency domain statistical characteristics of client k; For the frequency domain energy of client k; The peak value of the amplitude for client k; Let k be the frequency domain standard deviation of the client. Let k be the frequency domain mean of the client. , , , They are respectively , , , The weighting coefficients.

4. The federated learning global model training method based on fuzzy weighting and dynamic clustering according to claim 1, characterized in that, In S3, the local average training loss for client k is... Frequency domain statistical characteristics Perform maximum and minimum value normalization processing to obtain the following results: and , After normalization , After normalization ,Will and Combined into a two-dimensional joint representation vector ; exist Introducing the class entropy of client k The joint feature vector is obtained. The superscript T indicates transpose.

5. The federated learning global model training method based on fuzzy weighting and dynamic clustering according to claim 4, characterized in that, In S4, based on and Calculate the loss and the heterogeneity in the frequency domain: ； ； In the formula, Indicates loss heterogeneity; Indicates frequency domain characteristic heterogeneity; It is a function of standard deviation; The overall heterogeneity index is obtained by combining the results. : ； Using a hybrid linear-truncation strategy, the number of clusters K is calculated: ； ； ； In the formula, This indicates rounding to the nearest integer. Indicates the initial number of clusters; Indicates the number of intermediate clusters; This is the heterogeneity amplification factor in dynamic clustering; , , respectively, represent the preset maximum and minimum number of clusters; b is the basic bias term; For joint eigenvectors Perform K-Means clustering to divide all clients into K groups.

6. The federated learning global model training method based on fuzzy weighting and dynamic clustering according to claim 4, characterized in that, In S5, the calculation of the intra-group weight of each client in the group is specifically as follows: First, the initial weight is calculated, which is either an inverse intra-group weight or a Softmax intra-group weight. The inverse proportional group weights are expressed as follows: ； In the formula, This represents the inverse-proportional weight of client k within the g-th group; , They are respectively and Weighting coefficients; The softmax type within-group weights are represented as follows: ； ； ； In the formula, T1 represents the unnormalized Softmax weight of client k within the g-th group; T1 represents the temperature coefficient in the Softmax function, used to control the smoothness of the weight distribution. This represents the Softmax type in-group weight of client k within the g-th group; This represents the sum of the unnormalized Softmax weights of all clients within the g-th group; Let j represent the set of clients in the g-th group, and j represent one of the clients. , Let represent the normalized local average training loss and normalized frequency domain statistical characteristics of client j within the g-th group, respectively. , These represent the weight coefficients of the normalized local average training loss and the normalized frequency domain statistical features, respectively. choose , Any one of them is used as the initial weight of client k within the g-th group. ; Subsequently Correcting using category entropy : ； In the formula, This represents the weight of client k within the g-th group; This represents the entropy adjustment factor, used to adjust the client's aggregation weights based on category entropy; Ultimately, the intra-group aggregation model is obtained: ； In the formula, This represents the parameters of the within-group aggregation model for the g-th group; This represents the updated model parameters for client k.

7. The federated learning global model training method based on fuzzy weighting and dynamic clustering according to claim 6, characterized in that, In S6, the calculation of the inter-group weights for each group specifically involves first calculating the group-level loss and frequency domain statistics: ； ； In the formula, This represents the average training loss within the g-th group; This represents the average frequency domain statistical characteristics within the g-th group; The between-group weights can be either inverse between-group weights or Softmax between-group weights. The inverse proportion between-group weights are expressed as follows: ； In the formula, This represents the inverse-proportional inter-group weights of the g-th group; , They are respectively , Weighting coefficients in inverse proportional intergroup weighting; The softmax type between-group weights are represented as follows: ； ； In the formula, This represents the Softmax type inter-group weights of the g-th group; , They are respectively , Weighting coefficients in Softmax type inter-group weights; This represents the unnormalized contribution index of the g-th group; This represents the unnormalized contribution index of the h-th group; choose , Any one of them is used as the inter-group weight of the g-th group. ; Finally, the updated global model is obtained: ； In the formula, This represents the final parameters of the global model when the training round is t+1.

8. The federated learning global model training method based on fuzzy weighting and dynamic clustering according to claim 7, characterized in that, In S7, a lower threshold is set. ,for and : ； ； In the formula, This indicates an update to; Then renormalize to ensure the weight sum is 1: ； ； In the formula, This represents the inter-group weight of the h-th group; This represents the weight of client j within the g-th group.

9. The federated learning global model training method based on fuzzy weighting and dynamic clustering according to claim 1, characterized in that, The three-stage strategy in S8, comprising a warm-up period, a stabilization period, and a decay period, is expressed as follows: ； In the formula, This represents the learning rate when there are t training epochs; Indicates the initial learning rate; This represents the learning rate decay coefficient; This represents the decay period constant; This indicates the end of the preset warm-up period; This indicates the start of the preset decay period.

10. The federated learning global model training method based on fuzzy weighting and dynamic clustering according to claim 1, characterized in that, In S9, an evaluation index is calculated after each round of training, and training terminates when a termination condition is met. The termination condition is... or ,in, This represents the test set accuracy when there are t training epochs. This represents the preset optimal accuracy, and t represents the number of training epochs. This is the preset maximum number of training rounds.

Images