Federal learning method and system based on data distribution similarity fuzzy clustering
By allowing customers to be associated with multiple clusters in federated learning and using fuzzy clustering based on data distribution similarity, the problem of low efficiency in customer data utilization in hard clustering methods is solved, and better model convergence, generalization and personalization capabilities are achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- BEIJING JIAOTONG UNIV
- Filing Date
- 2023-03-29
- Publication Date
- 2026-06-23
Smart Images
Figure CN116522184B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of federated learning technology, specifically to a federated learning method and system based on fuzzy clustering of data distribution similarity. Background Technology
[0002] The arrival of the big data era has brought numerous conveniences to people's lives. Data from various industries can be combined to generate immense value, leading to the rapid development of artificial intelligence and machine learning technologies based on data. However, because data is typically distributed across edge computing devices such as mobile phones and personal computers, traditional machine learning methods centralize the data from these devices for unified preprocessing, modeling, and training. The circulation and sharing of private data are strictly restricted, thus creating the problem of "data silos."
[0003] To address this issue, scholars have proposed federated learning technology. Federated learning is a machine learning framework that can both protect local data from leakage and collaboratively train high-performance models from multiple edge devices. It also complies with government regulations and can effectively solve the data silo problem, enabling participants to jointly model data without sharing existing data, thus achieving AI collaboration.
[0004] Currently, one of the most representative hard clustering federated learning methods in existing technologies involves aggregating clients with similar data distributions into several clusters to train a high-performance global model. This involves: first, specifying the number of clusters K (pre-defined), dividing all users into K clusters. Clients within the same cluster are generally considered to have more similar data distributions, allowing these users to train together and more fully utilize data knowledge. Before training iterations begin, the parameters of T clients after initial training are typically sent to the server, where the central server runs the K-means algorithm to cluster the T clients. A traditional federated learning algorithm (such as FedAvg) is then run separately for each cluster, and after multiple training rounds, the final model parameters for the clients in the final cluster are obtained. This method has a wide range of applications and high training efficiency, making it a representative hard clustering federated learning method.
[0005] The existing federated learning hard clustering methods described above have the following drawbacks:
[0006] 1. Hard clustering federated learning cannot effectively utilize the similarity between different clusters. Although the clients participating in federated learning training may have non-independent identically distributed data distributions, two different distributions may still exhibit some similarities.
[0007] 2. In real-world scenarios, customer data distribution is more complex, with the possibility of multiple distributions mixed together. Simply associating customers with a single cluster will lead to inefficient use of customer data, thereby affecting the convergence and generalization of the cluster model. Summary of the Invention
[0008] The purpose of this invention is to provide a federated learning method and system based on data distribution similarity fuzzy clustering that utilizes the correlation between customers and clusters, allows the local update information of participating customers to be learned simultaneously by multiple clusters in each iteration, and provides personalized models to customers through weighted fusion of cluster models, so as to solve at least one of the technical problems existing in the above-mentioned background art.
[0009] To achieve the above objectives, the present invention adopts the following technical solution:
[0010] On the one hand, this invention provides a federated learning method based on fuzzy clustering of data distribution similarity, comprising:
[0011] Broadcast all cluster models;
[0012] Calculate the loss value for each cluster model, select N clusters as customer association clusters based on the loss value, and evaluate the importance of the clusters, where N is a parameter that is predetermined based on the number of clusters and the number of customers.
[0013] Initialize the client-side model and perform local training;
[0014] The cluster model aggregation weights are updated based on the customer sample size and the degree of association between customers and the cluster, resulting in the updated cluster model.
[0015] Preferably, all cluster models are broadcast, including: for a federated learning system, if it is the first round of training, all cluster models are initialized and these initial models are distributed to all clients respectively; if it is in the subsequent rounds of training, the cluster models updated in the previous step are distributed to clients.
[0016] Preferably, the loss value in each cluster is calculated, and based on the loss value, N clusters are selected as the customer's associated clusters, and the importance of the clusters is evaluated, where N is a parameter predetermined based on the number of clusters and the number of customers, including:
[0017] Each user k will receive a broadcast of the cluster model cluster_weight c Users retain all received cluster models;
[0018] The user calculates the loss value for all cluster models using their own private data D, and then sorts the loss values for all cluster models in ascending order:
[0019] Wherein, the loss is calculated by the user using their own private data D in the cluster model cluster_weight. c The loss values are calculated; the N clusters corresponding to the N smallest loss values are selected, and the customer's identity set is these N clusters;
[0020] These N clusters, as the user's associated clusters, indicate that these N clusters have the greatest association with the customer, and the corresponding cluster model will perform better. The smaller the loss value, the higher the importance of the cluster to the customer. After determining the customer's identity set and evaluating the importance of the cluster, the customer performs local training.
[0021] Preferably, the initialization of the client model and local training include: according to the importance of the cluster, the user performs weighted fusion of the cluster models corresponding to the N clusters to which he / she belongs to obtain a new network model; the obtained new network model is used as the client's initial model and iteratively trained locally.
[0022] Preferably, the customer uploads their identity, i.e., which clusters they belong to and the updated model parameters, to the central server. This includes: each customer having their own set of identities, which they upload to the central server; and the customer then uploading the network model trained locally to the server.
[0023] Preferably, updating the cluster model aggregation weights based on the customer sample size and the degree of association between customers and clusters includes: determining the customer groups in each cluster based on the received set of customer identities and aggregating the customer models in them into a new cluster model; aggregating the new cluster model based on the customer sample size and the degree of association between customers and clusters; and using the aggregated cluster model as the new model for the cluster.
[0024] Secondly, the present invention provides a federated learning system based on fuzzy clustering of data distribution similarity, comprising:
[0025] The broadcast module is used to broadcast to all cluster models;
[0026] The evaluation module is used to calculate the loss value of each cluster model. Based on the loss value, N clusters are selected as the customer's associated clusters, and the importance of the clusters is evaluated. The value of N is a parameter that is predetermined based on the number of clusters and the number of customers.
[0027] The training module is used to initialize the client-side model and perform local training.
[0028] The update module is used to update the cluster model aggregation weights based on the customer sample size and the degree of association between customers and the cluster, resulting in an updated cluster model.
[0029] Thirdly, the present invention provides a non-transitory computer-readable storage medium for storing computer instructions, which, when executed by a processor, implement the federated learning method based on fuzzy clustering of data distribution similarity as described above.
[0030] Fourthly, the present invention provides a computer program product, including a computer program that, when run on one or more processors, is used to implement the federated learning method based on data distribution similarity fuzzy clustering as described above.
[0031] Fifthly, the present invention provides an electronic device, comprising: a processor, a memory, and a computer program; wherein the processor is connected to the memory, the computer program is stored in the memory, and when the electronic device is running, the processor executes the computer program stored in the memory to cause the electronic device to execute instructions for implementing the federated learning method based on data distribution similarity fuzzy clustering as described above.
[0032] The beneficial effects of this invention are: by associating a user with multiple clusters, it effectively improves the problem caused by the mixed distribution of various data, allowing the network model to converge and generalize better, and possessing superior personalization capabilities.
[0033] The advantages of additional aspects of the invention will be set forth more clearly in the following description or will be learned by practice of the invention. Attached Figure Description
[0034] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0035] Figure 1 This is a flowchart illustrating the federated learning method based on fuzzy clustering of data distribution similarity as described in an embodiment of the present invention.
[0036] Figure 2 This is a schematic diagram illustrating the problem of mixed data distribution as described in an embodiment of the present invention. Detailed Implementation
[0037] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.
[0038] It will be understood by those skilled in the art that, unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.
[0039] It should also be understood that terms such as those defined in general dictionaries should be understood to have meanings consistent with their meanings in the context of the prior art, and should not be interpreted in an idealized or overly formal sense unless defined as here.
[0040] Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the term “comprising” as used in this specification means the presence of the stated features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, and / or groups thereof.
[0041] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of those different embodiments or examples.
[0042] To facilitate understanding of the present invention, the present invention will be further explained and described below with reference to the accompanying drawings and specific embodiments. However, the specific embodiments do not constitute a limitation on the embodiments of the present invention.
[0043] Those skilled in the art should understand that the accompanying drawings are merely schematic diagrams of embodiments, and the components in the drawings are not necessarily essential for implementing the present invention.
[0044] Example 1
[0045] In this embodiment 1, a federated learning system based on fuzzy clustering of data distribution similarity is first provided, characterized by including:
[0046] The broadcast module is used to broadcast to all cluster models;
[0047] The evaluation module is used to calculate the loss value of each cluster model. Based on the loss value, N clusters are selected as the customer's associated clusters, and the importance of the clusters is evaluated. The value of N is a parameter that is predetermined based on the number of clusters and the number of customers.
[0048] The training module is used to initialize the client-side model and perform local training.
[0049] The update module is used to update the cluster model aggregation weights based on the customer sample size and the degree of association between customers and the cluster, resulting in an updated cluster model.
[0050] In this embodiment, the above-described system is used to implement a federated learning method based on fuzzy clustering of data distribution similarity, including: broadcasting all cluster models; users calculating the loss value in each cluster, selecting N clusters as customer association clusters based on the loss value, and evaluating the importance of the clusters, where N is a parameter predetermined based on the number of clusters and the number of customers; initializing the client model and performing local training; uploading the customer's identity, i.e., which clusters they belong to, and the updated model parameters to the central server; and the central server updating the cluster model aggregation weights based on the customer sample size and the degree of association between the customer and the cluster.
[0051] Broadcast all cluster models, including: for a federated learning system, if it is the first round of training, initialize all cluster models and distribute these initial models to all clients respectively; if it is in the subsequent rounds of training, distribute the cluster model updated in the previous step, i.e., the cluster model after the central server updates the cluster model with aggregated weights according to the client sample size and the degree of association between the client and the cluster, to the clients.
[0052] The user calculates the loss value in each cluster, selects N clusters as the customer's associated clusters based on the loss value, and evaluates the importance of the clusters. Here, N is a parameter predetermined based on the number of clusters and the number of customers, including:
[0053] Each client k will receive the cluster model cluster_weight broadcast by the central server. c The customer retains all cluster models received;
[0054] The customer uses their own private data D to calculate the loss value on all cluster models, and then arranges the loss values on all cluster models in ascending order:
[0055] Wherein, the loss is calculated by the user using their own private data D in the cluster model cluster_weight. c The loss value is calculated based on the number of clusters and clients. N is a hyperparameter predetermined based on the number of clusters and clients.
[0056] The N clusters corresponding to the N smallest loss values are selected, and the set of customer identities is these N clusters;
[0057] All users should perform the above steps;
[0058] These N clusters are considered as customer-related clusters, indicating that these N clusters have the highest degree of association with customers, and the corresponding cluster model will perform better; the smaller the loss value, the higher the importance of the cluster to the customer.
[0059] Unlike traditional clustering federated learning algorithms where a user can only belong to one cluster, this clustering method allows a user to belong to multiple clusters. This type of clustering method is also known as soft clustering federated learning algorithm. After determining the set of customer identities and evaluating the importance of the clusters, the customers undergo local training.
[0060] Initialize the client model and perform local training, including:
[0061] The customer performs a weighted fusion of the cluster models corresponding to their N clusters to obtain a new network model;
[0062] The weighted fusion utilizes the cluster importance described in claim 3;
[0063] The resulting new model is used as the customer's initial model and trained iteratively locally.
[0064] After training converges, the trained model is uploaded to the central server.
[0065] The client uploads its identity (which clusters it belongs to) and updated model parameters to the central server. This includes: each client having its own set of identities, which it uploads to the central server; and the client then uploading the network model it has trained locally to the server.
[0066] The central server updates the cluster model aggregation weights based on the customer sample size and the degree of association between customers and the cluster. This process includes: the central server receiving the identity set from each customer and the updated local model parameters; the central server determining the customer groups in each cluster based on the received customer identity set and aggregating the customer models within them into a new cluster model; the central server aggregating the new cluster model based on two parameters: the customer sample size and the degree of association between customers and the cluster; the aggregated cluster model serving as the new model for the cluster; and broadcasting all cluster models to each customer again, iterating in a loop until the model converges.
[0067] Example 2
[0068] In this embodiment 2, a federated learning method based on fuzzy clustering of data distribution similarity is provided. The specific processing flow is as follows: Figure 1 As shown, the processing steps include the following:
[0069] Step S0: Broadcast the cluster model: Broadcast the initialized or merged cluster model to each user. The purpose of this step is to send the cluster model to subordinate clients, allowing the federated learning training process to operate normally.
[0070] In one embodiment, for a federated learning system, k (k∈[K]) is the kth user, there are a total of K users, N is the number of clusters associated with the customer, and C is the total number of clusters.
[0071] If it is the first round of training, initialize all cluster models and distribute these initial models to all customers respectively;
[0072] If the training is in a subsequent round, the cluster model updated in the previous step, i.e., the cluster model after the central server updates the cluster model with aggregated weights based on the number of customer samples and the degree of association between the customer and the cluster, will be distributed to the customer.
[0073] Step S1: The user calculates the loss value in each cluster, selects N clusters as the customer's associated clusters based on the loss value, and evaluates the importance of the clusters, where N is a parameter that is predetermined based on the number of clusters and the number of customers.
[0074] Figure 2 This diagram illustrates the problem of mixed data distribution addressed in this invention. The diagram shows three data distributions across all user data: Distribution 1, Distribution 2, and Distribution 3. Furthermore, each user's data distribution differs not only in quantity but also in type.
[0075] Traditional hard clustering federated learning methods first specify the number of clusters K, i.e., K is predetermined, and all users are divided into K clusters. Clients in the same cluster are generally considered to have more similar data distributions, and training these users together can make fuller use of data knowledge. Before the training iteration begins, the parameters of the T clients after initial training are often... The data is sent to the server, and then the central server runs algorithms such as K-means to cluster the T clients into S1, S2, ..., S... k Among them, S k Given a set of clients with similar data distributions, users in each set are treated as a whole and federated training continues. For each class k (k∈[K]), a traditional federated learning algorithm (such as FedAvg) is run individually, and after multiple rounds of training, the final result is obtained. This refers to the final model parameters for the clients in cluster k (k∈[K]).
[0076] However, in this hard clustering method, a customer can only belong to one cluster, which cannot effectively utilize the similarity between different clusters. The remaining data knowledge in the client is used in very low efficiency, affecting the convergence and generalization of the cluster model.
[0077] Therefore, this embodiment provides a fuzzy clustering method, where a customer can belong to multiple clusters. Figure 2 As can be seen, customers 1 and 2, which have 70% data distribution 1 and 30% data distribution 2, can belong to both cluster 1 and cluster 2 at the same time. In this way, both data distributions of customers are fully utilized, and the convergence and generalization of the model are better.
[0078] Each client k will receive the cluster model cluster_weight broadcast by the central server. c Users retain all received cluster models;
[0079] Before local training, a client participating in a training round samples a small batch of data (x', y') to perform prediction tasks on all cluster models. That is, the user uses their own private data D to calculate the loss value on the i-th cluster model:
[0080] Then, the loss values of all cluster models are arranged in ascending order:
[0081]
[0082]
[0083] in, For each user, during the t-th iteration of training, their private data D is used to compute the cluster_weight in the cluster model. C The loss value; N is the number of clusters associated with the customer, and C is the total number of clusters; Let C be the loss value of the k-th customer in round t.
[0084] Select the N clusters corresponding to the N smallest loss values; the set of customer identities is these N clusters {C1, C2, ..., C...}. N};
[0085] These N clusters are considered as customer-related clusters, indicating that these N clusters have the highest degree of association with customers, and the corresponding cluster model will perform better; the smaller the loss value, the higher the importance of the cluster to the customer.
[0086] After determining the set of customer identities, the next step is to assess the importance of the cluster:
[0087]
[0088] Where impor[k][j] is the importance weight value corresponding to the j-th important cluster of the k-th customer.
[0089] Following this, the client conducts local training.
[0090] Step S2: Initialize the client model and perform local training.
[0091] The customer performs a weighted fusion of the cluster models corresponding to its N clusters to obtain a new network model. This weighted fusion utilizes the cluster importance described in claim 3. The initialization formula is:
[0092] Where inti_model[k] is the initial model for the kth customer; cluster_weight[i] is the cluster model for the i-th customer.
[0093] The new model is used as the user's initial model, and iterative training is performed locally until the trained model model[k] is obtained.
[0094] After training converges, the trained model is uploaded to the central server.
[0095] Step S3: Upload the customer's identity, i.e., which clusters they belong to, and the updated model parameters to the central server.
[0096] Each customer will have their own set of identities, defining their identity set {C1, C2, ..., C...}. N Uploaded to the central server.
[0097] The customer then uploads the locally trained network model to the central server;
[0098] Step S4: The central server updates the cluster model aggregation weights based on the customer sample size and the degree of association between the customer and the cluster.
[0099] The central server receives the set of identities belonging to each client and the updated local model parameters model[k] from each client;
[0100] The central server determines the customer groups in each cluster based on the received set of customer identities and aggregates the customer models within them into a new cluster model; the aggregation weight is:
[0101] aggregation_weight=α*aggregation_weight_loss+(1-α)*α*aggregation_weight_Sample The central server aggregates a new cluster model based on two parameters: the number of customer samples and the degree of association between the customer and the cluster. The aggregated cluster model is then used as the new model for the cluster.
[0102] The cluster model is then broadcast to each client again, and this process continues iteratively until the model converges.
[0103] In summary, by associating a user with multiple clusters, the problem of mixed data distribution is effectively mitigated, allowing the network model to converge and generalize better, and possessing superior personalization capabilities. Utilizing data distribution similarity and the correlation between customers and clusters for fuzzy clustering allows local updates from participating customers to be learned simultaneously by multiple clusters in each iteration, and provides a more personalized model for customers through weighted fusion of cluster models.
[0104] Example 3
[0105] This embodiment 3 provides a non-transitory computer-readable storage medium for storing computer instructions. When executed by a processor, the computer instructions implement the federated learning method based on fuzzy clustering of data distribution similarity as described above. The method includes:
[0106] Broadcast all cluster models;
[0107] Calculate the loss value for each cluster model, select N clusters as customer association clusters based on the loss value, and evaluate the importance of the clusters, where N is a parameter that is predetermined based on the number of clusters and the number of customers.
[0108] Initialize the client-side model and perform local training;
[0109] The cluster model aggregation weights are updated based on the customer sample size and the degree of association between customers and the cluster, resulting in the updated cluster model.
[0110] Example 4
[0111] This embodiment 4 provides a computer program product, including a computer program that, when run on one or more processors, implements the federated learning method based on data distribution similarity fuzzy clustering as described above. The method includes:
[0112] Broadcast all cluster models;
[0113] Calculate the loss value for each cluster model, select N clusters as customer association clusters based on the loss value, and evaluate the importance of the clusters, where N is a parameter that is predetermined based on the number of clusters and the number of customers.
[0114] Initialize the client-side model and perform local training;
[0115] The cluster model aggregation weights are updated based on the customer sample size and the degree of association between customers and the cluster, resulting in the updated cluster model.
[0116] Example 5
[0117] This embodiment 5 provides an electronic device, including: a processor, a memory, and a computer program; wherein, the processor is connected to the memory, and the computer program is stored in the memory. When the electronic device is running, the processor executes the computer program stored in the memory to cause the electronic device to execute instructions implementing the federated learning method based on data distribution similarity fuzzy clustering as described above. The method includes:
[0118] Broadcast all cluster models;
[0119] Calculate the loss value for each cluster model, select N clusters as customer association clusters based on the loss value, and evaluate the importance of the clusters, where N is a parameter that is predetermined based on the number of clusters and the number of customers.
[0120] Initialize the client-side model and perform local training;
[0121] The cluster model aggregation weights are updated based on the customer sample size and the degree of association between customers and the cluster, resulting in the updated cluster model.
[0122] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0123] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.
[0124] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.
[0125] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment, whereby a series of operational steps are performed to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.
[0126] While the specific embodiments of the present invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that, based on the technical solutions disclosed in the present invention, various modifications or variations that can be made by those skilled in the art without creative effort should be included within the scope of protection of the present invention.
Claims
1. A federated learning method based on fuzzy clustering of data distribution similarity, characterized in that, include: Broadcast all cluster models; Calculate the loss value for each cluster model. Based on the loss value, select N clusters as customer-related clusters and evaluate the importance of each cluster. Here, N is a parameter pre-determined based on the number of clusters and customers, including: each user... Cluster model that receives broadcasts Users retain all received cluster models; users use their own private data. Calculate the loss value for each cluster model, and then sort the loss values for all cluster models in ascending order: ;in, Allowing users to use their own private data Computation in cluster model The loss values are calculated; the N clusters corresponding to the N smallest loss values are selected, and the customer identity set is these N clusters; these N clusters are used as the user association clusters, indicating that these N clusters have the greatest association with the customer, and the corresponding cluster model will perform better; the smaller the loss value, the higher the importance of the cluster to the customer; after determining the customer identity set and evaluating the importance of the clusters, the customer performs local training. Initialize the client-side model and perform local training; The cluster model aggregation weights are updated based on the customer sample size and the degree of association between customers and the cluster, resulting in the updated cluster model.
2. The federated learning method based on fuzzy clustering of data distribution similarity according to claim 1, characterized in that, Broadcast all cluster models, including: for a federated learning system, if it is the first round of training, initialize all cluster models and distribute these initial models to all clients respectively; if it is in a subsequent round of training, distribute the cluster models updated in the previous step to the clients.
3. The federated learning method based on fuzzy clustering of data distribution similarity according to claim 1, characterized in that, Initialize the client model and perform local training, including: according to the importance of the cluster, the user performs weighted fusion of the cluster models corresponding to the N clusters to which they belong to, to obtain a new network model; use the obtained new network model as the client's initial model and perform iterative training on the local machine.
4. The federated learning method based on fuzzy clustering of data distribution similarity according to claim 1, characterized in that, The client uploads its identity, i.e., which clusters it belongs to, and the updated model parameters to the central server. This includes: each client having its own set of identities, which it uploads to the central server; and the client then uploading the network model it has trained locally to the server.
5. The federated learning method based on fuzzy clustering of data distribution similarity according to claim 1, characterized in that, The cluster model aggregation weights are updated based on the customer sample size and the degree of association between customers and clusters. This includes: determining the customer groups in each cluster based on the received set of customer identities and aggregating the customer models in each cluster into a new cluster model; aggregating the new cluster model based on the customer sample size and the degree of association between customers and clusters; and using the aggregated cluster model as the new model for the cluster.
6. A federated learning system based on fuzzy clustering of data distribution similarity, characterized in that, include: The broadcast module is used to broadcast to all cluster models; The evaluation module calculates the loss value for each cluster model. Based on the loss value, it selects N clusters as the customer's associated clusters and evaluates the importance of the clusters. The value of N is a pre-determined parameter based on the number of clusters and customers, including: each user... Cluster model that receives broadcasts Users retain all received cluster models; users use their own private data. Calculate the loss value for each cluster model, and then sort the loss values for all cluster models in ascending order: ;in, Allowing users to use their own private data Computation in cluster model The loss values are calculated; the N clusters corresponding to the N smallest loss values are selected, and the customer identity set is these N clusters; these N clusters are used as the user association clusters, indicating that these N clusters have the greatest association with the customer, and the corresponding cluster model will perform better; the smaller the loss value, the higher the importance of the cluster to the customer; after determining the customer identity set and evaluating the importance of the clusters, the customer performs local training. The training module is used to initialize the client-side model and perform local training. The update module is used to update the cluster model aggregation weights based on the customer sample size and the degree of association between customers and the cluster, resulting in an updated cluster model.
7. A non-transitory computer-readable storage medium, characterized in that, The non-transitory computer-readable storage medium is used to store computer instructions, which, when executed by a processor, implement the federated learning method based on fuzzy clustering of data distribution similarity as described in any one of claims 1-5.
8. A computer program product, characterized in that, Includes a computer program, which, when run on one or more processors, is used to implement the federated learning method based on fuzzy clustering of data distribution similarity as described in any one of claims 1-5.
9. An electronic device, characterized in that, include: The electronic device includes a processor, a memory, and a computer program; wherein the processor is connected to the memory, the computer program is stored in the memory, and when the electronic device is running, the processor executes the computer program stored in the memory to cause the electronic device to execute instructions that implement the federated learning method based on data distribution similarity fuzzy clustering as described in any one of claims 1-5.