A cluster-in-and-out aggregation period construction method and system for federated learning facing clustering and a storage medium thereof

Abstract

The application discloses a cluster-in and cluster-out aggregation period construction method and system for clustered federal learning and a storage medium thereof, and comprises the following steps: all clients in each non-intersecting cluster receive a global model corresponding to a certain training round issued by a center server, and the global model is iteratively trained and updated by using a local data set in the client to obtain a local model; when each preset aggregation iteration period I is reached, each cluster head collects the local model in the cluster to which the cluster head belongs to perform intra-cluster aggregation, and issues the obtained intra-cluster aggregation model to the clients in the cluster to repeatedly perform training and updating; when each preset aggregation iteration period G is reached, each cluster head uploads the latest intra-cluster aggregation model to the center server to perform inter-cluster aggregation, and issues the global model obtained after the inter-cluster aggregation to all clients in each cluster; a new round of intra-cluster and inter-cluster training and updating is repeatedly performed until the global model reaches a predetermined training precision.

Description

Technical Field

[0001] This invention belongs to the field of device clustering and resource scheduling technology in wireless federated learning, specifically involving a method, system and storage medium for constructing cluster-inside-outside aggregation cycles for clustered federated learning. Background Technology

[0002] With the advancement of computing power, machine learning, as a technology for analyzing and processing massive amounts of data, has been widely applied to human society. The integration of deep learning-based technologies with industrial artificial intelligence has attracted widespread attention to solve data-driven industrial problems in applications such as intelligent connected vehicles, smart grids, and intelligent manufacturing. However, the development of machine learning technology faces two major challenges: privacy data breaches and data silos. Furthermore, developing and training high-quality learning models requires collecting large amounts of data from different IoT devices or industrial nodes for centralized learning, and aggregating data on a central server incurs significant communication costs. Federated learning, as a promising distributed machine learning method, involves a central server and multiple data-holding clients collaborating to train a general machine learning model without exchanging data. Iterative gradient descent algorithms are used to share and update model parameters only. Therefore, learning from private client data becomes possible while protecting client privacy. Moreover, federated learning can achieve mutual benefit for all participating clients because training algorithms using a larger dataset by aggregating the private data of all participants can significantly improve the performance of the learning model.

[0003] Traditional centralized federated learning architectures still employ a single central server for global aggregation and multiple rounds of local model updates and uploads. Global resource consumption depends on the communication energy consumption of client-side local training and multiple rounds of model uploads. Furthermore, the degree of client non-independent and identically distributed nature indirectly determines the number of times clients need to perform uplink communication to transmit models and conduct local training. The resource scheduling scheme based on clustered federated learning aims to reduce the number of inter-cluster communication aggregations by setting up multiple clusters and performing local averaging with corresponding cluster heads before global model aggregation. Based on the similarity between the data distribution of the defined clusters and the global data distribution, the mathematical relationship between the inter-cluster model communication aggregation period and the intra-cluster model communication aggregation period is dynamically adjusted. Since local communication bandwidth is typically much higher than WAN bandwidth, this method can alleviate network congestion, improve client communication robustness, and reduce the overall inbound traffic to the server, thereby improving the efficiency of federated learning, accelerating model convergence, and reducing global energy consumption and latency. Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings of the prior art and provide a method, system and storage medium for constructing intra- and extra-cluster aggregation cycles for clustered federated learning. By deriving and analyzing relevant iterative formulas, the influencing factors between inter- and intra-cluster aggregation effects are sought. Based on this, the number of intra- and inter-cluster communication times is dynamically adjusted to achieve the goal of reducing resource consumption.

[0005] To achieve the above objectives, the present invention is implemented using the following technical solution:

[0006] In a first aspect, the present invention provides a method for constructing intra- and extra-cluster aggregation cycles for clustered federated learning, comprising the following steps:

[0007] Each client in a disjoint cluster receives the global model corresponding to a certain training round from the central server, and uses the local dataset to iteratively train and update the global model to obtain the local model.

[0008] At each preset aggregation iteration period I, each cluster head collects the local models of all clients within its cluster, performs intra-cluster aggregation, and distributes the obtained intra-cluster aggregated model to the intra-cluster clients to repeat the training and update process.

[0009] Every preset aggregation iteration period G, each cluster head uploads the latest intra-cluster aggregation model to the central server for inter-cluster aggregation, and distributes the global model obtained after inter-cluster aggregation to all clients within each cluster.

[0010] Repeat the new round of intra-cluster and inter-cluster training and updates until the global model reaches the predetermined training accuracy.

[0011] Furthermore, it also includes the following steps:

[0012] Construct a system model consisting of a single central server and multiple clients, where the client set is represented as... Model parameters after co-training parameterization All clients are divided Non-intersecting clusters The corresponding cluster heads within each cluster are as follows: The number of clients within each cluster satisfies: , Client Have local dataset ,in, For the client The size of the dataset.

[0013] Furthermore, during the process of iteratively training and updating the global model using the local dataset within the client to obtain the local model, the client... Update the local model using stochastic gradient descent. It can be expressed by the formula:

[0014]

[0015] in, This is the local model used for training round t. For federal learning rate, It is a client Local stochastic gradient, Indicates that it comes from the client. Local dataset Random data samples.

[0016] Furthermore, the client in local iteration rounds During cluster aggregation, local models from all clients within the cluster are locally averaged and then aggregated at the cluster head. The intra-cluster aggregation result completed above is expressed by the following formula:

[0017]

[0018] In the formula, For iterative rounds Time cluster head Parameters of the intra-cluster aggregation model.

[0019] Furthermore, the client in local iteration rounds Inter-cluster model aggregation is performed, and the intra-cluster aggregation models from all clusters are globally averaged. The inter-cluster aggregation result completed on the central server is expressed by the formula:

[0020]

[0021] In the formula, for The global model parameters are obtained by the time-center server through inter-cluster aggregation.

[0022] Furthermore, the method for determining the preset aggregation iteration cycle transformation relationship between intra-cluster model aggregation and inter-cluster model aggregation, using a known and fixed clustering method, includes the following steps:

[0023] Step 2-1): For a fixed clustering method, and considering the non-convexity of the loss function, the convergence conclusion is as follows:

[0024]

[0025] In the formula, For global training iteration rounds, , Let be the expected function. The gradient of the global loss function. The initial global loss function is... The optimal global loss function, learning rate satisfy , Let be the Lipschitz constant, satisfying ,in , They represent the client respectively. In the model , The gradient value of the local loss function under the given conditions; t represents the number of iterations. For local iteration rounds The global model parameters obtained by the time-center server through inter-cluster aggregation are used to divide all clients. Non-intersecting clusters The value of i ranges from 1 to N;

[0026] The client gradient variance threshold satisfies ,in It is a client Local stochastic gradient, Indicates that it comes from the client. Local dataset Random data samples;

[0027] The global divergence threshold satisfies ,in , Representing clusters The global gradient of the loss function; For clusters The intra-cluster divergence threshold satisfies:

[0028] ;

[0029] Step 2-2): When the global training reaches the preset convergence accuracy and the total number of training iterations... If the following conditions are met without change: , ,but , That is, the number of inter-cluster aggregations is reduced. The number of intra-cluster aggregations increases. ;at this time ,make ,but To ensure that the polymerization cycle is changed to , Even after global training, the results still tend to be close to the initial cycle. , Global convergence accuracy;

[0030] Steps 2-3): Let , , , , , All the above settings are constants. By rearranging and eliminating terms in step 2-2, we can obtain:

[0031] ;

[0032] Since the left side of the above equation is a constant, let:

[0033] , get about Quadratic function in one variable:

[0034] ,

[0035] Solving the above equation and discarding the negative while keeping the positive yields the transformation relationship between the intra-cluster model aggregation period and the inter-cluster model aggregation period:

[0036] .

[0037] Secondly, this invention provides a cluster-inside / outside aggregation cycle system for clustered federated learning, the system consisting of a single central server and multiple clients, wherein the client set is represented as follows: Model parameters after co-training parameterization All clients are divided Non-intersecting clusters The corresponding cluster heads within each cluster are as follows: The number of clients within each cluster satisfies: , Client Have local dataset ,in, For the client The size of the dataset.

[0038] The central server is used to send the global model corresponding to a certain training round to each cluster client; and to receive the latest cluster aggregated model uploaded by each cluster head, perform inter-cluster aggregation, and send the global model obtained after inter-cluster aggregation to all clients in each cluster.

[0039] The cluster head is used to perform intra-cluster aggregation of the local models of all clients within its cluster at each preset aggregation iteration period I, and then distribute the obtained intra-cluster aggregated model to the clients within the cluster for repeated training and updates; and to upload the latest intra-cluster aggregated model to the central server for inter-cluster aggregation at each preset aggregation iteration period G.

[0040] The client is used to receive the global model transmitted by the central server, and to iteratively train and update the global model using the local dataset to obtain the local model.

[0041] Furthermore, the client uses stochastic gradient descent to update the local model. It can be expressed by the formula:

[0042]

[0043] in, This is the local model used for training round t. For federal learning rate, It is a client Local stochastic gradient, Indicates that it comes from the client. Local dataset Random data samples;

[0044] At the cluster head The intra-cluster aggregation result completed above is expressed by the following formula:

[0045]

[0046] In the formula, For iterative rounds Time cluster head Parameters of the intra-cluster aggregation model;

[0047] The inter-cluster aggregation results completed on the central server are expressed by the following formula:

[0048]

[0049] In the formula, for The global model parameters are obtained by the time-center server through inter-cluster aggregation.

[0050] Thirdly, the present invention provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the cluster-inside-outside aggregation cycle construction method for cluster-oriented federated learning as described in the first aspect.

[0051] Compared with the prior art, the beneficial effects achieved by the present invention are as follows: The present invention provides a method, system and storage medium for constructing cluster-inside and outside aggregation cycles for clustered federated learning. The method adopted is different from the federated learning framework of single central server and direct communication between multiple clients considered in most previous studies. Considering the complexity of wireless networks and transmission and communication cost issues, the present invention focuses on the learning architecture of clustered federation, and only selects cluster heads to communicate directly with the central server.

[0052] The method of this invention takes into account the high cost of inter-cluster communication aggregation under the clustered federated learning architecture. By considering the period settings of intra-cluster and inter-cluster model aggregation, it dynamically adjusts the training period relationship between intra-cluster model aggregation and inter-cluster communication aggregation, aiming to reduce global communication energy consumption and improve the efficiency of federated learning. Attached Figure Description

[0053] Figure 1 This is a model diagram of an intra-cluster aggregation cycle system for clustered federated learning provided in an embodiment of the present invention.

[0054] Figure 2 The flowchart illustrates the overall process of constructing an intra- and extra-cluster aggregation cycle for clustered federated learning, as provided in this embodiment of the invention. Detailed Implementation

[0055] The present invention will be further described below with reference to the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and should not be used to limit the scope of protection of the present invention.

[0056] like Figure 2 As shown in the figure, this embodiment of the invention provides a method for constructing cluster-inside-outside aggregation cycles for clustered federated learning, including the following steps:

[0057] Step S1: All clients within each disjoint cluster receive the global model corresponding to a certain training round from the central server, and iteratively train and update the global model using their local datasets to obtain their local models; the central server broadcasts this directly to all clients, including the cluster heads.

[0058] Step S2: During each preset aggregation iteration period I, each cluster head collects the local models of all clients within its cluster, performs intra-cluster aggregation, and distributes the obtained intra-cluster aggregated model to the intra-cluster clients to repeat the training and update process.

[0059] Step S3: Every preset aggregation iteration period G, each cluster head uploads the latest intra-cluster aggregation model to the central server for inter-cluster aggregation, and distributes the global model obtained after inter-cluster aggregation to all clients within each cluster.

[0060] Step S4: Repeat steps S1 to S3 above: perform a new round of intra-cluster and inter-cluster training updates until the global model reaches the predetermined training accuracy.

[0061] In clustered federated learning, when clients are clustered, each client within a cluster needs to upload the trained model to the cluster head after local updates and iterations. The cluster head then performs local intra-cluster aggregation and distributes the model to other clients within the cluster for a new round of local updates, or uploads it to the central server to perform inter-cluster aggregation to obtain a new global model, which is then distributed to all clients for a new round of training iterations. Considering that the communication cost of inter-cluster aggregation is higher than that of intra-cluster aggregation, this invention derives and analyzes relevant iterative formulas to seek the influencing factors between the effects of inter-cluster and intra-cluster aggregation. Based on this, the number of intra-cluster and inter-cluster communication times is dynamically adjusted to reduce resource consumption.

[0062] Since the communication cost of inter-cluster model aggregation is higher than that of intra-cluster model aggregation, this study uses iterative formulas to derive and analyze the influencing factors between the inter-cluster and intra-cluster model aggregation cycles. Based on this analysis, the communication cycles for both inter-cluster and intra-cluster model aggregation are dynamically adjusted to reduce the number of communications between the cluster head and the central server, while increasing the number of communications between the cluster head and intra-cluster clients. This achieves the goal of reducing communication costs.

[0063] This invention provides a cluster-inside / outside aggregation cycle design method for clustered federated learning, wherein a system consists of a single central server and multiple clients, and the client set is represented as follows. Model parameters after co-training parameterization All clients are divided Non-intersecting clusters The corresponding cluster heads are respectively The number of clients in each cluster meets the following requirements. , The client has a local dataset. ,in For the client The size of the dataset.

[0064] The specific steps are as follows:

[0065] Step 1: The central server in the training round The time-center server will display the global model. It is distributed to all clients in all clusters. The clients use their local datasets to train and update the global model, thus obtaining their local models. ;

[0066] Step 2: Design the aggregation iteration period within each cluster as follows: ,go through After round-local iterative training updates, the client Upload the local model to its respective cluster head. Cluster head Collect the local models of all clients within its cluster and perform intra-cluster aggregation;

[0067] Step 3: Design the inter-cluster aggregation iteration cycle as follows Cluster head The cluster aggregation model is then distributed to the cluster clients, and step two is repeated. After each round, the latest intra-cluster aggregation model is uploaded to the central server for inter-cluster aggregation, and the new global model is then distributed to all clients.

[0068] Step 4: Repeat steps 1, 2, and 3 to conduct a new round of global training until the global model reaches the predetermined training accuracy.

[0069] In step one of this embodiment, during local iterative training, the client updates the local model using stochastic gradient descent:

[0070]

[0071] in, For federal learning rate, It is the client's local stochastic gradient. Indicates data sourced from the client's local dataset. Random data samples.

[0072] In step two of this embodiment, the client performs local iteration. In-cluster model aggregation is performed, and local models from all clients within the cluster are averaged locally. The calculation is performed at the cluster head. The results of the intra-cluster aggregation completed above are as follows:

[0073]

[0074] in, for Time cluster head The cluster head then broadcasts the intra-cluster aggregation model parameters to the intra-cluster clients for continued local training.

[0075] In step three of this embodiment, the client performs local iteration. Inter-cluster model aggregation is performed, and the intra-cluster aggregated models from all clusters are globally averaged. The inter-cluster aggregation results calculated on the central server are as follows:

[0076]

[0077] in, for The central server performs inter-cluster aggregation to obtain global model parameters, and then distributes the new global model to all clients for a new round of training iterations.

[0078] Assuming all clients perform synchronous updates, federated learning occurs both within and between clusters.

[0079] In this embodiment, given a known and fixed clustering method, the training cycle transformation relationship between intra-cluster model aggregation and inter-cluster model aggregation is achieved through the following steps:

[0080] Step 2-1): For a fixed clustering method, and considering the non-convexity of the loss function, the convergence conclusion is as follows:

[0081]

[0082] in, For global training iteration rounds, , Let be the expected function. The gradient of the global loss function. The initial global loss function is... The optimal global loss function, learning rate satisfy , Let be the Lipschitz constant, satisfying ,in , They represent the client respectively. In the model , The gradient value of the local loss function under the given conditions, where, It also represents the global model, and its sum The two can be the same or different;

[0083] The client gradient variance threshold satisfies ,in It is the client's local stochastic gradient. This represents a local dataset from the client. Random data samples;

[0084] The global divergence threshold satisfies ,in , Representing clusters The global gradient of the loss function; For clusters The intra-cluster divergence threshold satisfies .

[0085] Step 2-2): When the global training reaches the preset convergence accuracy and the total number of training iterations... If the following conditions are met without change: , ,but , That is, the number of inter-cluster aggregations is reduced. The number of intra-cluster aggregations increases. .at this time ,make ,but To ensure that the polymerization cycle is changed to , Post-training global operations still tend to be close to the initial cycle. , The global convergence accuracy, when rearranged from the above equation, is:

[0086]

[0087] Steps 2-3): Let , , , , , Substituting into the above equation and rearranging the terms, we get:

[0088]

[0089] Steps 2-4): Multiply both sides of the equation by... ,get

[0090]

[0091] Steps 2-5): Since the left side of the above equation is a constant, let:

[0092] , get about Quadratic function in one variable:

[0093]

[0094] Solving the above equation yields:

[0095]

[0096] Discarding the negative and taking the positive, we get:

[0097]

[0098] The above equation reflects the transformation relationship between the intra-cluster model aggregation cycle and the inter-cluster model aggregation cycle.

[0099] This invention provides a cluster-inside / outside aggregation cycle system for clustered federated learning. The system consists of a single central server and multiple clients, where the client set is represented as follows: Model parameters after co-training parameterization All clients are divided Non-intersecting clusters The corresponding cluster heads within each cluster are as follows: The number of clients within each cluster satisfies: , The client has a local dataset. ,in, The size of the client's dataset.

[0100] The central server is used to send the global model corresponding to a certain training round to each cluster client; and to receive the latest cluster aggregated model uploaded by each cluster head, perform inter-cluster aggregation, and send the global model obtained after inter-cluster aggregation to all clients in each cluster.

[0101] The cluster head is used to perform aggregation iterations every preset period. At that time, the local models of all clients within the cluster are aggregated within the cluster, and the resulting aggregated model is distributed to the clients within the cluster for repeated training and updates; and in each preset aggregation iteration cycle... At that time, the latest intra-cluster aggregation model is uploaded to the central server for inter-cluster aggregation;

[0102] The client is used to receive the global model transmitted by the cluster head, and to iteratively train and update the global model using the local dataset to obtain the local model.

[0103] Among them, the client Update the local model using stochastic gradient descent. It can be expressed by the formula:

[0104]

[0105] in, This is the local model used for training round t. For federal learning rate, It is the client's local stochastic gradient. Indicates data sourced from the client's local dataset. Random data samples;

[0106] At the cluster head The intra-cluster aggregation result completed above is expressed by the following formula:

[0107]

[0108] In the formula, For iterative rounds Time cluster head Parameters of the intra-cluster aggregation model;

[0109] The inter-cluster aggregation results completed on the central server are expressed by the following formula:

[0110]

[0111] In the formula, for The global model parameters are obtained by the time-center server through inter-cluster aggregation.

[0112] Furthermore, embodiments of the present invention also provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the above-described method steps for constructing cluster-inside-outside aggregation cycles for clustered federated learning.

[0113] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the technical principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for constructing intra- and extra-cluster aggregation cycles for clustered federated learning, characterized in that, Includes the following steps: Each client in a disjoint cluster receives the global model corresponding to a certain training round from the central server, and uses the local dataset to iteratively train and update the global model to obtain the local model. In each preset aggregation iteration cycle At that time, each cluster head collects the local models of all clients within its cluster, performs intra-cluster aggregation, and distributes the obtained intra-cluster aggregated model to the clients within the cluster to repeat the training and update process. In each preset aggregation iteration cycle At that time, each cluster head uploads the latest intra-cluster aggregation model to the central server for inter-cluster aggregation, and distributes the global model obtained after inter-cluster aggregation to all clients within each cluster; Repeat the new round of intra-cluster and inter-cluster training and updates until the global model reaches the predetermined training accuracy; The method for determining the preset aggregation iteration cycle transformation relationship between intra-cluster model aggregation and inter-cluster model aggregation, using a known and fixed clustering method, includes the following steps: Step 2-1): For a fixed clustering method, and considering the non-convexity of the loss function, the convergence conclusion is as follows: In the formula, For global training iteration rounds, , Let be the expected function. The gradient of the global loss function. The initial global loss function is... The optimal global loss function, learning rate satisfy , Let be the Lipschitz constant, satisfying ,in , They represent the client respectively. In the model , The gradient value of the local loss function under the given conditions; t represents the number of iterations. For local iteration rounds The global model parameters obtained by the time-center server through inter-cluster aggregation are used to divide all clients. Non-intersecting clusters The value of i ranges from 1 to N; The client gradient variance threshold satisfies ,in It is a client Local stochastic gradient, Indicates that it comes from the client. Local dataset Random data samples; The global divergence threshold satisfies ,in , Representing clusters The global gradient of the loss function; For clusters The intra-cluster divergence threshold satisfies ; Step 2-2): When the global training reaches the preset convergence accuracy and the total number of training iterations... If the following conditions are met without change: , ,but , That is, the number of inter-cluster aggregations is reduced. The number of intra-cluster aggregations increases. ;at this time ,make ,but To ensure that the polymerization cycle is changed to , Even after global training, the results still tend to be close to the initial cycle. , Global convergence accuracy; Steps 2-3): Let , , , , , All the above settings are constants. By rearranging and eliminating terms in step 2-2, we can obtain: ； Since the left side of the above equation is a constant, let: , get about Quadratic function in one variable: ， Solving the above equation and discarding the negative while keeping the positive yields the transformation relationship between the intra-cluster model aggregation period and the inter-cluster model aggregation period: 。 2. The method for constructing intra- and extra-cluster aggregation cycles for clustered federated learning according to claim 1, characterized in that, It also includes the following steps: Construct a system model consisting of a single central server and multiple clients, where the client set is represented as... Model parameters after co-training parameterization All clients are divided Non-intersecting clusters The corresponding cluster heads within each cluster are as follows: The number of clients within each cluster satisfies: , Client Have local dataset ,in, For the client The size of the dataset.

3. The method for constructing cluster-inside-outside aggregation cycles for clustered federated learning according to claim 2, characterized in that, During the process of iteratively training and updating the global model using a local dataset within the client to obtain the local model, the client... Update the local model using stochastic gradient descent. It can be expressed by the formula: in, This is the local model used for training round t. For federal learning rate, It is a client Local stochastic gradient, Indicates that it comes from the client. Local dataset Random data samples.

4. The method for constructing intra- and extra-cluster aggregation cycles for clustered federated learning according to claim 3, characterized in that, The client in local iteration rounds During cluster aggregation, local models from all clients within the cluster are locally averaged and then aggregated at the cluster head. The intra-cluster aggregation result completed above is expressed by the following formula: In the formula, For iterative rounds Time cluster head Parameters of the intra-cluster aggregation model.

5. The method for constructing intra- and extra-cluster aggregation cycles for clustered federated learning according to claim 4, characterized in that... The client in the local iteration round Inter-cluster model aggregation is performed, and the intra-cluster aggregation models from all clusters are globally averaged. The inter-cluster aggregation result completed on the central server is expressed by the formula: In the formula, for The global model parameters are obtained by the time-center server through inter-cluster aggregation.

6. A cluster-intra-cluster aggregation cycle system for clustered federated learning, characterized in that, The system implementing the cluster-inside / outside aggregation cycle construction method for clustered federated learning as described in any one of claims 1 to 5 consists of a single central server and multiple clients, wherein the client set is represented as follows: Model parameters after co-training parameterization All clients are divided Non-intersecting clusters The corresponding cluster heads within each cluster are as follows: The number of clients within each cluster satisfies: , Client Have local dataset ,in, For the client The size of the dataset; The central server is used to send the global model corresponding to a certain training round to each cluster client; and to receive the latest cluster aggregated model uploaded by each cluster head, perform inter-cluster aggregation, and send the global model obtained after inter-cluster aggregation to all clients in each cluster. The cluster head is used to perform aggregation iterations every preset period. At that time, the local models of all clients within the cluster are aggregated within the cluster, and the resulting aggregated model is distributed to the clients within the cluster for repeated training and updates; and in each preset aggregation iteration cycle... At that time, the latest intra-cluster aggregation model is uploaded to the central server for inter-cluster aggregation; The client is used to receive the global model transmitted by the central server, and to iteratively train and update the global model using the local dataset to obtain the local model.

7. The cluster-inside / outside aggregation cycle system for clustered federated learning according to claim 6, characterized in that, The client updates the local model using stochastic gradient descent. It can be expressed by the formula: in, This is the local model used for training round t. For federal learning rate, It is a client Local stochastic gradient, Indicates that it comes from the client. Local dataset Random data samples; At the cluster head The intra-cluster aggregation result completed above is expressed by the following formula: In the formula, For iterative rounds Time cluster head Parameters of the intra-cluster aggregation model; The inter-cluster aggregation results completed on the central server are expressed by the following formula: In the formula, for The global model parameters are obtained by the time-center server through inter-cluster aggregation.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the cluster-inside-outside aggregation cycle construction method for cluster-oriented federated learning as described in any one of claims 1 to 5.

Images