Federal learning optimization method and system based on knowledge distillation, and computer readable storage medium

Abstract

The invention provides a federal learning optimization method and system based on knowledge distillation and a computer readable storage medium, and the optimization method comprises the following steps: S1, a server initializes model parameters and issues the model parameters to at least two clients; the client receives model parameters and initializes a local model; s2, training a local model by the client through local data, calculating and combining experience loss and distillation loss of the local data, obtaining updated model parameters through stochastic gradient descent, and uploading the updated model parameters to the server for aggregation; wherein the step of calculating the distillation loss comprises setting different distillation corrections for each sample category; and S3, the server side aggregates the model until the model converges. According to the method, the data heterogeneous property of federal learning in practical application is considered, global knowledge is transmitted to the client by utilizing knowledge distillation, the distillation intensity is adaptively adjusted in combination with data distribution of the client, and the problem of global knowledge forgetting caused by data missing is relieved, so that the model precision is effectively improved.

Description

Technical Field

[0001] This invention relates to the field of federated technology, and in particular to a federated learning optimization method, system, and computer-readable storage medium based on knowledge distillation. Background Technology

[0002] In the internet age, the explosive growth of data, coupled with the continuous maturation of traditional machine learning and deep learning technologies, has led to rapid development in big data analytics and artificial intelligence. Significant applications have been achieved in areas such as image processing, speech recognition, natural language processing, autonomous driving, and medical statistics. Currently, with the widespread use of modern sensor devices, smartphones, and IoT devices, massive amounts of personal information are stored. This data carries information about different users' habits and needs, providing strong data support for personalized services on AI platforms (such as facial recognition payment, assisted diagnosis, and shopping recommendations). Therefore, the need for analyzing users' private data is becoming increasingly urgent. At the same time, behind these seemingly sophisticated intelligent technologies lies a vast amount of private user data, such as physiological characteristics, medical records, and purchasing habits, being collected by various companies. While abundant data can bring better training results and greater economic benefits to models, it also introduces potential data privacy and security risks in traditional centralized machine learning scenarios.

[0003] In traditional machine learning, servers directly collect private data from clients or edge devices and send it to a data center, then use this data for model training. While simple and direct, this approach exposes sensitive private data to the data center, posing a risk of privacy breaches, and incurs significant transmission costs. Furthermore, centralized training places extremely high demands on the computing power of the central server. Against this backdrop, Google AI Research first proposed the theory of federated learning. Federated learning eliminates the need for local data to be sent to the external environment; multiple participants collaboratively train a robust model locally through coordination by a central server. This approach avoids direct data leakage while still achieving the goal of using private data for model training, providing a distributed machine learning method that supports a large number of participants.

[0004] As an emerging paradigm in machine learning, federated learning offers a novel solution for user data sharing, enabling a more optimized model to be generated from the user's original data without leaving the local machine—achieving "model changes without data." This breaks down data silos while ensuring user data privacy and security, fully unlocking the potential value within the data. Currently, federated learning has been preliminarily applied in fields such as medical imaging, smart terminals, and computer vision. Particularly in the medical field, where data is highly sensitive to the protection of personal health information, requiring extremely strict protection of information integrity and confidentiality, federated learning can utilize fragmented and sensitive data from across medical institutions to train usable models.

[0005] Federated learning is essentially a distributed machine learning framework with cryptographic features, enabling collaborative training among multiple participants while ensuring privacy and data security. Specifically, in federated learning, multiple participants transmit encrypted parameters to a central server, collaboratively training to obtain a shared model. The performance of this shared model is nearly identical to that of the model trained using the aggregated raw data from each participant. Unlike traditional distributed machine learning where each computing node samples data from the same dataset on a server, in federated learning, each client generates its own data independently. This results in a misalignment between the local data distribution on each client and the overall data distribution, and the local data on each client is generally heterogeneous, exhibiting a non-independent identically distributed (Non-IID) pattern. In general, data heterogeneity includes five scenarios: biased data feature distribution (different distribution characteristics across client data); biased data label distribution (different label distributions across client data); biased data feature and label correspondence (including cases of the same label with different features and the same feature with different labels); and data skew and class imbalance (unequal quantity and class distribution of local data on each client).

[0006] Experiments have shown that the existing non-IID data can affect the accuracy of the model. Summary of the Invention

[0007] In view of this, the technical problem to be solved by the present invention is to provide a federated learning optimization method, system and computer-readable storage medium based on knowledge distillation, which can effectively improve the accuracy of the model.

[0008] The technical solution of this invention is implemented as follows:

[0009] This invention proposes a federated learning optimization method based on knowledge distillation, comprising the following steps:

[0010] S1. The server initializes the model parameter w and sends it to at least two clients; the clients receive the model parameter w and initialize their local models.

[0011] S2. The client trains a local model using local data, calculates and combines the empirical loss and distillation loss of the local data, and obtains updated model parameters through stochastic gradient descent, which are then uploaded to the server for aggregation.

[0012] The calculation of the distillation loss includes setting different distillation corrections for each sample category;

[0013] S3, The server-side aggregation model converges.

[0014] In detail, S1 specifically includes:

[0015] The server randomly selects a set of active clients and sends the model parameter w to the clients in the set. Each client receives the model parameter w and uses this parameter to initialize its local model.

[0016] Preferably, S2 specifically includes:

[0017] The client employs stochastic gradient descent, simultaneously calculating the empirical loss and distillation loss for local data, and updates the data based on a weighted sum of these two losses.

[0018]

[0019] In the above formula, 1 y The sample label representing unique heat, p c and q c These represent the soft outputs of the client model and the global model under this category after normalization, respectively.

[0020] Preferably, S2 specifically includes:

[0021] Different distillation corrections are set for each category, i.e.

[0022]

[0023] α c =β(1-q) c )·ln(n c / N+∈)

[0024] In the above formula, the hyperparameter τ represents the softening temperature of knowledge distillation, w c and h x These represent the classifier parameters and feature matrix of the client-side model, respectively, α c is the distillation correction factor, and β and ∈ are hyperparameters.

[0025] Preferably, S3 specifically includes:

[0026] After receiving all the client model parameters that participated in the training, the server aggregates a new round of global model data based on the amount of data on each client's local machine.

[0027]

[0028] The server continuously repeats the above process until the model converges.

[0029] This invention also proposes a federated learning optimization system based on knowledge distillation, comprising a server and at least two clients;

[0030] The server is used to initialize model parameters w and send them to at least two clients; and to accept model parameters w uploaded by the clients. (t+1) And aggregate the models until they converge;

[0031] The client is used to receive model parameters w from the server and initialize the local model; through local data D m Train a local model, calculate and combine the local data D m The empirical loss and distillation loss are used to obtain updated model parameters w through stochastic gradient descent. (t+1) Uploaded to the server for aggregation;

[0032] The calculation of the distillation loss includes setting different distillation corrections for each sample category.

[0033] Preferably, the client employs stochastic gradient descent while simultaneously calculating local data D. m The experience loss and distillation loss are calculated based on a weighted sum and update of both.

[0034]

[0035] In the above formula, 1 y The sample label representing unique heat, p c and q c These represent the soft outputs of the client model and the global model under this category after normalization, respectively.

[0036] Preferably, the client is specifically used for:

[0037] Different distillation corrections are set for each category, i.e.

[0038]

[0039] In the above formula, the hyperparameter τ represents the softening temperature of knowledge distillation, w c and h x These represent the classifier parameters and feature matrix of the client-side model, respectively, α c is the distillation correction factor, and β and ∈ are hyperparameters.

[0040] Preferably, after receiving all client model parameters participating in training, the server aggregates a new round of global models based on the amount of local data on each client, i.e.:

[0041]

[0042] The server continuously repeats the above process until the model converges.

[0043] This invention also proposes a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the above-described knowledge distillation-based federated learning optimization method.

[0044] The present invention proposes a federated learning optimization method, system, and computer-readable storage medium based on knowledge distillation. It takes into account the heterogeneous nature of data in federated learning in practical applications, uses knowledge distillation to transmit global knowledge to the client, and adaptively adjusts the distillation intensity based on the client's data distribution to alleviate the problem of global knowledge forgetting caused by data loss, thereby effectively improving the model accuracy. Attached Figure Description

[0045] Figure 1 This is a flowchart of the federated learning optimization method based on knowledge distillation proposed in an embodiment of the present invention;

[0046] Figure 2 This is a schematic diagram of the federated learning optimization method based on knowledge distillation proposed in an embodiment of the present invention;

[0047] Figure 3 This is a schematic diagram of the optimized federated learning structure based on knowledge distillation proposed in an embodiment of the present invention. Detailed Implementation

[0048] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0049] like Figure 1 and Figure 2 As shown in the figure, this embodiment of the invention proposes a federated learning optimization method based on knowledge distillation, which includes the following steps:

[0050] S1. The server initializes the model parameter w and sends it to at least two clients; the client receives the model parameter w and initializes its local model.

[0051] S2. The client trains a local model using local data, calculates and combines the empirical loss and distillation loss of the local data, and obtains updated model parameters through stochastic gradient descent, which are then uploaded to the server for aggregation.

[0052] The calculation of distillation loss includes setting different distillation corrections for each sample category;

[0053] S3, server-side aggregation model until model convergence.

[0054] As can be seen, the federated learning optimization method based on knowledge distillation proposed in this embodiment of the invention takes into account the heterogeneous nature of data in federated learning in practical applications, uses knowledge distillation to transmit global knowledge to the client, and adaptively adjusts the distillation intensity based on the client's data distribution to alleviate the problem of global knowledge forgetting caused by data loss, thereby effectively improving the model accuracy.

[0055] In one embodiment of the present invention, the federated learning process involves K clients (k = 1, 2, ..., K) and an aggregation server participating in the training, wherein each client has an independent dataset D. k Consider a global dataset D that follows a long-tailed distribution, where D = ∪ k D k At the start of training, the server initializes the global model parameters w. At the start of each training round, the server randomly selects a set of active clients S. t The global model parameter w is sent to the clients in the collection. Each client receives the model parameter w and uses it to initialize its local model.

[0056] In this embodiment of the invention, the client uses local data D m Training the model. During training, the client-side distillation loss is the distance between the locally updated model parameters and the original global model parameters. In each round of local training, the client uses stochastic gradient descent and simultaneously calculates the empirical loss and distillation loss for the local data, updating the model based on a weighted sum of the two.

[0057]

[0058] In the above formula, 1 y The sample label representing unique heat, p c and q c These represent the soft outputs of the client model and the global model after normalization, respectively. Since the number of samples for each class in the client's local dataset can vary greatly—in other words, the client might only have samples for a few classes, while the number of samples for other classes is zero—maintaining the same distillation intensity for each class will result in a deterioration in distillation performance as the amount of data decreases. Therefore, different distillation corrections need to be applied to each class.

[0059]

[0060] In the above formula, the hyperparameter τ represents the softening temperature of knowledge distillation, w c and h x These represent the classifier parameters and feature matrix of the client-side model, respectively, α c is the distillation correction factor, and β and ∈ are hyperparameters. When category c is missing in the local data (n... c =0), α c This will increase, thereby strengthening the learning of that category during knowledge distillation and avoiding the "forgetting effect" caused by missing local data. For the majority class (n) in the local data c (larger), α c This reduces the number of parameters, thus relaxing the constraints on these categories and preventing the local model from overfitting the majority class. This dynamic adjustment mechanism allows the model to achieve a better balance between the majority and minority classes, improving overall generalization performance. Finally, each client updates the model parameters w. (t+1) Uploaded to the server for aggregation.

[0061] After receiving all the client model parameters that participated in the training, the server aggregates a new round of global model data based on the amount of data on each client's local machine.

[0062]

[0063] The server continuously repeats the above process until the model converges.

[0064] In this embodiment, the goal is to integrate the local model parameters trained on multiple clients into a unified global model. Since different clients have different amounts of data, the weights of the clients can be dynamically adjusted based on the amount of data.

[0065] Taking the medical diagnosis federated learning system as an example, Client 1 (tertiary hospital): 100,000 medical images (weight 50%); Client 2 (community hospital): 80,000 images (weight 40%); Client 3 (clinic): 20,000 images (weight 10%). Then, during the aggregation process, the global model parameters = (0.5 × tertiary hospital model) + (0.4 × community model) + (0.1 × clinic model).

[0066] This aggregation method can use high-quality data from hospitals to dominate the model while retaining key features from smaller data sources.

[0067] like Figure 3 As shown, in another embodiment of the present invention, the present invention also proposes a federated learning optimization system based on knowledge distillation, including a server 1 and at least two clients 2;

[0068] Server 1 is used to initialize model parameters w and send them to at least two clients 2; it also receives model parameters w uploaded by client 2. (t+1) And aggregate the models until they converge;

[0069] Client 2 is used to receive model parameters w from server 1 and initialize the local model; it uses local data D m Train a local model, compute and combine local data D m The empirical loss and distillation loss are used to obtain updated model parameters w through stochastic gradient descent. (t+1) Uploaded to the server for aggregation;

[0070] The calculation of distillation loss includes setting different distillation corrections for each sample category.

[0071] As can be seen, the federated learning optimization system based on knowledge distillation proposed in this embodiment of the invention takes into account the heterogeneous nature of data in federated learning in practical applications, uses knowledge distillation to transmit global knowledge to the client, and adaptively adjusts the distillation intensity based on the client's data distribution to alleviate the problem of global knowledge forgetting caused by data loss, thereby effectively improving the model accuracy.

[0072] In one embodiment of the present invention, the federated learning process involves K clients (k = 1, 2, ..., K) and an aggregation server participating in the training, wherein each client has an independent dataset D. k Considering that the global dataset D follows a long-tailed distribution, where D = ∪ k D k At the start of training, the server initializes the global model parameters w. At the start of each training round, the server randomly selects a set of active clients S. t The global model parameter w is sent to the clients in the collection. Each client receives the model parameter w and uses it to initialize its local model.

[0073] The client uses local data D m Training the model. During training, the client-side distillation loss is the distance between the locally updated model parameters and the original global model parameters. In each round of local training, the client uses stochastic gradient descent and simultaneously calculates the empirical loss and distillation loss for the local data, updating the model based on a weighted sum of the two.

[0074]

[0075] In the above formula, 1 y The sample label representing unique heat, p c and q cThese represent the soft outputs of the client model and the global model after normalization, respectively. Since the number of samples for each class in the client's local dataset can vary greatly—in other words, the client might only have samples for a few classes, while the number of samples for other classes is zero—maintaining the same distillation intensity for each class will result in a deterioration in distillation performance as the amount of data decreases. Therefore, different distillation corrections need to be applied to each class.

[0076]

[0077] In the above formula, the hyperparameter τ represents the softening temperature of knowledge distillation, w c and h x These represent the classifier parameters and feature matrix of the client-side model, respectively, α c is the distillation correction factor, and β and ∈ are hyperparameters. When category c is missing in the local data (n... c =0), α c This will increase, thereby strengthening the learning of that category during knowledge distillation and avoiding the "forgetting effect" caused by missing local data. For the majority class (n) in the local data c (larger), α c This reduces the number of parameters, thus relaxing the constraints on these categories and preventing the local model from overfitting the majority class. This dynamic adjustment mechanism allows the model to achieve a better balance between the majority and minority classes, improving overall generalization performance. Finally, each client updates the model parameters w. (t+1) Uploaded to the server for aggregation.

[0078] After receiving all the client model parameters that participated in the training, the server aggregates a new round of global model data based on the amount of data on each client's local machine.

[0079]

[0080] The server continuously repeats the above process until the model converges.

[0081] In one embodiment of the present invention, the technical effectiveness of the present invention in processing long-tail data in federated learning is further illustrated through a comparative simulation experiment. In the following comparative simulation experiment, the default settings of the system parameters are as follows. The datasets used are CIFAR-10 / 100 and CINIC-10. A common data heterogeneity scenario is considered, namely the Dirichlet distribution, controlled by κ. Furthermore, to further enhance data heterogeneity and simulate real-world scenarios, the original balanced dataset is shaped into a long-tail distribution for the Dirichlet setting. In all experiments, the number of clients is 100, and a total of 100 rounds of communication are performed globally.

[0082] The comparison method mainly considers the following three aspects:

[0083] 1. Traditional federated learning algorithms, such as FedAvg, FedProx, and FedMOON.

[0084] 2. Correction algorithms based on data heterogeneity, such as FedRS and FedLC.

[0085] 3. The federal knowledge distillation algorithm most relevant to this application, such as FedNTD and FedVLS.

[0086] The training accuracy of all methods, the number of communication rounds required to reach the target accuracy, and the training time are used as performance comparison standards. Based on the above methods and specific parameter settings, the performance of the algorithms is evaluated by changing the dataset and the degree of imbalance. Each experiment is repeated 5 times, and the average of the 5 experiments is taken as the experimental result. In the experimental comparison table below, the method for processing long-tail data in federated learning according to the embodiments of this invention is abbreviated as: FedLCD.

[0087] Table 1: Accuracy of different algorithms on different datasets and with different levels of imbalance

[0088]

[0089] Table 1 compares the accuracy of the algorithm of this invention with seven other algorithms on different datasets and with varying degrees of imbalance. Overall, the method of this invention achieves the highest accuracy among all compared methods. As the label skewness increases, other compared methods struggle to maintain their performance. Taking the FedLC method as an example, on the CIFAR100 dataset, when the skewness coefficient κ = 0.05, its performance significantly decreases compared to CIFAR10, even approaching the level of the traditional FedAvg method. This performance degradation stems from the presence of a large number of missing classes in each client under extreme conditions, especially as the number of classes increases. In contrast, the method of this invention consistently maintains excellent performance, particularly in highly skewed label distribution scenarios. For example, on the CIFAR100 dataset (κ = 0.05), this method achieves a test accuracy of 37.69%, which not only outperforms other methods but also demonstrates that this method maintains high accuracy under varying degrees of label skewness. This achievement highlights the effectiveness of this method in addressing the problem of decreased accuracy for missing classes and successfully alleviates the phenomenon of overfitting to local data distributions.

[0090] Compared with the prior art, the present invention has the following obvious and prominent substantive features and significant technological advancements:

[0091] This invention takes into account the case where the data distribution in federated learning is non-independent and identically distributed, and the proposed method can effectively improve the accuracy of the model.

[0092] This invention takes into account the optimization of knowledge distillation in federated learning for cases with uneven distribution of client data, thereby improving the effectiveness of knowledge distillation in this scenario.

[0093] The method of this invention can be combined with most traditional federated learning algorithms to further improve performance.

[0094] This invention also proposes a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the above-described knowledge distillation-based federated learning optimization method.

[0095] In summary, the present invention provides a federated learning optimization method, system, and computer-readable storage medium based on knowledge distillation. In a federated learning system with multiple clients and an aggregation server (server), considering the heterogeneous nature of data in practical applications of federated learning, the invention utilizes knowledge distillation to transmit global knowledge to clients and adaptively adjusts the distillation intensity based on the client data distribution to alleviate the problem of global knowledge forgetting caused by data loss, thereby effectively improving model accuracy.

[0096] Through the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware, and of course, it can also be implemented by special hardware including application-specific integrated circuits, special CPUs, special memory, special components, etc. Generally, any function performed by a computer program can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be diverse, such as analog circuits, digital circuits, or special-purpose circuits. However, for this application, software program implementation is more often the preferred implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a readable storage medium, such as a computer floppy disk, USB flash drive, mobile hard disk, ROM, RAM, magnetic disk, or optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods of the various embodiments of this application.

[0097] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the flow or function according to the embodiments of this application is generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can store or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0098] Finally, it should be noted that the above description is merely a preferred embodiment of the present invention and is only used to illustrate the technical solution of the present invention, and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention are included within the scope of protection of the present invention.

Claims

1. A federated learning optimization method based on knowledge distillation, characterized in that, Includes the following steps: S1. The server initializes the model parameter w and sends it to at least two clients; the clients receive the model parameter w and initialize their local models. S2, the client accesses local data D m Train a local model, calculate and combine the local data D m The empirical loss and distillation loss are used to obtain updated model parameters w through stochastic gradient descent. (t+1) Uploaded to the server for aggregation; The calculation of the distillation loss includes setting different distillation corrections for each sample category; S3, The server-side aggregation model converges.

2. The federated learning optimization method based on knowledge distillation as described in claim 1, characterized in that, S1 specifically includes: The server randomly selects an active client set S. t The model parameter w is sent to the clients in the set, and each client receives the model parameter w and uses it to initialize its local model.

3. The federated learning optimization method based on knowledge distillation as described in claim 1, characterized in that, S2 specifically includes: The client employs stochastic gradient descent while simultaneously computing local data D. m The experience loss and distillation loss are calculated based on a weighted sum and update of both. In the above formula, 1 y The sample label representing unique heat, p c and q c These represent the soft outputs of the client model and the global model under this category after normalization, respectively.

4. The federated learning optimization method based on knowledge distillation as described in claim 3, characterized in that, S2 specifically includes: Set different distillation corrections for each category, i.e. a c =β(1-q c )·ln(n c / N+∈) In the above formula, the hyperparameter τ represents the softening temperature of knowledge distillation, w c and h x These represent the classifier parameters and feature matrix of the client-side model, respectively, α c is the distillation correction factor, and β and ∈ are hyperparameters.

5. The federated learning optimization method based on knowledge distillation as described in any one of claims 1-4, characterized in that, S3 specifically includes: After receiving all the client model parameters that participated in the training, the server aggregates a new round of global model data based on the amount of data on each client's local machine. The server continuously repeats the above process until the model converges.

6. A federated learning optimization system based on knowledge distillation, characterized in that, Includes a server and at least two clients; The server is used to initialize the model parameters w and send them to at least two clients. Accept the model parameters w uploaded by the client. (t+1) And aggregate the models until they converge; The client is used to receive model parameters w from the server and initialize the local model; through local data D m Train a local model, calculate and combine the local data D m The empirical loss and distillation loss are used to obtain updated model parameters w through stochastic gradient descent. (t+1) Uploaded to the server for aggregation; The calculation of the distillation loss includes setting different distillation corrections for each sample category.

7. The federated learning optimization system based on knowledge distillation as described in claim 6, characterized in that, The client employs stochastic gradient descent while simultaneously calculating local data D. m The experience loss and distillation loss are calculated based on a weighted sum and update of both. In the above formula, 1 y The sample label representing unique heat, p c and q c These represent the soft outputs of the client model and the global model under this category after normalization, respectively.

8. The federated learning optimization system based on knowledge distillation as described in claim 7, characterized in that, The client is specifically used for: Set different distillation corrections for each category, i.e. a c =β(1-q c )·ln(n c / N+∈) In the above formula, the hyperparameter τ represents the softening temperature of knowledge distillation, w c and h x These represent the classifier parameters and feature matrix of the client-side model, respectively, α c is the distillation correction factor, and β and ∈ are hyperparameters.

9. The federated learning optimization system based on knowledge distillation as described in any one of claims 6-8, characterized in that, After receiving the parameters of all client model participants in the training, the server aggregates a new round of global model data based on the amount of local data on each client. The server continuously repeats the above process until the model converges.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the knowledge distillation-based federated learning optimization method as described in any one of claims 1-5.

Images