Long-tail federated learning method and system based on knowledge distillation feature generation

Abstract

The application discloses a long-tail data federated learning method and system based on knowledge distillation feature generation, and a lightweight generation model is trained on a server. The input of the generation model is a label, and the output is corresponding features. The generated features can simulate the distribution of real features. In the training process of the personalized model, more features are generated for the local minority class on each client, and fewer features are generated for other classes, which will help to alleviate the serious imbalance of each client data. In addition, the global model is aggregated from the local model, performs better in the global head class, and can obtain high-quality and rich-information features. Extracting the knowledge of the global model into the personalized model can help improve the performance of the personalized model. The personalized model is trained by generating features for knowledge distillation. This can effectively solve the problem of federated heterogeneous long-tail data distribution, and further improve the model performance under personalized federated learning.

Description

Technical Field

[0001] This invention belongs to the field of federated learning technology, specifically relating to a federated learning method and system for imbalanced data based on knowledge distillation feature generation. Background Technology

[0002] In recent years, deep learning technology has played a crucial role in the field of artificial intelligence, and its success largely depends on massive amounts of training data. When using deep learning models, the common practice is to collect large amounts of data for training on the server side, with data not being shared between models. However, in reality, data silos exist between enterprises, making centralized modeling on the server side increasingly difficult. Federated learning was proposed to enable cross-organizational collaborative modeling while ensuring data privacy, security, and compliance, thereby improving the performance of artificial intelligence models. It is now widely used in artificial intelligence research.

[0003] A key challenge in federated learning is the heterogeneity of data distribution across different organizations (clients). FedAvg is a classic approach that maintains a global model on the server side. Clients first download the global model and train it on their local data. Then, the server aggregates updates from the local models uploaded by the clients, repeating this process until convergence. However, FedAvg does not effectively address the heterogeneity problem because it does not consider the differences in data distribution among clients, and a single global model cannot perform well on every client. Therefore, personalized federated learning has been proposed, aiming to learn a customized model for each client and enable them to benefit from participating in federated learning.

[0004] One approach to federated learning is to divide the global model into individualized parts and a global part. Another common method in personalized federated learning is to restrict the distance between the individualized model and the global model using regularization.

[0005] However, these methods typically assume a balanced global distribution, while in reality, data distributions are often long-tailed. A few classes (the head class) contain a large number of samples, but most classes (the tail class) occupy only a small number of samples. This long-tailed distribution exacerbates the imbalance within each client under similar heterogeneous conditions, leading to a sharp decline in local model performance. In federated learning, Ratio Loss addresses imbalance by estimating the global imbalance. CReFF addresses the problem of a global long-tailed distribution by retraining the classifier using federated features. However, neither of these methods considers generating personalized models. Summary of the Invention

[0006] In view of the above, the purpose of this invention is to provide an imbalanced data federated learning method and system based on knowledge distillation feature generation, which can effectively solve the problem of federated heterogeneous long-tail data distribution and further improve the model performance under personalized federated learning.

[0007] To achieve the above-mentioned objectives, an embodiment provides a long-tail data federated learning method based on knowledge distillation feature generation, comprising the following steps: After receiving the uploaded local model, label distribution for each class, and local feature prototype for each class, the server updates the global model based on the local model parameters, updates the generated model based on the label distribution for each class and local feature prototype for each class, and then distributes the updated generated model and global model to each client. After receiving the global model and the generated model, each client generates local features for each class based on the generated model and calculates the prototype of each local feature. It updates the local model based on each local feature and local data, and uploads the updated local model, the label distribution for each class, and the prototype of each local feature. It initializes the personalized model based on the global model, generates multiple features for minority class samples and fewer features for majority class samples based on the generated model, updates the initialized personalized model based on the multiple features, fewer features, and local data, and transfers the knowledge of the global model to the personalized model based on knowledge distillation.

[0008] In one embodiment, the generative model is used to generate features based on input labels and random noise, and to update the loss function used by the generative model based on the distribution of each label class and the prototype of each feature class. for: in, , , Indicates the weighting coefficient. Represents classification loss. Indicates sample adjustment loss. Indicates comparative loss; Classification loss Represented as: in, y Indicates the input label, f Indicates the generated features, v k This represents the classifier in the local model of the k-th client. L CE (( v k , f ),y) means to f Enter to v k The predicted value andy The cross-entropy loss, where p(y) represents the label distribution. This means inputting y into the parameter. The feature distribution obtained from the generative model, where E represents the expectation. S T Represents a collection of clients; Sample adjustment loss Represented as: in, z Represents a random noise vector. i and j Represents a vector index. M Indicates the total number of vectors. Represents the L2 norm; Comparative loss Represented as: in, This represents the local feature prototype of the y-th class for the k-th client. Indicates that the sample number is 1 y A collection of clients of the class. This represents the prototype of the global feature of class y. Indicates the first Class tags, This represents the feature generated based on the label of the y-th class. A ( y ) represents a set of labels.

[0009] In one embodiment, generating local features for each class based on a generative model and calculating a prototype for each class of local features includes: in, For the first k The first client's y Local data of the class, where x represents a local sample. Indicates the first k Local data of each client, Indicates the first k The first client's y The local characteristic prototype of the class, Indicates the first k The local model in each client includes feature extractors. t Indicates the round, R This indicates the final batch.

[0010] In one embodiment, the loss function used to update the local model based on each type of local feature and local data is: in, Labels indicating random sampling This indicates the use of generative models based on labels. The generated features Indicates will Enter to The predicted value and Cross-entropy loss, t Indicates the round, r Indicates batch, Indicates the first k Local data of each client, Indicates the first k A local model Indicates that the basis will Enter to The balanced softmax loss is calculated from the predicted values.

[0011] In one embodiment, initializing the personalized model based on the global model includes: The global model includes a global feature extractor and a global classifier, and the personalized model includes a local feature extractor and a local classifier. The local feature extractor is initialized using the global feature extractor.

[0012] In one embodiment, generating multiple features for minority class samples and generating fewer features for majority class samples based on the generative model includes: For minority class samples, multiple labels are sampled, and a generative model is used to generate multiple features based on the multiple labels. For samples of the majority class with a small number of labels, a generative model is used to generate a small number of features based on the small number of labels.

[0013] In one embodiment, the personalized model is updated based on multiple features, few features, and local data after initialization. Simultaneously, knowledge distillation is used to transfer knowledge from the global model to the personalized model. The loss function employed is... for: in, Indicates weight, Represents classification loss. Indicates distillation loss; Classification loss Represented as: in, This represents multi-label sampling for minority class samples and few-label sampling for majority class samples. Indicates based on The generated features Indicates the first k Each personalized model contains a classifier. Indicates will Enter to The predicted value and Cross-entropy loss, Indicates the first k Local data of each client, Indicates the first k A personalized model, Indicates will Enter to The cross-entropy between the predicted value and the original label y; Distillation loss Represented as: in, Denotes KL divergence, Represents the global model. t Indicates the round, r Indicates batch, This represents the softmax function. Indicates will Input to global model The softmax value, Indicates will Input to personalized model The softmax value.

[0014] To achieve the above-mentioned objectives, the embodiments also provide a long-tail data federated learning system based on knowledge distillation feature generation, including a server and various clients. The server receives the uploaded local model, label distribution for each class, and local feature prototype for each class. It then updates the global model based on the local model parameters, updates the generated model based on the label distribution for each class and local feature prototype for each class, and distributes the updated generated model and global model to each client. Each client is used to receive the global model and the generated model, generate local features for each class based on the generated model and calculate the prototype of each local feature, update the local model based on each local feature and local data, and upload the updated local model, the label distribution for each class and the prototype of each local feature. It initializes the personalized model based on the global model, generates multiple features for minority class samples based on the generated model, updates the initialized personalized model based on the multiple features and local data, and transfers the knowledge of the global model to the personalized model based on knowledge distillation.

[0015] Compared with the prior art, the beneficial effects of the present invention include at least the following: A lightweight generative model is trained on the server. The input to the generative model is labels, and the output is the corresponding features. The generated features can simulate the distribution of real features. During the training of the personalized model, more features are generated for the local minority class on each client, and fewer features are generated for other classes. This helps alleviate the severe imbalance of data on each client. Furthermore, the global model is aggregated from the local models, performs better in the global head class, and obtains high-quality and information-rich features. Extracting knowledge from the global model into the personalized model can help improve its performance. The personalized model is trained through knowledge distillation using generated features. This effectively addresses the problem of federated heterogeneous long-tail data distribution, further improving the performance of the model under personalized federated learning. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a flowchart of the long-tail data federated learning method based on knowledge distillation feature generation provided in this embodiment of the invention; Figure 2 This is a flowchart of the server-side workflow provided in an embodiment of the present invention; Figure 3 This is a flowchart illustrating the local model update process provided by the client in this embodiment of the invention. Figure 4 This is a flowchart of the client-side personalized model update process provided in this embodiment of the invention; Figure 5 This is a schematic diagram of the structure of a long-tail data federated learning system based on knowledge distillation feature generation provided in an embodiment of the present invention. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and do not limit the scope of protection of this invention.

[0019] To address the technical problem of poor performance of personalized models caused by the combined issues of heterogeneous data and long-tail distributions in personalized federated learning, this embodiment provides a long-tail data federated learning method and system based on knowledge distillation feature generation to improve model performance under personalized federated learning.

[0020] like Figure 1 As shown in the embodiment, the long-tail data federated learning method based on knowledge distillation feature generation provides a system that implements this federated learning method, which includes a server and clients. Each client communicates with the server, but the clients do not communicate with each other. Based on this system, the federated learning method includes the following steps: S110: The server receives local models, label distributions for each class, and prototypes of local features for each class uploaded by each client, and generates and updates and distributes the model and the global model.

[0021] Research based on federated learning and long-tail learning methods shows that, in cases of heterogeneous data and long-tailed data distribution, the model's feature extractor is less affected by the data distribution. Therefore, the global model is divided into two parts: the first part is the global feature extractor. The parameterization is given by u. The second part is the global classifier. Parameterize it as At this time, the sample Features Depend on The generation and prediction results are generated by The server distributes the global model to each selected client. Each selected client uploads its local model, label distribution, and local feature prototypes, such as... Figure 2 As shown, the server executes the following steps: S210, after receiving the uploaded local model, label distribution for each class, and prototype of local feature for each class; S220, the server updates the global model based on the local model parameters, and updates the generated model based on the label distribution for each class and prototype of local feature for each class; S230, the updated generated model and global model are distributed to each client.

[0022] In this embodiment, for the global model, a global update is performed based on the uploaded local model, and the update method is not limited.

[0023] In this embodiment, the generative model generates features f based on the input label y and random noise z. Since the generated features need to be correctly classified, parameter updates are required. During the update process, the loss function used for the update is determined based on the label distribution and feature prototypes uploaded by the client for each class. for: in, , , Indicates the weighting coefficient. Represents classification loss, Indicates sample adjustment loss. Indicates comparative loss; Classification loss Represented as: in, y Indicates the input label, f Indicates the generated features, v k This represents the classifier in the local model of the k-th client. L CE (( v k , f ),y) means to f Enter to v k The predicted value and y The cross-entropy loss, where p(y) represents the label distribution. This means inputting y into the parameter. The feature distribution obtained from the generative model, where E represents the expectation. S T Represents a collection of clients; Since using only classification loss leads to insufficient diversity in generated features, sample adjustment loss is added. , represented as: in, z Represents a random noise vector. i and j Represents a vector index. M Indicates the total number of vectors. Represents the L2 norm; Since there are cases where generated features can be correctly identified but are dissimilar to real features, these generated features may harm the model's performance, thus adding contrastive loss. To limit the distance between generated features and true features, it is expressed as: in, This represents the local feature prototype of the y-th class for the k-th client. Indicates that the sample number is 1 y A collection of clients of the class. This represents the prototype of the global feature of class y. Indicates the first Class tags, This represents the feature generated based on the label of the y-th class. A ( y ) represents a set of labels.

[0024] After updating the generated model and the global model, the updated generated model and global model are distributed to each client.

[0025] S120: Each client receives the global model and the generated model, generates feature and local feature prototypes based on the generated model, and updates the local model and the personalized model.

[0026] In the embodiments, such as Figure 3 As shown, each client performs the following steps: S310, each client receives the global model and the generated model sent by the server; S320, generates local features for each class based on the generated model and calculates the prototype of each class of local features; S330, updates the local model based on each class of local features and local data; S340, uploads the updated local model, the label distribution for each class, and the prototype of each class of local features.

[0027] Since the generator model carries global information, it updates the local model by generating features corresponding to the corresponding categories through random sampling of labels and combining them with real data. The following loss function is used for training the local model: in, Labels indicating random sampling This indicates the use of generative models based on labels. The generated features Indicates will Enter to The predicted value and Cross-entropy loss, t Indicates the round, r Indicates batch, Indicates the first k Local data of each client, Indicates the first k A local model Indicates that the basis will Enter to The balanced Softmax loss is calculated from the predicted values.

[0028] To ensure that the features generated by the generative model closely approximate those of real samples, the mean value of each class's features is used as the local feature prototype for each client. The local feature prototype is constructed as follows: in, For the first k The first client's y Local data of the class, where x represents a local sample. Indicates the first k Local data of each client, Indicates the firstk The first client's y The local characteristic prototype of the class, Indicates the first k The local model in each client includes feature extractors. t Indicates the round, R This indicates the final batch. To reduce computational costs, only features from the last epoch are counted.

[0029] After updating the local model and local feature prototypes, upload the updated local model, label distribution for each class, and local feature prototypes for each class to the server for the next round of federated learning.

[0030] In this embodiment, after receiving the global model and the generated model, each client also updates its personalized model, such as... Figure 4 As shown, the specific steps include: S410, initializing the personalized model based on the global model; S420, generating multiple features for minority class samples and generating fewer features for majority class samples based on the generative model; S430, updating the initialized personalized model based on multiple features, fewer features, and local data, while simultaneously transferring the knowledge of the global model to the personalized model based on knowledge distillation.

[0031] In this embodiment, the personalized model includes a local feature extractor and a local classifier. When initializing the personalized model based on the global model, the local feature extractor is initialized using the global feature extractor.

[0032] In this embodiment, based on the sample distribution of each client, more features are generated for minority class samples and fewer features are generated for majority class samples to alleviate the problem of extreme imbalance in data distribution among clients. Specifically, generating more features for minority class samples and fewer features for majority class samples based on the generative model includes: sampling a large number of labels for minority class samples and using the generative model to generate a large number of features based on the large number of labels; sampling a small number of labels for majority class samples and using the generative model to generate a small number of features based on the small number of labels.

[0033] In the training process of the personalized model, generated features and real samples participate in the training together. First, it's crucial to ensure that real samples can be correctly classified, a fundamental operation in image classification tasks. Since the generative model can simulate the real feature distribution and carries information from other clients, generated features can be used to further rebalance features, helping to alleviate the imbalance in data distribution within the client. Furthermore, the global model can acquire global knowledge and provide a global perspective. Knowledge distillation is introduced to transfer knowledge from the global model to the personalized model. Kullback-Leibler divergence is used to measure the difference between the global and personalized models. Therefore, the loss function used for optimizing the personalized model parameters is... for: in, Indicates weight, Represents classification loss, Indicates distillation loss; Classification loss Represented as: in, This represents multi-label sampling for minority class samples and few-label sampling for majority class samples. Indicates based on The generated features Indicates the first k Each personalized model contains a classifier. Indicates will Enter to The predicted value and Cross-entropy loss, Indicates the first k Local data of each client, Indicates the first k A personalized model, Indicates will Enter to The cross-entropy between the predicted value and the original label y; Distillation loss Represented as: in, Denotes KL divergence, Represents the global model. t Indicates the round, r Indicates batch, This represents the softmax function. Indicates will Input to global model The softmax value, Indicates will Input to personalized model The softmax value.

[0034] Based on the same inventive concept, the embodiments also provide a long-tail data federated learning system based on knowledge distillation feature generation, such as... Figure 5 As shown, it includes server 510 and each client 520.

[0035] Among them, the server 510 is used to receive the uploaded local model, the label distribution of each class and the prototype of each class of local features, update the global model based on the local model parameters, update the generated model based on the label distribution of each class and the prototype of each class of local features, and send the updated generated model and global model to each client. Each client 520 is used to receive the global model and the generative model, generate local features for each class based on the generative model and calculate the prototype of each local feature, update the local model based on each local feature and local data, and upload the updated local model, the label distribution for each class and the prototype of each local feature. It initializes the personalized model based on the global model, generates multiple features for minority class samples based on the generative model, updates the initialized personalized model based on the multiple features and local data, and transfers the knowledge of the global model to the personalized model based on knowledge distillation.

[0036] It should be noted that the long-tail data federated learning system based on knowledge distillation feature generation provided in the above embodiments and the long-tail data federated learning method based on knowledge distillation feature generation belong to the same concept. For details of its specific implementation process, please refer to the long-tail data federated learning method based on knowledge distillation feature generation, which will not be repeated here.

[0037] To verify the effectiveness of the federated learning of long-tail data based on knowledge distillation features, the embodiment was validated on a dataset, specifically including: First, prepare a long-tail classification dataset and split the training dataset for each client.

[0038] CIFAR-10 and CIFAR-100 were used. CIFAR-10 is a small dataset for object recognition, containing RGB color images for 10 classes. The images are 32x32 pixels, with 50,000 training images and 10,000 test images. CIFAR-100 has 100 classes, with 50,000 training images and 10,000 test images. The imbalance factors for all datasets were 10, 50, and 100, calculated by dividing the number of images from the class with the largest global sample size by the number of images from the class with the smallest global sample size. Hyperparameters were used. The Dirichlet distribution is used to control the degree of heterogeneity of client data. The closer the data is to zero, the more severe the data heterogeneity. In this invention, a simulation was performed... The degree of data heterogeneity at =0.1.

[0039] Then, the federated learning framework is set up and the model is initialized.

[0040] The ResNet32 architecture was used as the backbone model on the CIFAR-10-LT and CIFAR-100-LT datasets. All experiments were implemented using the PyTorch framework and run on an NVIDIA GeForce RTX 3080. A total of 20 client data distributions were simulated, and in each iteration, 10 clients' local models were randomly selected for federated aggregation. The batch size for generated features of the real data was set to 32, and the SGD optimizer was used. The global training epochs were set to 200.

[0041] Next, federated learning will be performed using the long-tail data federated learning method described above.

[0042] Table 1 shows the Top-1 accuracy (%) comparison results of the present invention with several other federated learning methods on the CIFAR-10-LT and CIFAR-100-LT datasets with imbalance levels of 100, 50, and 10, and a data heterogeneity of 0.1. The bolded results in the table represent the optimal results for each metric. Analysis shows that the method provided by the present invention achieves high model accuracy.

[0043] Table 1 The specific embodiments described above illustrate the technical solution and beneficial effects of the present invention in detail. It should be understood that the above description is only the most preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, additions, and equivalent substitutions made within the scope of the principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A federated learning method for long-tail data based on knowledge distillation feature generation, characterized in that, Includes the following steps: After receiving the uploaded local model, label distribution for each class, and local feature prototype for each class, the server updates the global model based on the local model parameters. The generated model is used to generate features based on the input labels and random noise. The generated features need to be correctly classified. The generated model is updated based on the label distribution for each class and local feature prototype for each class. The updated generated model and global model are then distributed to each client. The loss functions used to update the generative model based on the label distribution and local feature prototypes of each class include classification loss, sample conditioning loss, and contrast loss used to limit the distance between generated features and real features; After receiving the global model and the generated model, each client generates local features for each class based on the generated model and calculates the prototype of each local feature. It updates the local model based on each local feature and local data, and uploads the updated local model, the label distribution for each class, and the prototype of each local feature. It initializes the personalized model based on the global model, generates multiple features for minority class samples and few features for majority class samples based on the generated model, updates the initialized personalized model based on the multiple features, few features, and local data, and transfers the knowledge of the global model to the personalized model based on knowledge distillation. Among them, generating multiple features for minority class samples and generating fewer features for majority class samples based on the generative model includes: sampling multiple labels for minority class samples and using the generative model to generate multiple features based on the multiple labels; sampling fewer labels for majority class samples and using the generative model to generate fewer features based on the fewer labels.

2. The long-tail data federated learning method based on knowledge distillation feature generation according to claim 1, characterized in that, The loss function used in the generation model is updated based on the label distribution and feature prototype of each class. for: in, , , Indicates the weighting coefficient. Represents classification loss, Indicates sample adjustment loss. Indicates comparative loss; Classification loss Represented as: in, y Indicates the input label, f Indicates the generated features, v k This represents the classifier in the local model of the k-th client. L CE (( v k , f ),y) means to f Enter to v k The predicted value and y The cross-entropy loss, where p(y) represents the label distribution. This means inputting y into the parameter. The feature distribution obtained from the generative model, where E represents the expectation. S T Represents a collection of clients; Sample adjustment loss Represented as: in, z Represents a random noise vector. i and j Represents a vector index. M Indicates the total number of vectors. Represents the L2 norm; Comparative loss Represented as: in, This represents the local feature prototype of the y-th class for the k-th client. Indicates that the sample number is 1 y A collection of clients of the class. This represents the prototype of the global feature of class y. Indicates the first Class tags, This represents the feature generated based on the label of the y-th class. A ( y ) represents a set of labels.

3. The long-tail data federated learning method based on knowledge distillation feature generation according to claim 1, characterized in that, The process of generating local features for each class based on a generative model and calculating the prototype of each local feature class includes: in, For the first k The first client's y Local data of the class, where x represents a local sample. Indicates the first k Local data of each client, Indicates the first k The first client's y The local characteristic prototype of the class, Indicates the first k The local model in each client includes feature extractors. t Indicates the round, R This indicates the final batch.

4. The long-tail data federated learning method based on knowledge distillation feature generation according to claim 1, characterized in that, The loss function used to update the local model based on each type of local feature and local data is: in, Labels indicating random sampling This indicates the use of generative models based on labels. The generated features Indicates will Enter to The predicted value and Cross-entropy loss, t Indicates the round, r Indicates batch, Indicates the first k Local data of each client, Indicates the first k A local model Indicates that the basis will Enter to The balanced softmax loss is calculated from the predicted values.

5. The long-tail data federated learning method based on knowledge distillation feature generation according to claim 1, characterized in that, The process of initializing the personalized model based on the global model includes: The global model includes a global feature extractor and a global classifier, and the personalized model includes a local feature extractor and a local classifier. The local feature extractor is initialized using the global feature extractor.

6. The long-tail data federated learning method based on knowledge distillation feature generation according to claim 1, characterized in that, The personalized model is updated based on multiple features, few features, and local data. Simultaneously, knowledge distillation is used to transfer knowledge from the global model to the personalized model. The loss function employed is... for: in, Indicates weight, Represents classification loss, Indicates distillation loss; Classification loss Represented as: in, This represents multi-label sampling for minority class samples and few-label sampling for majority class samples. Indicates based on The generated features Indicates the first k Each personalized model contains a classifier. Indicates will Enter to The predicted value and Cross-entropy loss, Indicates the first k Local data of each client, Indicates the first k A personalized model, Indicates will Enter to The cross-entropy between the predicted value and the original label y; Distillation loss Represented as: in, Denotes KL divergence, Represents the global model. t Indicates the round, r Indicates batch, This represents the softmax function. Indicates will Input to global model The softmax value, Indicates will Input to personalized model The softmax value.

7. A long-tail data federated learning system based on knowledge distillation feature generation, characterized in that, Including the server side and various clients, The server receives the uploaded local model, label distribution for each class, and local feature prototype for each class. It then updates the global model based on the local model parameters. The generated model is used to generate features based on the input labels and random noise. The generated features need to be correctly classified. The generated model is updated based on the label distribution for each class and local feature prototype for each class. The updated generated model and global model are then sent to each client. The loss functions used to update the generative model based on the label distribution and local feature prototypes of each class include classification loss, sample conditioning loss, and contrast loss used to limit the distance between generated features and real features; Each client is used to receive the global model and the generative model, generate local features for each class based on the generative model and calculate the prototype of each local feature, update the local model based on each local feature and local data, and upload the updated local model, the label distribution for each class and the prototype of each local feature, initialize the personalized model based on the global model, generate multiple features for minority class samples based on the generative model, update the initialized personalized model based on the multiple features and local data, and transfer the knowledge of the global model to the personalized model based on knowledge distillation. Among them, generating multiple features for minority class samples and generating fewer features for majority class samples based on the generative model includes: sampling multiple labels for minority class samples and using the generative model to generate multiple features based on the multiple labels; sampling fewer labels for majority class samples and using the generative model to generate fewer features based on the fewer labels.

8. The long-tail data federated learning system based on knowledge distillation feature generation according to claim 7, characterized in that, On the server side, the loss function used to update the generation model is based on the label distribution and feature prototype of each class. for: in, , , Indicates the weighting coefficient. Represents classification loss, Indicates sample adjustment loss. Indicates comparative loss; Classification loss Represented as: in, y Indicates the input label, f Indicates the generated features, v k This represents the classifier in the local model of the k-th client. L CE (( v k , f ),y) means to f Enter to v k The predicted value and y The cross-entropy loss, where p(y) represents the label distribution. This means inputting y into the parameter. The feature distribution obtained from the generative model, where E represents the expectation. S T Represents a collection of clients; Sample adjustment loss Represented as: in, z Represents a random noise vector. i and j Represents a vector index. M Indicates the total number of vectors. Represents the L2 norm; Comparative loss Represented as: in, This represents the local feature prototype of the y-th class for the k-th client. Indicates that the sample number is 1 y A collection of clients of the class. This represents the prototype of the global feature of class y. Indicates the first Class tags, This represents the feature generated based on the label of the y-th class. A ( y ) represents a set of labels.

9. The long-tail data federated learning system based on knowledge distillation feature generation according to claim 7, characterized in that, In each client, the loss function used to update the local model based on each type of local feature and local data is as follows: in, Labels indicating random sampling This indicates the use of generative models based on labels. The generated features Indicates will Enter to The predicted value and Cross-entropy loss, t Indicates the round, r Indicates batch, Indicates the first k Local data of each client, Indicates the first k A local model Indicates that the basis will Enter to The balanced softmax loss is calculated from the predicted values; The personalized model is updated based on multiple features, few features, and local data. Simultaneously, knowledge distillation is used to transfer knowledge from the global model to the personalized model. The loss function employed is... for: in, Indicates weight, Represents classification loss, Indicates distillation loss; Classification loss Represented as: in, This represents multi-label sampling for minority class samples and few-label sampling for majority class samples. Indicates based on The generated features Indicates the first k Each personalized model contains a classifier. Indicates will Enter to The predicted value and Cross-entropy loss, Indicates the first k Local data of each client, Indicates the first k A personalized model, Indicates will Enter to The cross-entropy between the predicted value and the original label y; Distillation loss Represented as: in, Denotes KL divergence, Represents the global model. t Indicates the round, r Indicates batch, This represents the softmax function. Indicates will Input to global model The softmax value, Indicates will Input to personalized model The softmax value.

Images