A personalized federated learning method, system, device and storage medium for long-tail data based on adversarial feature augmentation

Abstract

The present invention discloses a personalized federated learning method for long-tail data based on adversarial feature augmentation, comprising: dividing the entire global model into a global feature extractor g and a global classifier f on the server side and sending them to several clients; balancing the distribution of local sample data of several clients by random sampling, and constructing several pairs of source large category samples y using Bernoulli distribution. s and target small class sample y t Label pair; obtain the source category sample feature h s And the target small class sample features and obtain the sampling balanced sample feature set D bal And generate a balanced sample feature set using the sampled balanced sample feature set D bal The global classifier f is trained by generating a balanced sample feature set, and combined with the global feature extractor g to obtain a local personalized model. This can solve the data heterogeneity problem under the global long-tail data distribution.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence, and in particular to a method, system, device and storage medium for personalized federated learning of long-tail data based on adversarial feature augmentation. Background Art

[0002] In recent years, deep learning technology has played a vital role in the field of artificial intelligence. Its success relies heavily on large amounts of training data. When using deep learning models, it's common practice to collect large amounts of data on the server side for training. However, in real life, data silos exist between enterprises, and data is not interoperable. Furthermore, with increasing awareness of data security and privacy, centralized server-side modeling is becoming increasingly difficult. Federated learning was proposed to enhance the effectiveness of AI models by enabling cross-organizational joint modeling while ensuring data privacy and compliance. It is currently widely used in AI research.

[0003] Federated learning was proposed to enhance the effectiveness of AI models by enabling cross-organizational joint modeling while ensuring data privacy, security, and compliance. It has been widely used in AI research. A challenge in horizontal federated learning is the varying data distributions across different organizations (referred to as clients), a problem known as data heterogeneity. A global model aggregated on the server cannot guarantee good generalization across diverse client data. Therefore, personalized federated learning algorithms are tailored to the data distributions of different clients. When trained on client-side data, the global model ensures generalization while quickly adapting to the local data distribution. Furthermore, in real-world scenarios, the underlying global data distribution involved in federated training (assuming data from different organizations is aggregated) exhibits a long-tail distribution. This can lead to varying degrees of local data imbalance. When trained on the client side, the global model focuses solely on information from the majority class, which has a large number of samples, and fails to capture information from the minority class, which has a small number of samples. Therefore, data heterogeneity and long-tail distributions need to be addressed in federated learning.

[0004] Among existing personalized federated learning algorithms addressing data heterogeneity, the simplest approach, proposed by Wang et al., is to directly fine-tune the global model on client-side data. However, due to insufficient client-side training data, model overfitting is often a problem. To obtain robust local personalized models, some personalized federated learning algorithms adopt a meta-learning training paradigm. Per-FedAvg, proposed by Flanagan et al., uses meta-learning to train a rapidly adaptable global model that generalizes well across diverse data distributions. Other methods use regularization to constrain the differences between the global and local models, designing unique personalized optimization objectives to obtain robust local models. FedProx, proposed by Li et al., introduces a proximal term to control the distance from the initial global model when updating the local model. pFedMe, proposed by T. Dinh et al., uses an L2-norm regularized loss function for each client. This approach allows clients to update their local models in different directions without deviating from the global model reference point. Another method uses clustering to achieve local personalization. FedAMP proposed by Huang et al. weights different local models on the server side and sums them to obtain multiple different global models. Each client selects the global model that is closest to its own data distribution for local personalized training.

[0005] However, the above methods only consider the problem of heterogeneous data under balanced data distribution in federated learning. In real-world applications, the global data distribution often exhibits a long-tail distribution. In this case, because the above personalized federated learning algorithms fail to account for local data imbalance, the model is only able to capture information from classes with a large number of samples and cannot effectively learn information from classes with a small number of samples. This significantly degrades the performance of the entire personalized model. Numerous methods have been proposed to address data imbalance, which can be broadly categorized into three categories: class balancing, information augmentation, and two-stage training. Class balancing methods aim to improve the generalization ability of classes with fewer samples by balancing the training dataset or applying different weights during training. The most straightforward resampling method is the Class-aware Sampling method proposed by Shen, L., et al., in which samples from each class are selected with equal probability in each mini-batch. The class-balanced loss function (CB Loss) proposed by Cui, Y., et al. is also based on a class-weighted loss that calculates the effective number of classes per class using a simple formula. Regarding information augmentation, Chu, P. et al. proposed OFA, which uses class activation maps to divide features into class-specific features and class-shared features. This method expands the small-class information training model by fusing the rich class-shared features of large classes with the class-shared features of small classes. Zang, Y. et al. proposed FASA, which uses Gaussian distribution to fuse features to generate a small-class feature training model. Regarding two-stage training, Kang, B. et al. proposed decoupling the deep neural network learning process into representation learning and classification. They verified that feature extractors learned on imbalanced datasets still have good performance, and that strong minority recognition capabilities can be achieved simply by fine-tuning the classifier.

[0006] However, none of the aforementioned methods that combine data heterogeneity and long-tail data distribution consider both issues simultaneously. To address the long-tail data distribution problem in federated learning, Duan et al. proposed Astraea and Wang et al. proposed Ratio loss, both of which consider data heterogeneity and long-tail data distribution. However, their optimization goals are to obtain a high-performance global model rather than a robust, client-side, personalized model. Summary of the Invention

[0007] In view of this, the purpose of the present invention is to propose a personalized federated learning method for long-tail data based on adversarial feature augmentation, which can solve the above problems.

[0008] One aspect of the present invention provides a personalized federated learning method for long-tail data based on adversarial feature augmentation, comprising:

[0009] On the server side, the entire global model is divided into a global feature extractor g and a global classifier f and sent to several clients;

[0010] By balancing the distribution of local sample data of several clients through random sampling, several pairs of source category samples y are constructed using Bernoulli distribution s and target small class sample y t Tag pairs;

[0011] Get the source category sample feature h s and target small class sample characteristics And get the sampling balanced sample feature set D bal and generate a balanced sample feature set

[0012] Using the sampling balanced sample feature set D bal and generate a balanced sample feature set The global classifier f is trained and combined with the global feature extractor g to obtain a local personalized model.

[0013] In which, the global feature extractor g is parameterized as u, and the global classifier f is parameterized as v; the feature of sample x is generated by the global feature extractor g, that is, h=g(x;u); the prediction result of sample x is given by the global classifier f, that is, f(h;v).

[0014] Among them, the source category sample y s is the number of samples before sampling n s More classes, the target small class sample y t is the number of samples before sampling n t For a smaller class, the total number of training samples owned by the kth client is n k , the calculation formula of the Bernoulli distribution is as follows:

[0015]

[0016] Wherein, the feature h of the source category sample is obtained s and target small class sample characteristics The steps are as follows:

[0017] Use the global feature extractor g to obtain the source large category sample y s The source category sample feature h s ;

[0018] Use adversarial samples to obtain the characteristics of the target small class samples

[0019] Among them, the target small class sample features are obtained by using the adversarial sample The specific steps include:

[0020] The source class sample feature h is continuously optimized by the gradient descent algorithm s , and get the gradient during back propagation gradient The calculation formula is as follows:

[0021]

[0022] The gradient Regularization obtains the final characteristic perturbation δ, and the calculation formula of characteristic perturbation δ is as follows:

[0023]

[0024] Add feature perturbation δ to the source class sample feature h s On the above, the target small class sample features are obtained The target subclass sample characteristics The calculation formula is as follows:

[0025]

[0026] For the target small class sample characteristics Optimize, the specific optimization formula is as follows:

[0027]

[0028] Among them, the sampling balanced sample feature set D bal is the randomly sampled source category sample feature h d The generated balanced sample feature set Target small class sample features generated for adversarial A collection of .

[0029] The specific steps of obtaining the local personalized model include:

[0030] Design local classifier v for client k k In generating a balanced sample feature set The loss function on Loss Function The calculation formula is as follows:

[0031]

[0032] Design local classifier v for client k k In the sampling balanced sample feature set D bal The loss function on Loss Function The calculation formula is as follows:

[0033]

[0034] Adjust the loss function using the hyperparameter λ and Get the final loss function Final loss function The calculation formula is as follows:

[0035]

[0036] Another aspect of the present invention provides a personalized federated learning system for long-tail data based on adversarial feature augmentation, comprising:

[0037] The segmentation module is used to split the entire global model into a global feature extractor g and a global classifier f on the server side and send them to several clients;

[0038] The construction module is used to balance the distribution of local sample data of several clients through random sampling, and construct several pairs of source large category samples y using Bernoulli distribution s and target small class sample y t Tag pairs;

[0039] Acquisition module, used to obtain the source category sample feature h s and target small class sample characteristics And get the sampling balanced sample feature set D bal and generate a balanced sample feature set

[0040] A training module is used to utilize the sampling balanced sample feature set D bal and generate a balanced sample feature set The global classifier f is trained and combined with the global feature extractor g to obtain a local personalized model.

[0041] Another aspect of the present invention provides a personalized federated learning device for long-tail data based on adversarial feature augmentation, comprising a memory and a processor:

[0042] The memory is used to store computer programs;

[0043] The processor is configured to perform the above method when executing the computer program.

[0044] Another aspect of the present invention provides a storage medium for personalized federated learning of long-tail data based on adversarial feature augmentation, comprising: a computer program is stored on the storage medium, and when the computer program is executed by a processor, the method as described above is performed.

[0045] Beneficial effects of the present invention:

[0046] 1. The present invention divides the global model into a global feature extractor and a global classifier. Such a global feature extractor is less affected by data distribution, avoiding the impact of data heterogeneity under global long-tail data distribution in horizontal federated learning.

[0047] 2. The present invention uses Bernoulli distribution to migrate the source large category sample information to the target small category sample, uses the source large category sample information to supplement the target small category sample information, enriches the target small category sample information, and avoids the impact of data heterogeneity under the global long-tail data distribution in horizontal federated learning.

[0048] 3. In the present invention, the client uses a global feature extractor to obtain the source large category features, avoiding the overfitting of the extracted features caused by the noise of the local data distribution brought by the local feature extractor. After obtaining the source large category features, the idea of ​​adversarial samples is used to generate target small category features to balance the local feature set, and adversarial samples are used to generate sample supplementary information, which solves the problem of data heterogeneity caused by the global long-tail data distribution in horizontal federated learning.

[0049] 4. The present invention designs separate loss functions for the sampling balanced sample feature set and the generation balanced sample feature set, introduces hyperparameters to control the training direction of the local model, and can improve the performance of the local model of each client under personalized federated learning. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0051] Figure 1 is an overall flow chart of an embodiment of the present invention;

[0052] Figure 2 is a system module diagram in an embodiment of the present invention;

[0053] Figure 3 is a schematic diagram of a device in an embodiment of the present invention; DETAILED DESCRIPTION

[0054] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the invention for which protection is claimed, but merely represents selected embodiments of the present invention.

[0055] In the description of the present invention, the terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the number of the technical features indicated. Therefore, a feature specified as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, "plurality" means two or more, unless otherwise specifically defined.

[0056] like Figure 1 As shown, an embodiment of the present invention provides a personalized federated learning method for long-tail data based on adversarial feature augmentation, including:

[0057] S1 prepares the data set, builds the federated learning framework, and initializes the learning network;

[0058] S101 prepares several long-tail classification datasets and divides the training datasets into several clients;

[0059] In this example, the three long-tail classification datasets used are all image classification datasets: Fashion-MNIST, CIFAR-10, and CIFAR-100. Specifically, Fashion-MNIST is a dataset of fashion apparel thumbnails, consisting of 60,000 training images and 10,000 test images, each with a 28x28 pixel grayscale image. CIFAR-10 is a small dataset for object recognition, containing RGB color data images from 10 categories, each with a size of 32x32 pixels, including 50,000 training images and 10,000 test images. CIFAR-100 is a large dataset for object recognition, containing RGB color data images from 100 categories, each with a size of 32x32 pixels, including 50,000 training images and 10,000 test images. All long-tail classification datasets have an imbalance factor of 100, which is the value of the category with the largest global sample size divided by the category with the smallest global sample size. A Dirichlet distribution with a hyperparameter α is used to control the degree of client data heterogeneity. The closer α is to 0, the more severe the data heterogeneity. In this example, two data heterogeneity levels of α = 0.5 and α = 0.2 are simulated. The data from several long-tail classification datasets is evenly divided among several clients, with each client receiving the same amount of data.

[0060] S102 builds a federated learning framework and initializes the deep learning network;

[0061] In this embodiment, a three-layer CNN model is used for training on the Fashion-MNIST dataset, and the Resnet32 architecture is used as the backbone model for the CIFAR-10 dataset and the CIFAR-100 dataset. For the CNN model, the features after the second convolutional layer are selected for feature augmentation; for Resnet32, the features after the second block are selected for feature augmentation. All experiments are implemented using the PyTorch framework and run on two NVIDIA GeForce RTX 3080 GPUs. A total of 20 client data distributions are simulated and designed, and 10 client models are randomly selected each time for federated aggregation. The batch size is set to 128, the learning rate is set to 0.005, the optimizer is SGD, the number of global training rounds is set to 500, and the number of local update rounds is set to 1.

[0062] S2 divides the entire global model into a global feature extractor g and a global classifier f on the server side and sends them to several clients;

[0063] In this embodiment, based on the research of existing federated learning and long-tail learning methods, the feature extractor of the global model is less affected by the data distribution in the case of data heterogeneity and long-tail data distribution. Therefore, the global model is divided into two parts. The first part is the global feature extractor g, which is parameterized as u. The second part is the global classifier f, which is parameterized as v. At this time, the features of sample x are generated by the global feature extractor g, that is, generated by h=g(x;u); the prediction result of sample x is given by the global classifier f, that is, given by f(h;v). The global feature extractor and the global classifier are distributed to each selected client, where the parameters of the global feature extractor remain fixed, and each client is assigned a global feature extractor g and a global classifier f.

[0064] S3 balances the distribution of local sample data of several clients through random sampling, and uses Bernoulli distribution to construct several pairs of source large category samples y s and target small class sample y t Tag pairs;

[0065] In this embodiment, for the imbalanced distribution of sample data in several clients, the local sample data set is balanced by random sampling. For any two different categories, the number of samples before sampling n is defined as s The majority of classes are source large class samples y s , the number of samples before sampling n t The smaller class is the target small class sample y t Because the source large class sample y s The information of the source class sample y is richer, so it is necessary to s Information transfer to target small class sample y t In the present invention, information transfer is achieved through Bernoulli distribution. t , after selecting the corresponding source category sample y s When follows the Bernoulli distribution:

[0066]

[0067] Among them, n k is the total number of training samples owned by client k.

[0068] S4 obtains the source category sample feature h s and target small class sample characteristics And get the sampling balanced sample feature set D bal and generate a balanced sample feature set

[0069] Wherein, the feature h of the source category sample is obtained s and target small class sample characteristics The steps are as follows:

[0070] S401 uses the global feature extractor g to obtain the source large category sample y s The source category sample feature h s ;

[0071] In this embodiment, there are two reasons for using the global feature extractor g instead of the local feature extractor for local feature augmentation: first, the global feature extractor g is aggregated from each local feature extractor and contains more data information for information transfer; second, the local feature extractor carries the noise of the local data distribution, and the features generated by the local feature extractor are more prone to overfitting. Therefore, the selected sample label pair (y s ,y t ), the source class sample features are obtained through the global feature extractor g: h s =g(x s ;u), where (x s ,y s ) are sample features from the randomly selected source categories.

[0072] S402 uses the adversarial sample to obtain the target small class sample features

[0073] Among them, the target small class sample features are obtained by using the adversarial sample The specific steps include:

[0074] S4021 continuously optimizes the source large category sample feature h through the gradient descent algorithm s , and get the gradient during back propagation

[0075] In this embodiment, the idea of ​​generating adversarial samples in adversarial learning is used to continuously give the source large class sample features h s Add specific disturbances so that the model thinks that the source class sample features h with specific disturbances are added s is the target small class sample feature. Therefore, the source large class sample feature h s The label is marked as the target small class sample y t , continuously optimize the source large category sample feature h through the gradient descent algorithm s During back propagation, the gradient obtained is as follows

[0076]

[0077] S4022 will be gradient Regularization obtains the final feature perturbation δ;

[0078] In this embodiment, it can be regarded as the source large category sample feature h s Converted into the importance of the target small class sample features. Further regularize the gradient to obtain the final feature perturbation δ:

[0079]

[0080] S4023 adds the feature perturbation δ to the source large class sample feature h s On the above, the target small class sample features are obtained The target subclass sample characteristics The calculation formula is as follows:

[0081]

[0082] S4024: Characteristics of the target small class samples Optimize.

[0083] In this embodiment, in order to ensure the generation of target small class sample features The model can predict the target small class sample features with a high success rate, and only retain the predicted results For the sample features of the generated target subclass that exceed a certain threshold, in the present invention, the sample features of the generated target subclass whose prediction results exceed 0.5 are selected, and the sample features of the generated target subclass that do not exceed 0.5 are directly abandoned. The optimization objectives of the sample features of the generated target subclass are as follows:

[0084]

[0085] In this embodiment, after completing the entire adversarial sample generation operation, two feature sets can be obtained: one is the source class sample feature h obtained by random sampling s The sampling balanced sample feature set D bal , the other is the target small class sample feature generated by the adversarial sample Generate a balanced sample feature set

[0086] S5 uses the sampling balanced sample feature set D bal and generate a balanced sample feature set The global classifier f is trained and combined with the global feature extractor g to obtain a local personalized model.

[0087] The specific steps of obtaining the local personalized model include:

[0088] S501 designs the local classifier v of client k k In generating a balanced sample feature set The loss function on

[0089] In this embodiment, a balanced sample feature set is generated by Design local classifier v for client k k In generating a balanced feature set The loss function on

[0090]

[0091] Through this loss function Continuously optimize the local classifier v k For local sample categories with a small number of samples, the feature information of the local large category samples is transferred to the local small category samples, solving the problem of local classifier v k The overfitting problem is solved by improving the performance of the local model for local small class samples.

[0092] S502 designs the local classifier v of client k k In the sampling balanced sample feature set D bal The loss function on

[0093] In this embodiment, considering that the source large class sample features with added disturbances are used to generate the target small class sample features, the source large class sample features h before the disturbance are not added s It may be misclassified as a target small class sample feature, so in the original sampling balanced sample data set D bal Another loss function is set on

[0094]

[0095] S503 uses hyperparameter λ to adjust the loss function and Get the final loss function

[0096] In this implementation, combined with the above loss function and The final loss function is:

[0097]

[0098] Here, λ is a hyperparameter that acts as a balancing factor, adjusting the strength of training the original sample feature set and the generated sample feature set. By adversarially augmenting small class features to balance the data distribution of each client, a new loss function is proposed to optimize the local model, fully migrating the rich feature information of local large class samples to local small class samples. Ultimately, the classification accuracy of all local classes is improved. This effectively solves the problem of unbalanced data distribution in federated heterogeneity and further improves the performance of each client's local model in horizontal federated learning.

[0099] Table 1 compares the accuracy (%) of the proposed method with several other federated learning methods on the Fashion-MNIST, CIFAR-10, and CIFAR-100 datasets, with an imbalance level of 100 and data heterogeneity levels of 0.5 and 0.2, respectively. The bolded results in the table represent the optimal results for each metric.

[0100] Table 1

[0101]

[0102] In Table 1:

[0103] FedAvg-FT corresponds to the method proposed by Wang, K et al. (Federated evaluation of on-device personalization.arXiv(2019));

[0104] FedProx corresponds to the method proposed by Li, T. et al. (Federated optimization inheterogeneous networks.arXiv.);

[0105] LG-FedAvg corresponds to the method proposed by Hanzely, F. et al. (Federated learning of a mixture of global and local models.arXiv(2020));

[0106] Per-FedAvg corresponds to the method proposed by Fallah, A. et al. (Personalized federated learning with theoretical guarantees: A model-agnostic meta-learning approach. In: Advances in Neural Information Processing Systems. pp. 3557{3568(2020)});

[0107] pFedMe corresponds to the method proposed by T. Dinh et al. (Personalized federated learning with moreauenvelopes. In: Advances in Neural Information Processing Systems. pp. 21394-21405 (2020));

[0108] Ditto corresponds to the proposal by Li, T. et al. (Fair and robust federated learning through personalization. In: International Conference on Machine Learning. pp. 6357-6368 (2021));

[0109] FedBN corresponds to the one proposed by Li, X. et al. (Federated learning on non-iid features via local batch normalization. In International Conference on Learning Representations, 1–27);

[0110] FedAvg+M2m corresponds to the method proposed by Kim, J. et al. (M2m: Imbalanced classification via major-to-minor translation. In IEEE / CVF Conference on Computer Vision and Pattern Recognition, 13896–13905);

[0111] FedAvg+cRT corresponds to the one proposed by Kang, B. et al. (Decoupling representation and classifier for long-tailed recognition. In International Conference on Learning Representations, 1–16);

[0112] like Figure 2 As shown, the present invention also provides a long-tail data personalized federated learning system based on adversarial feature augmentation, comprising:

[0113] A segmentation module 601 is used to divide the entire global model into a global feature extractor g and a global classifier f on the server side and send them to several clients;

[0114] Construction module 602 is used to balance the distribution of local sample data of several clients through random sampling, and construct several pairs of source large category samples y using Bernoulli distribution s and target small class sample y t Tag pairs;

[0115] Acquisition module 603, used to obtain the source category sample feature h s and target small class sample characteristics And get the sampling balanced sample feature set D bal and generate a balanced sample feature set

[0116] Training module 604, used to use the sampling balanced sample feature set D bal and generate a balanced sample feature set The global classifier f is trained and combined with the global feature extractor g to obtain a local personalized model.

[0117] like Figure 3 As shown, the present invention also provides a long-tail data personalized federated learning device based on adversarial feature augmentation, including: a memory 701 and a processor 702;

[0118] The memory 701 is used to store computer programs;

[0119] The processor 702 is configured to implement the above method when executing the computer program.

[0120] The present invention also provides a long-tail data personalized federated learning storage medium based on adversarial feature augmentation, comprising:

[0121] The storage medium stores a computer program, and when the computer program is executed by the processor, the above method is implemented.

[0122] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, platforms, or computer program products. Thus, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0123] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0124] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0125] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0126] The above descriptions are only some embodiments of the present invention and do not limit the scope of protection of the present invention. Any equivalent device or equivalent process transformation made according to the contents of the present invention description and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A personalized federated learning method for long-tail data based on adversarial feature augmentation, characterized by: include: Prepare the data set, build the federated learning framework, and initialize the learning network; Prepare several long-tail classification datasets and divide the training datasets into several clients. The long-tail classification datasets used are all image classification datasets. Build a federated learning framework and initialize the deep learning network; On the server side, the entire global model is divided into a global feature extractor g and a global classifier f and sent to several clients; By balancing the distribution of local sample data of several clients through random sampling, several pairs of source category samples y are constructed using Bernoulli distribution s and target small class sample y t Tag pairs; Get the source category sample feature h s and target small class sample characteristics And get the sampling balanced sample feature set D bal and generate a balanced sample feature set Using the sampling balanced sample feature set D bal and generate a balanced sample feature set Training the global classifier f and combining it with the global feature extractor g to obtain a local personalized model; The source category sample feature h is obtained s and target small class sample characteristics The steps are as follows: Use the global feature extractor g to obtain the source large category sample y s The source category sample feature h s ; Use adversarial samples to obtain the characteristics of the target small class samples The method uses the adversarial sample to obtain the target small class sample feature The specific steps include: The source class sample feature h is continuously optimized by the gradient descent algorithm s , and get the gradient during back propagation gradient The calculation formula is as follows: The gradient Regularization obtains the final characteristic perturbation δ, and the calculation formula of characteristic perturbation δ is as follows: Add feature perturbation δ to the source class sample feature h s On the above, the target small class sample features are obtained The target subclass sample characteristics The calculation formula is as follows: For the target small class sample characteristics Optimize, the specific optimization formula is as follows:

2. The personalized federated learning method for long-tail data based on adversarial feature augmentation as claimed in claim 1, characterized in that: The global feature extractor g is parameterized as u, and the global classifier f is parameterized as v; the feature of sample x is generated by the global feature extractor g, that is, h=g(x;u); the prediction result of sample x is given by the global classifier f, that is, f(h;v).

3. The personalized federated learning method for long-tail data based on adversarial feature augmentation as claimed in claim 1, characterized in that: The source category sample y s is the number of samples before sampling n s More classes, the target small class sample y t is the number of samples before sampling n t For a smaller class, the total number of training samples owned by the kth client is n k , the calculation formula of the Bernoulli distribution is as follows:

4. The personalized federated learning method for long-tail data based on adversarial feature augmentation according to claim 1, characterized in that: The sampling balanced sample feature set D bal is the randomly sampled source category sample feature h s The generated balanced sample feature set Target small class sample features generated for adversarial A collection of .

5. The personalized federated learning method for long-tail data based on adversarial feature augmentation according to claim 1, characterized in that: The specific steps of obtaining the local personalized model include: Design local classifier v for client k k In generating a balanced sample feature set The loss function on Loss Function The calculation formula is as follows: Design local classifier v for client k k In the sampling balanced sample feature set D bal The loss function on Loss Function The calculation formula is as follows: Adjust the loss function using the hyperparameter λ and Get the final loss function Final loss function The calculation formula is as follows:

6. A personalized federated learning system for long-tail data based on adversarial feature augmentation, characterized by: The long-tail data personalized federated learning method based on adversarial feature augmentation as described in any one of claims 1 to 5 comprises: The segmentation module is used to split the entire global model into a global feature extractor g and a global classifier f on the server side and send them to several clients; The construction module is used to balance the distribution of local sample data of several clients through random sampling, and construct several pairs of source large category samples y using Bernoulli distribution s and target small class sample y t Tag pairs; Acquisition module, used to obtain the source category sample feature h s and target small class sample characteristics And get the sampling balanced sample feature set D bal and generate a balanced sample feature set A training module is used to utilize the sampling balanced sample feature set D bal and generate a balanced sample feature set The global classifier f is trained and combined with the global feature extractor g to obtain a local personalized model.

7. A personalized federated learning device for long-tail data based on adversarial feature augmentation, characterized in that: including memory and processor; The memory is used to store computer programs; The processor is configured to implement the long-tail data personalized federated learning method based on adversarial feature augmentation as described in any one of claims 1 to 5 when executing the computer program.

8. A personalized federated learning storage medium for long-tail data based on adversarial feature augmentation, characterized in that: The storage medium stores a computer program, and when the computer program is executed by the processor, the long-tail data personalized federated learning method based on adversarial feature augmentation as described in any one of claims 1 to 5 is implemented.

Images