A federated learning method and system based on heterogeneous and long-tail data
By adopting federated learning methods based on heterogeneous and long-tail data in e-shopping scenarios, the client conducts local training and calculates the real feature gradient, and aggregates and constructs virtual features on the server side, solving the data heterogeneity and long-tail distribution caused by data The problem of low accuracy achieves more efficient feature extraction and accuracy, and protects data privacy.
Patent Information
- Application Number
- CN202411104260.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-13
- Publication Date
- 2024-11-08
- Estimated Expiration
- 2044-08-13
AI Technical Summary
In e-shopping scenarios, the heterogeneity of data and long-tail distribution lead to low accuracy of federated learning. Traditional methods have poor performance when processing heterogeneous data, and long-tail data affects the learning effect and reduces the accuracy.
Using federated learning methods based on heterogeneous and long-tail data, the client conducts local training and calculates the real feature gradient, the server-side aggregation model and gradient, constructs virtual features, selects the optimal virtual features through the gradient matching loss supervision method, and retrains the global Model, improve feature extraction capabilities and protect data privacy.
It improves the personalized training effect of the model, improves the accuracy of federated learning, effectively solves data heterogeneity and long-tail problems, and protects data privacy.
Smart Images

Figure CN118644765B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to a federated learning method and system based on heterogeneous and long-tail data, belonging to the technical field of image recognition. Background Art
[0002] In recent years, with the rapid development of Internet technology, e-shopping has become quite popular. In the application scenario of e-shopping, in order to correctly identify customers' shopping preferences and push appropriate products to the corresponding customers, the first task is to effectively classify the electronic images of the products, and the classification of electronic images is closely related to the type and structure of the data. Specifically, in the field of e-shopping, clients can be different e-commerce platforms or individual consumers, who keep their private data in local electronic devices. Among these data, the pictures of the products often contain a lot of key information (such as product prices, product features, and product promotions), which is crucial for the online push and sale of products. Among them, there are a large number of pictures in the user's purchase records and browsing records, so it is necessary to apply efficient machine learning algorithms to efficiently classify these large numbers of electronic pictures and effectively push and sell products. In addition, with the rapid development of the consumer electronics industry in recent years, the amount of data of various online transactions has also grown rapidly, and has the characteristics of wide distribution, decentralized data storage, and difficult data privacy protection. In order to effectively solve these challenges and achieve effective image classification in consumer electronics, the depth and breadth of the application and development of federated learning in e-shopping are also increasing.
[0003] Since its proposal, federated learning has achieved many remarkable results, but it still faces some major challenges. Among them, data heterogeneity and long-tail distribution are two common and difficult problems. Specifically, in the e-shopping scenario, the images of different clients will show heterogeneity due to different personal shopping habits, resulting in different optimization directions and poor generalization performance of each client model. Traditional federated learning methods usually perform poorly when processing such heterogeneous data. In addition, some special or rare goods or categories may have a small sample size in e-shopping, which belongs to long-tail data. The existence of long-tail data leads to poor learning effect for a few categories of goods, further reducing the accuracy of existing federated learning. In the e-shopping scenario, the image data between different clients has both heterogeneity and long-tail characteristics, making the federated learning problem more complex and challenging. Summary of the invention
[0004] The purpose of the present invention is to overcome the deficiencies in the prior art and provide a federated learning method and system based on heterogeneous and long-tail data to solve the problem of low accuracy of federated learning of heterogeneous and long-tail data, thereby effectively protecting the privacy of data.
[0005] In order to solve the above technical problems, the present invention is implemented by adopting the following technical solutions:
[0006] In a first aspect, the present invention provides a federated learning method based on heterogeneous and long-tail data, including in each iterative training round:
[0007] The client receives the global model and retrained model from the server, as well as heterogeneous and long-tail datasets;
[0008] The client locally trains the global model based on the heterogeneous and long-tailed data set using a feature representation learning method to obtain a local model, and extracts the real features of the heterogeneous and long-tailed data set using the retrained model, and calculates the gradient of the real features;
[0009] The client sends the local model and the gradient of the real feature to the server;
[0010] The server aggregates the local models and the gradients of the real features from multiple clients respectively to obtain an aggregated global model and an aggregated feature gradient of class C, where class C represents any class in the heterogeneous and long-tail data sets of all clients;
[0011] The server constructs a virtual feature of class C, calculates a virtual feature gradient based on the virtual feature of class C, obtains the difference between the aggregate feature gradient of class C and the virtual feature gradient using a gradient matching loss supervision method, selects an optimal virtual feature gradient based on the difference between the feature gradients, and obtains the corresponding latest virtual feature based on the optimal virtual feature gradient;
[0012] The server side retrains the classifier in the aggregated global model using the latest virtual features to obtain an updated retrained model; the server side sends the aggregated global model and the updated retrained model to the client side to complete the training of the current iteration round;
[0013] When the preset iteration rounds are reached, the model trained by federated learning is obtained.
[0014] In combination with the first aspect, further, after the client receives the global model from the server, the global model is divided into a feature extractor and a classifier, the classifier is updated to the classifier of the local model, and the local model is trained.
[0015] In combination with the first aspect, further, calculating the gradient of the real feature includes:
[0016] The client uses the feature extractor to extract real features from the heterogeneous and long-tail data sets;
[0017] The gradient of the real feature is calculated from the classifier in the retrained model received from the server.
[0018] In combination with the first aspect, further, the calculation expression of the gradient of the real feature includes:
[0019] ;
[0020] in, represents the gradient of the true feature generated by client k for class C; represents the number of class c in client k; Represents a variable The gradient of Represents the loss between the predicted label and the true label; Represents the true feature gradient In the classifier The predicted label under ; Represents the true label of the i-th data; represents the real feature generated by client k for class C; i represents the i-th data.
[0021] In combination with the first aspect, further, the local models and the gradients of the real features from multiple clients are aggregated respectively to obtain a global model, and the calculation expression includes:
[0022] ;
[0023] in, Represents the global model generated by aggregating all local models; represents the set of clients selected in round t; represents the amount of data of client k; represents the updated local model of client k; k represents the client;
[0024] The expression for obtaining the aggregate feature gradient of class C includes:
[0025] ;
[0026] in, represents the aggregate feature gradient of class C; Represents a client collection containing class C.
[0027] In combination with the first aspect, further, the virtual feature gradient is calculated according to the virtual feature of the class C, and the expression is:
[0028] ;
[0029] in, represents the virtual feature gradient; n represents the number of virtual features for each class; Represents the loss of predicted labels and virtual labels; represents the virtual feature gradient Retraining the classifier The predicted label under ; represents the gradient of the virtual feature generated by client k for class C; Represents the true label of the i-th data.
[0030] In combination with the first aspect, further, selecting an optimal virtual feature gradient according to the difference between the feature gradients includes:
[0031] Based on the difference between the aggregated feature gradient of class C and the virtual feature gradient, the minimum loss function is constructed using gradient matching loss supervision;
[0032] Adjusting the model parameters by the minimum loss function minimizes the difference between the aggregated feature gradient and the virtual feature gradient of the class C;
[0033] The optimal virtual feature gradient is obtained based on the minimum difference in gradients.
[0034] In combination with the first aspect, further, the calculation expression of the minimum loss function includes:
[0035] ;
[0036] in, Representation based on model The minimum loss function of Indicates the use of global model weights and the model output for input x; represents the global model weight; x represents the original input data; y is the true label; represents the supervised contrast loss function; represents the model output for the enhanced input data x'; represents the input data after the first data enhancement; represents the model output for the enhanced input data x''; Represents the input data after the second data enhancement; represents the cross entropy loss function; Represents the prediction results of the model; represents the balance factor; r represents the number of rounds; represents the accuracy; C represents the classifier of the global model; represents the classifier of the local model of client k; represents a dynamically adjusted balancing factor that depends on the number of rounds r and the accuracy a; Represents the classifier C of the global model and the classifier of the local model Between distance.
[0037] In combination with the first aspect, further, the server uses the latest virtual features to retrain the classifier in the global model, and the expression is:
[0038] ;
[0039] in, represents the classifier in the retrained model after retraining; Represents the classifier in the global model; represents the learning rate; represents the loss calculated using the retrained classifier and dummy features; represents the i-th virtual feature; Represents the true label of the i-th data.
[0040] Secondly, a federated learning system based on heterogeneous and long-tail data includes a client and a server. The client includes an acquisition module, a local training module, and a sending module. The server includes an aggregation module, a virtual feature selection module, and a retraining module.
[0041] The acquisition module is used to receive the global model and the retrained model, as well as to obtain heterogeneous and long-tail datasets;
[0042] A local training module, for locally training the global model based on the heterogeneous and long-tailed data set using a feature representation learning method to obtain a local model, and for extracting the real features of the heterogeneous and long-tailed data set using the retrained model, and calculating the gradient of the real features;
[0043] A sending module, used for sending the local model and the gradient of the real feature from the server to the server;
[0044] An aggregation module, used to aggregate the gradients of the local models and the real features from multiple clients respectively, to obtain a global model and an aggregated feature gradient of class C, where class C represents any class in the heterogeneous and long-tail data sets of all clients;
[0045] A virtual feature selection module is used to construct a virtual feature of class C, calculate a virtual feature gradient according to the virtual feature of class C, obtain the difference between the aggregate feature gradient of class C and the virtual feature gradient using a gradient matching loss supervision method, select an optimal virtual feature gradient according to the difference between the feature gradients, and obtain the corresponding latest virtual feature according to the optimal virtual feature gradient;
[0046] The retraining module is used to retrain the classifier in the aggregated global model using the latest virtual features to obtain an updated retrained model, and send the aggregated global model and the updated retrained model to the client to complete the current iteration round of training.
[0047] Compared with the prior art, the present invention has the following beneficial effects:
[0048] The present invention uses a feature representation learning method on the client to improve the feature extraction capability of the local model. It effectively ensures the personalization of model training and solves the problem of data heterogeneity. Retraining the classifier in the global model using the virtual features trained on the server side can fully protect data privacy, and update the virtual features to virtual features similar to the real features, thereby solving the problem of long tail data and improving the accuracy of federated learning. BRIEF DESCRIPTION OF THE DRAWINGS
[0049] Figure 1 It is a system block diagram of a federated learning method based on heterogeneous and long-tail data provided by an embodiment of the present invention;
[0050] Figure 2 A system block diagram of local training of a local model provided by an embodiment of the present invention;
[0051] Figure 3 A system block diagram after the global model is updated according to an embodiment of the present invention;
[0052] Figure 4 A system block diagram of a classifier in a retrained global model provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0053] The technical solution of the present invention is described in detail below through the accompanying drawings and specific embodiments. It should be understood that the embodiments of the present application and the specific features in the embodiments are detailed descriptions of the technical solution of the present application, rather than limitations on the technical solution of the present application. In the absence of conflict, the embodiments of the present application and the technical features in the embodiments can be combined with each other.
[0054] The term "and / or" in this article is only a description of the association relationship of the associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.
[0055] Example 1
[0056] See also Figure 1 This embodiment introduces a federated learning method based on heterogeneous and long-tail data, which specifically includes the following steps:
[0057] Step S1, the client receives the global model and the retrained model from the server, and obtains heterogeneous and long-tail data sets;
[0058] It should be noted that heterogeneous and long-tail data are subsets of image datasets, and the image datasets may be natural images, medical images, satellite images, etc. In an embodiment of the present invention, image datasets are used to train models, and during the training process, these images usually require a large amount of annotations, thereby dividing the images into heterogeneous data and long-tail data.
[0059] In the application scenario of electronic shopping, in order to correctly identify customers' shopping preferences and push appropriate products to the corresponding customers, the electronic images of the products must first be effectively classified, and the classification of electronic images is closely related to the type and structure of the data. In order to ensure the privacy of one's own private data in local electronic devices and to perform efficient classification and effectively push and sell products, the federated learning method is used to solve the problems of data heterogeneity and long-tail distribution.
[0060] Specifically, in the e-shopping scenario, images from different clients may be heterogeneous due to different personal shopping habits, and some special or rare products or categories may have a smaller sample size in e-shopping, i.e., long-tail data. The accuracy of federated learning can be improved by solving the problems of heterogeneous and long-tail data.
[0061] Step S2: Based on heterogeneous and long-tail data sets, the client uses feature representation learning methods to perform local training on the global model to obtain a local model. , and using the retrained model Extract the true features of the heterogeneous and long-tailed datasets and calculate the gradient of the true features .
[0062] Specifically, the client receives the global model at the beginning of each training round Then, the global model Decoupled into feature extractor and classifier, during local training phase, the client retains the global model feature extractor and use the global model The classifier in replaces the original local model Then, the client uses the model with the updated classifier to perform local training.
[0063] In order to improve the feature extraction capability during local model training, feature representation learning methods are used to better extract the true features of heterogeneous and long-tail data sets.
[0064] Specifically, in order for the server to generate virtual features that approximate real features, it is necessary to calculate the gradient of the real features. First, the client extracts the real features in the heterogeneous and long-tailed datasets through the feature extractor, and then uses the classifier in the retrained model to calculate the gradient of the real features. , the calculation expression of the gradient of its true feature is as follows:
[0065] ;
[0066] in, represents the gradient of the true feature generated by client k for class C; represents the number of class c in client k; Represents a variable The gradient of Represents the loss between the predicted label and the true label; Represents the true feature gradient In the classifier The predicted label under ; Represents the true label of the i-th data; represents the real feature generated by client k for class C; i represents the i-th data.
[0067] Step S3: The client sends the local model and the gradient of the true feature Sent to the server.
[0068] Step S4: The server side processes the local models from multiple clients. and the gradient of the true feature Aggregate them separately to get the aggregated global model and the aggregate feature gradient of class C , where class C represents any class in the heterogeneous and long-tail datasets of all clients.
[0069] Specifically, select the preset number of clients and add multiple local models Perform aggregation to obtain the aggregated global model , the calculation expression of the aggregated global model is as follows:
[0070] ;
[0071] in, Represents the global model generated by aggregating all local models; represents the set of clients selected in round t; represents the amount of data of client k; represents the updated local model of client k; k represents the client;
[0072] The gradients of multiple real features Aggregate to obtain the aggregate feature gradient of class C , the calculation expression of the aggregate feature gradient of class C is as follows:
[0073] ;
[0074] in, represents the aggregate feature gradient of class C; Represents a client collection containing class C.
[0075] It should be noted that class C is the category or label of the data, which represents any category in the local heterogeneous and long-tail data set; for example, apples and oranges are two categories respectively. The aggregation here is to obtain the aggregated feature gradient of different clients for the same class C.
[0076] Step S5, see Figure 3 , the server builds the virtual features of class C , calculate the virtual feature gradient based on the virtual feature of class C , the gradient matching loss supervision method is used to obtain the aggregate feature gradient of class C and virtual feature gradient The difference between them is used to select an optimal virtual feature gradient according to the difference between the feature gradients, and the corresponding latest virtual feature is obtained according to the optimal virtual feature gradient. .
[0077] Specifically, an optimal virtual feature gradient is selected according to the difference between feature gradients, including:
[0078] Aggregated feature gradient based on class C and virtual feature gradient The difference between , and the gradient matching loss supervision method is used to construct the minimum loss function;
[0079] The model parameters are adjusted by the minimum loss function so that the aggregate feature gradient of class C and virtual feature gradient The difference is minimal;
[0080] According to the minimum difference of feature gradients, the optimal virtual feature gradient is obtained.
[0081] Furthermore, the calculation expression of the minimum loss function includes:
[0082] ;
[0083] in, Representation based on model The minimum loss function of Indicates the use of global model weights and the model output for input x; represents the global model weight; x represents the original input data; y is the true label; represents the supervised contrast loss function; represents the model output for the enhanced input data x'; represents the input data after the first data enhancement; represents the model output for the enhanced input data x''; Represents the input data after the second data enhancement; represents the cross entropy loss function; Represents the prediction results of the model; represents the balance factor; r represents the number of rounds; represents the accuracy; C represents the classifier of the global model; represents the classifier of the local model of client k; represents a dynamically adjusted balancing factor that depends on the number of rounds r and the accuracy a; Represents the classifier C of the global model and the classifier of the local model Between distance.
[0084] It should be noted that in order to ensure the effect of personalized model training, the balance factor is introduced , so that the local model and the global model remain consistent.
[0085] See also Figure 2 First, the original local data x is enhanced twice to obtain x' and x'', and then feature extraction is performed to obtain , and use this to calculate the supervised contrast loss function ; Then calculate the cross entropy loss function based on the data label and the predicted label ; Finally, the classifier C of the global model and the classifier of the local model are calculated Between distance , adding the three losses together can get the final minimum loss function.
[0086] On actual electronic application platforms, rare products or categories may have fewer samples in e-shopping, so there will be some long-tail data. In order to solve the data imbalance problem of long-tail data, the server uses the gradient of the transmitted real features to optimize virtual features similar to the real features, thereby simulating the effect of training the model with long-tail data without using long-tail data.
[0087] Furthermore, the virtual feature gradient is calculated through the virtual feature of class C, and its calculation expression is as follows:
[0088] ;
[0089] in, represents the virtual feature gradient; n represents the number of virtual features for each class; Represents the loss of predicted labels and virtual labels; represents the virtual feature gradient Retraining the classifier The predicted label under ; represents the gradient of the virtual feature generated by client k for class C; Represents the true label of the i-th data.
[0090] Step S6: The server uses the latest virtual features to retrain the classifier in the global model to obtain the latest retrained model; the server sends the aggregated global model and the latest retrained model to the client to complete the training of the current iteration round;
[0091] For details, see Figure 4 , the latest virtual features retrain the classifier in the aggregated global model , get the latest classifier , and the latest classifier Sent to the retrained model to replace the classifier in the previous iteration, so as to obtain the latest retrained model .
[0092] Furthermore, the calculation expression of the classifier in the global model retrained by the latest virtual feature is as follows:
[0093] ;
[0094] in, represents the classifier in the retrained model after retraining; Represents the classifier in the global model; represents the learning rate; represents the loss calculated using the retrained classifier and dummy features; represents the i-th virtual feature; Represents the true label of the i-th data.
[0095] As the number of iterations increases, the model accuracy becomes higher and higher, ensuring the consistency between the local model and the global model, which is conducive to the personalized effect of model training. When product data is input into the model, the products can be effectively classified, solving the problem of low accuracy of federated learning caused by data heterogeneity and long tail.
[0096] Step S7: When the preset iteration round is reached, the federated learning training model is obtained.
[0097] Example 2
[0098] A federated learning system based on heterogeneous and long-tail data includes a client and a server. The client includes an acquisition module, a local training module, and a sending module. The server includes an aggregation module, a virtual feature optimization module, and a retraining module.
[0099] The acquisition module is used to receive the global model and retrained model from the server, and to obtain heterogeneous and long-tail datasets;
[0100] A local training module, for locally training the global model based on the heterogeneous and long-tailed data set using a feature representation learning method to obtain a local model, and for extracting the real features of the heterogeneous and long-tailed data set using the retrained model, and calculating the gradient of the real features;
[0101] A sending module, used for sending the local model and the gradient of the real feature to a server side;
[0102] An aggregation module, used to aggregate the gradients of the local models and the real features from multiple clients respectively, to obtain a global model and an aggregated feature gradient of class C, where class C represents any class in the heterogeneous and long-tail data sets of all clients;
[0103] The server constructs a virtual feature of class C, calculates a virtual feature gradient based on the virtual feature of class C, obtains the difference between the aggregate feature gradient of class C and the virtual feature gradient using a gradient matching loss supervision method, selects an optimal virtual feature gradient based on the difference between the feature gradients, and obtains the corresponding latest virtual feature based on the optimal virtual feature gradient;
[0104] A retraining module is used to retrain the classifier in the global model using the latest virtual features to obtain a retrained model; the server sends the global model and the retrained model to the client to complete the training of the current iteration round;
[0105] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.
Claims
1. A federated learning method based on heterogeneous and long-tail data, It is characterized in that Including, in each iterative training round: The client receives the global model and the retrained model from the server, and obtains heterogeneous and long-tail datasets; wherein the heterogeneous and long-tail datasets are subsets of the image dataset; The client locally trains the global model based on the heterogeneous and long-tailed data set using a feature representation learning method to obtain a local model, and extracts the real features of the heterogeneous and long-tailed data set using the retrained model, and calculates the gradient of the real features; The client sends the local model and the gradient of the real feature to the server; The server aggregates the local models and the gradients of the real features from multiple clients respectively to obtain an aggregated global model and an aggregated feature gradient of class C, where class C represents any class in the heterogeneous and long-tail data sets of all clients; The server constructs a virtual feature of class C, calculates a virtual feature gradient based on the virtual feature of class C, obtains the difference between the aggregate feature gradient and the virtual feature gradient of class C using a gradient matching loss supervision method, selects an optimal virtual feature gradient based on the difference between the feature gradients, and obtains the corresponding latest virtual feature based on the optimal virtual feature gradient; wherein, based on the difference between the aggregate feature gradient and the virtual feature gradient of class C, a minimum loss function is constructed using gradient matching loss supervision; the model parameters are adjusted using the minimum loss function so that the difference between the aggregate feature gradient and the virtual feature gradient of class C is minimized; the optimal virtual feature gradient is obtained based on the minimum difference in gradients; the calculation expression of the minimum loss function includes: ; in, Representation based on model The minimum loss function of Indicates the use of global model weights and the model output for input x; represents the global model weight; x represents the original input data; y is the true label; represents the supervised contrast loss function; represents the model output for the enhanced input data x'; represents the input data after the first data enhancement; represents the model output for the enhanced input data x''; Represents the input data after the second data enhancement; represents the cross entropy loss function; Represents the prediction results of the model; represents the balance factor; r represents the number of rounds; represents the accuracy; C represents the classifier of the global model; represents the classifier of the local model of client k; represents a dynamically adjusted balancing factor that depends on the number of rounds r and the accuracy a; Represents the classifier C of the global model and the classifier of the local model Between distance; The server side retrains the classifier in the aggregated global model using the latest virtual features to obtain an updated retrained model; the server side sends the aggregated global model and the updated retrained model to the client side to complete the training of the current iteration round; When the preset iteration rounds are reached, the model trained by federated learning is obtained.
2. The federated learning method based on heterogeneous and long-tail data according to claim 1, It is characterized in that After receiving the global model from the server, the client divides the global model into a feature extractor and a classifier, updates the classifier to the classifier of the local model, and trains the local model.
3. The federated learning method based on heterogeneous and long-tail data according to claim 2, It is characterized in that The step of calculating the gradient of the real feature comprises: The client uses the feature extractor to extract real features from the heterogeneous and long-tail data sets; The gradient of the real feature is calculated from the classifier in the retrained model received from the server.
4. The federated learning method based on heterogeneous and long-tail data according to claim 3, It is characterized in that The calculation expression of the gradient of the real feature includes: ; in, represents the gradient of the true feature generated by client k for class C; represents the number of class c in client k; Represents a variable The gradient of Represents the loss between the predicted label and the true label; Represents the true feature gradient In the classifier The predicted label under ; Represents the true label of the i-th data; represents the real feature generated by client k for class C; i represents the i-th data.
5. The federated learning method based on heterogeneous and long-tail data according to claim 1, It is characterized in that Aggregate the local models and the gradients of the real features from multiple clients to obtain a global model, and calculate expressions, including: ; in, Represents the global model generated by aggregating all local models; represents the set of clients selected in round t; represents the amount of data of client k; represents the updated local model of client k; k represents the client; The expression for obtaining the aggregate feature gradient of class C includes: ; in, represents the aggregate feature gradient of class C; Represents a client set containing class C; represents the gradient of the true feature generated by client k for class C.
6. The federated learning method based on heterogeneous and long-tail data according to claim 1, It is characterized in that The virtual feature gradient is calculated according to the virtual feature of the class C, and the expression is: ; in, represents the virtual feature gradient; n represents the number of virtual features for each class; Represents the loss of predicted labels and virtual labels; represents the virtual feature gradient Retraining the classifier The predicted label under ; represents the gradient of the virtual feature generated by client k for class C; Represents the true label of the i-th data.
7. The federated learning method based on heterogeneous and long-tail data according to claim 1, It is characterized in that The server uses the latest virtual features to retrain the classifier in the global model, and the expression is: ; in, represents the classifier in the retrained model after retraining; represents the classifier in the global model; represents the learning rate; represents the loss calculated using the retrained classifier and dummy features; represents the i-th virtual feature; Represents the true label of the i-th data.
8. A federated learning system based on heterogeneous and long-tail data, It is characterized in that It includes client and server. The client includes acquisition module, local training module and sending module. The server includes aggregation module, virtual feature selection module and retraining module. An acquisition module, used for receiving a global model and a retrained model, and acquiring heterogeneous and long-tailed data sets; wherein the heterogeneous and long-tailed data sets are subsets of the image data set; A local training module, for locally training the global model based on the heterogeneous and long-tailed data set using a feature representation learning method to obtain a local model, and for extracting the real features of the heterogeneous and long-tailed data set using the retrained model, and calculating the gradient of the real features; A sending module, used for sending the local model and the gradient of the real feature from the server to the server; An aggregation module, used to aggregate the gradients of the local models and the real features from multiple clients respectively, to obtain a global model and an aggregated feature gradient of class C, where class C represents any class in the heterogeneous and long-tail data sets of all clients; A virtual feature selection module is used to construct a virtual feature of class C, calculate a virtual feature gradient according to the virtual feature of class C, use a gradient matching loss supervision method to obtain the difference between the aggregate feature gradient and the virtual feature gradient of class C, select an optimal virtual feature gradient according to the difference between the feature gradients, and obtain the corresponding latest virtual feature according to the optimal virtual feature gradient; the server side constructs the virtual feature of class C, calculates the virtual feature gradient according to the virtual feature of class C, uses a gradient matching loss supervision method to obtain the difference between the aggregate feature gradient and the virtual feature gradient of class C, selects an optimal virtual feature gradient according to the difference between the feature gradients, and obtains the corresponding latest virtual feature according to the optimal virtual feature gradient; wherein, based on the difference between the aggregate feature gradient and the virtual feature gradient of class C, a minimum loss function is constructed by using gradient matching loss supervision; the model parameters are adjusted by the minimum loss function so that the difference between the aggregate feature gradient and the virtual feature gradient of class C is minimized; the optimal virtual feature gradient is obtained according to the minimum difference in gradients; the calculation expression of the minimum loss function includes: ; in, Representation based on model The minimum loss function of Indicates the use of global model weights and the model output for input x; represents the global model weight; x represents the original input data; y is the true label; represents the supervised contrast loss function; represents the model output for the enhanced input data x'; represents the input data after the first data enhancement; represents the model output for the enhanced input data x''; Represents the input data after the second data enhancement; represents the cross entropy loss function; Represents the prediction results of the model; represents the balance factor; r represents the number of rounds; represents the accuracy; C represents the classifier of the global model; represents the classifier of the local model of client k; represents a dynamically adjusted balancing factor that depends on the number of rounds r and the accuracy a; Represents the classifier C of the global model and the classifier of the local model Between distance; The retraining module is used to retrain the classifier in the aggregated global model using the latest virtual features to obtain an updated retrained model, and send the aggregated global model and the updated retrained model to the client to complete the current iteration round of training.
