Personalized federal element learning method for data isomerism

A learning method and heterogeneous technology, applied in neural learning methods, machine learning, database updates, etc., can solve problems such as reducing the overall performance of personalized models, avoid negative transfer problems, and promote collaborative training

Pending Publication Date: 2022-04-15
SOUTH CHINA UNIV OF TECH
View PDF0 Cites 9 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

In this case, if one only relies on the guidance of a global model, it can easily degrade the overall performance of the personalized model due to the negative transfer of generalization

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Personalized federal element learning method for data isomerism
  • Personalized federal element learning method for data isomerism
  • Personalized federal element learning method for data isomerism

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0067] like figure 1 The described personalized federated meta-learning method for data heterogeneity includes the following steps:

[0068] The first step is to determine the auto-encoder structure in the initialization phase of each client and the meta-model structure in the personalization phase;

[0069] Before each user's mobile device participates in federated learning, it first needs to download a unified auto-encoder and the model structure of the meta-model from the cloud server; the auto-encoder used in the initialization phase is a kind of neural network, which is generally used dimensionality reduction or feature learning, which is used here to represent the distribution of the user's local language data; the meta-model used in the personalization stage refers to a model under meta-learning, which can be trained by a small number of samples. A learning model adapted to new tasks. In order to predict the next word, a common language model is used here, such as LSTM...

Embodiment 2

[0110] like image 3 As shown, a total of 5 users participate in federated training, and they are divided into 2 groups in the cloud server. In any communication round, it is assumed that the grouping results of the previous round are {1, 2, 3} in the first group and {4, 5} in the second group. At this time, the cloud server selects users {1, 3, 5 } Participate in the federated training process, then users {1, 3 train each receive the meta model φ 1 , user {5} will receive the metamodel φ 2 .

[0111] For user 1, it is assumed that he has 100 local data, T=5 local updates are performed locally, and the batch size of random sampling for each update is , then after the local update is completed, the total sampled data size is min(2*5*5, 100)=50. Then calculate the distribution vector of this batch of sampled data together with the locally updated metamodel Upload to cloud server. Similarly, users {3, 5} also perform the above process.

[0112] For the cloud server, it ...

Embodiment 3

[0119] In an embodiment, the model structure of the auto-encoder may also be one of a convolutional auto-encoder and a cyclic auto-encoder,

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a personalized federated meta-learning method for data isomerism. The method comprises the following steps: determining an automatic encoder structure in an initialization stage of each client and a meta-model structure in a personalized stage; initializing parameters of a federation training stage; grouping the clients according to the local data distribution vectors uploaded by the clients; aggregating the client models in each group, and issuing the aggregated client models to the clients in the group to carry out the next round of iteration; and after federation training is finished, the client performs fine tuning on the meta-model in the group and local data thereof to generate a personalized model. According to the method, when the clients participate in federation training, the clients with approximate data distribution are dynamically divided into the same group according to the local data distribution vectors uploaded in each round, and the corresponding meta-model is set for each group, so that the problems of slow model convergence and low accuracy caused in a highly heterogeneous data environment are solved.

Description

technical field [0001] The invention relates to the research field of distributed machine learning under data heterogeneity, in particular to a personalized federated meta-learning method for data heterogeneity. Background technique [0002] The proliferation of edge devices in modern society, such as mobile phones and wearables, has led to the rapid growth of distributed private data generated by people. While this abundance of data presents enormous opportunities for machine learning applications, with regulations such as the General Data Protection Regulation (GDPR) and the Health Insurance Privacy and Accountability Act (HIPAA), there has been a surge in societal concerns about data privacy. The attention is getting higher and higher. This has made federated learning increasingly popular, a new distributed machine learning paradigm that enables the development and training of machine learning models on data silos in a cooperative and privacy-preserving manner. The main...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F16/27G06F16/23G06K9/62G06N3/04G06N3/08G06N20/00
Inventor 杨磊黄家明
Owner SOUTH CHINA UNIV OF TECH
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap