A nested meta-learning method and system based on federated architecture

Abstract

The present invention proposes a nested meta-learning method and system based on federated architecture. The method includes: selecting m from N clients 1 clients, m 1 Clients train m based on their local data and global model parameters in the current state. 1 Each client's local model parameters, N and m 1 Both are positive integers and N≥m 1 ; The central server according to the received m 1 local model parameters of clients to update the global model parameters; select m from N clients 2 clients, m 2 Clients determine m based on their local data and updated global model parameters. 2 Each client's global subtask and calculate the parameter gradient generated by the global subtask through learning the objective function, m 2 It is a positive integer and N≥m 2 ; The central server according to the received m 2 The parameter gradients of the global subtasks of each client are used to adjust the updated global model parameters.

Description

technical field [0001] The invention belongs to the field of machine learning, and in particular relates to a nested meta-learning method and system based on federated architecture. Background technique [0002] Federated learning is a decentralized machine learning architecture designed to learn models from distributed mobile devices, which solves the problem that data centers do not always have access to large-scale training data. Furthermore, since the data in each mobile device is privacy-sensitive, federated learning also has significant privacy-preserving advantages over the centralized machine learning model training process. [0003] Small sample learning can quickly grasp new target concepts with only a small amount of data. It aims to explore how to use the knowledge and experience summed up from existing samples to solve new problems when the sample size of new target categories is very small. Methods. Meta-learning is one of the current mainstream solutions for...

AI Technical Summary

Problems solved by technology

Some researchers proposed a personalized federated average algorithm, that is, in the federated average algorithm, each user trains the meta-learning model locally, and then passes the parameters to the server for aggregation to promote the generalization ability of the model, but the performance of the global model and The aggregation speed needs to be improved; a similar research program also includes the federated meta-learning algorithm, that is, the meta-learning algorithm is used in the federated stochastic gradient descent architecture to obtain the local gradient of each user and update the model parameters, but as mentioned above, The federated stochastic gradient descent algorithm requires frequent communication between users and servers, resulting in excessive communication overhead for this scheme

Images