Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Continuous learning method for federated learning

A learning method and federated technology, applied in machine learning, instrument, character and pattern recognition, etc., can solve problems such as catastrophic forgetting in complex federated learning systems, achieve the effects of reducing catastrophic forgetting, improving performance, and enhancing learning effects

Pending Publication Date: 2021-11-16
ZHEJIANG UNIV
View PDF0 Cites 4 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0007] The purpose of the present invention is to address the deficiencies of the prior art, provide a continuous learning method for federated learning, and solve the problem of catastrophic forgetting that occurs in the process of learning a single task and sequentially learning multiple tasks in a complex federated learning system

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Continuous learning method for federated learning
  • Continuous learning method for federated learning
  • Continuous learning method for federated learning

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0033] The present invention will be further described below in conjunction with the specific embodiments of the accompanying drawings

[0034] Specific embodiments of the present invention and its implementation process are as follows:

[0035] Step 1: Federated learning must first determine the type of task to be learned. Here, the Chinese news headline classification task is taken as an example. It is assumed that each model learns three news classification tasks in sequence, and each task contains three different types of news data. . After the federated learning system determines the type of learning task, the server and each client collect several Chinese text data that match the task type. Here, it is assumed that the public Chinese social network data set NLPIR is selected as the auxiliary data set, which has been professionally processing, no privacy issues, the data set is randomly divided into four parts, which are used by three clients and one server respectively;...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a continuous learning method for federated learning. After determining a learning task, a federal learning server and each client independently construct an auxiliary data set for a subsequent training process by collecting a plurality of public data sets which are matched with a task type and have no privacy problem; a client side enables a local model to learn an auxiliary data set and an auxiliary label while learning a new task by means of knowledge distillation loss, so that forgetting of old knowledge is reduced; the server side enables the aggregation model to learn the auxiliary data set and the auxiliary label at the same time by means of knowledge distillation loss, so that forgetting of the model in the aggregation process is reduced. According to the invention, on the basis of privacy security and low communication cost, the continuous learning ability of the federal model is improved.

Description

technical field [0001] The invention relates to a continuous learning method in the field of artificial intelligence, in particular to a continuous learning method oriented to federated learning. Background technique [0002] With the increase in the number of mobile computing devices and sensors such as mobile phones and smart wearable devices, the data they generate is also increasing. The traditional training paradigm that collects and aggregates data for learning requires increasing computing and storage costs. . On the other hand, most edge devices collect highly sensitive private information of users. For the purpose of privacy protection, owners of devices and data are unwilling to share their own data. It is difficult to integrate data from all parties and there are privacy risks. Owners form "data islands" that are isolated from each other. The training mode of federated learning is proposed to solve the above-mentioned "data island" problem. [0003] Federated l...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G06N20/00G06K9/62G06F21/62
CPCG06N20/00G06F21/6245G06F18/24
Inventor 陈珂谢钟乐寿黎但江大伟马宇航伍赛
Owner ZHEJIANG UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com