Unlock instant, AI-driven research and patent intelligence for your innovation.

Federal distillation-based federal learning model training method and system, and medium

A technology for learning models and training methods, applied in the field of educational big data and model processing

Pending Publication Date: 2022-03-08
ZHEJIANG NORMAL UNIVERSITY
View PDF0 Cites 8 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0003] The present invention aims at solving at least one of the technical problems existing in the existing federated learning technology for educational big data

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Federal distillation-based federal learning model training method and system, and medium
  • Federal distillation-based federal learning model training method and system, and medium
  • Federal distillation-based federal learning model training method and system, and medium

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0051] Embodiments of the present invention are described in detail below, examples of which are shown in the drawings, wherein the same or similar reference numerals designate the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by referring to the figures are exemplary only for explaining the present invention and should not be construed as limiting the present invention.

[0052] In the description of the present invention, several means more than one, and multiple means more than two. Greater than, less than, exceeding, etc. are understood as not including the original number, and above, below, within, etc. are understood as including the original number. If the description of the first and second is only for the purpose of distinguishing the technical features, it cannot be understood as indicating or implying the relative importance or implicitly indicating the number of the indicated technical features ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a federated learning model training method and system based on federated distillation, and a medium, which can be applied to the technical field of model processing. The method comprises the following steps: a client adopts a federated learning algorithm and a local private data set to train and update a local model, adopts a public data set and the updated local model to predict a first output vector of a local normalization layer, and uploads a local average output vector corresponding to the first output vector to a server; the server performs global consensus operation on all the local average output vectors, determines a global average output vector and issues the global average output vector to all the clients; and the client performs knowledge distillation on the public data set by adopting the global average output vector and the updated local model, and trains and calls back the local model on the local private data set. According to the method, the communication cost only depends on model output and is not increased along with the multiple of the model size, so that the communication stability is effectively improved, and the delay is reduced.

Description

technical field [0001] The invention relates to the technical field of educational big data and model processing, in particular to a training method, system and medium of a federated learning model based on federated distillation. Background technique [0002] With the rapid development of the Internet and artificial intelligence (AI), users use countless mobile devices all the time to generate a large amount of data. Over time, the data explodes and grows exponentially. For example, educational big data with huge capacity and redundant types. As we all know, the successful application of artificial intelligence technologies such as Machine Learning (ML) in scientific research and business decision-making is benefiting from the drive of big data. Data with high representational capabilities can help us build more complex and accurate ML models. However, data in daily life often exists in the form of data islands, which are difficult to aggregate and store. Data silos becom...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06N20/00
CPCG06N20/00
Inventor 朱佳马晓东黄昌勤
Owner ZHEJIANG NORMAL UNIVERSITY