Unlock instant, AI-driven research and patent intelligence for your innovation.

Network parameter training method and system, server, client and storage medium

A technology of network parameters and training methods, applied in the field of deep learning, to achieve the effect of improving performance and enhancing generalization performance

Active Publication Date: 2021-11-09
MEGVII BEIJINGTECH CO LTD
View PDF4 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

However, this method of transfer learning has some shortcomings, and there is still room for improvement

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Network parameter training method and system, server, client and storage medium
  • Network parameter training method and system, server, client and storage medium
  • Network parameter training method and system, server, client and storage medium

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0030] In order to make the objects, technical solutions and advantages of the present invention more apparent, exemplary embodiments according to the present invention will be described in detail below with reference to the accompanying drawings. Apparently, the described embodiments are only some embodiments of the present invention, rather than all embodiments of the present invention, and it should be understood that the present invention is not limited by the exemplary embodiments described here.

[0031] The inventors found that the above transfer learning method has some shortcomings, that is, when the trained model parameters are transferred to the new network, the newly learned parameters will make the performance of the network on the original task worse, and it "forgets" the previous learned features. This leads to a greater possibility of overfitting when training for new tasks, making the performance of the network gradually deteriorate as the training progresses ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

Embodiments of the present invention provide a network parameter training method and system, a server, a client, and a storage medium. The method includes: for each neural network in the m neural networks, receiving the learning rate of the neural network and the gradient corresponding to each parameter of the neural network, wherein, m is an integer greater than or equal to 1; based on the m neural networks The learning rate and gradient, and the parameter values ​​corresponding to the m neural networks stored in the server, calculate new parameter values; update the parameter values ​​stored in the server with the new parameter values; and for the i-th neural network in the m A neural network that transmits the parameter values ​​corresponding to the i-th neural network stored in the server to the client running the i-th neural network for updating the parameters of the i-th neural network, where 1≤i≤m. Multi-task training can be performed and the generalization performance of the network model can be enhanced, so that the performance of the network model on various tasks can be improved.

Description

technical field [0001] The present invention relates to the field of deep learning, and more specifically relates to a network parameter training method and system, a server, a client and a storage medium. Background technique [0002] In the prior art, when building a neural network training model, a network structure is usually built and a loss function and optimization method are designed. After the network is initialized, the values ​​of each parameter in the neural network are updated based on the backpropagation algorithm, so that the loss function Reduce to achieve a more ideal value, so as to complete the learning of the network model. [0003] Considering that most of the data or tasks are related, people proposed the concept of transfer learning, which is to transfer the trained model parameters to the new network model to help the new model training, thereby speeding up and optimizing the learning efficiency of the new model, No need to learn from scratch like mo...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Patents(China)
IPC IPC(8): G06N3/08G06N3/04
CPCG06N3/08G06N3/045
Inventor 罗晶张祥雨
Owner MEGVII BEIJINGTECH CO LTD