Method and device for determining learning rate

A learning rate, learning and training technology, applied in the computing field, can solve the problem of long training time of deep learning models, and achieve the effect of avoiding training iteration time, increasing the number of intervals, and reducing the learning rate

Inactive Publication Date: 2019-07-19
HUAWEI TECH CO LTD
View PDF0 Cites 8 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

Since the corresponding learning rate needs to be calculated for each layer of neurons in each iterative training process, the training time of the deep learning model is longer

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Method and device for determining learning rate
  • Method and device for determining learning rate
  • Method and device for determining learning rate

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0052] The technical solution in this application will be described below with reference to the accompanying drawings.

[0053] In the field of AI, deep learning is a learning technique based on deep neural network algorithms. The deep learning model includes an input layer, a hidden layer, and an output layer, which use multiple nonlinear transformations to process data.

[0054] It should be understood that a neural network is a behavioral feature that imitates animal neural networks. This network relies on the complexity of the system to achieve the purpose of processing information by adjusting the interconnection relationship between a large number of internal nodes.

[0055] It should also be understood that a deep neural network (also called a deep learning model) can be understood as a neural network with multiple hidden layers, and the "multiple" here has no special metric standard. Theoretically speaking, a model with more parameters has a higher complexity and a gr...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention provides a method and a device for determining a learning rate. The method comprises the following steps: obtaining the number of intervals used for calculating the learning rate of a target layer in a deep learning model at intervals; and when the current iteration frequency is smaller than the interval frequency, in the process of carrying out learning training of current iterationon the target layer in the deep learning model, continuing to use the first learning rate in the previous iteration process to carry out learning training of the target layer. According to the technical scheme provided by the invention, the number of iterations for calculating the learning rate can be reduced, so that the long training iteration time caused by calculating the learning rate for the target layer in each iteration training in the prior art can be avoided.

Description

technical field [0001] The present application relates to the computing field, and more specifically, relates to a method, device, server and computer-readable storage medium for determining a learning rate. Background technique [0002] The deep learning model is a learning technology based on a deep neural network algorithm. The process of training the deep learning model is also the process of learning the parameter matrix, and its ultimate goal is to find a set of optimal values. The error between the predicted value output by the output layer of the deep learning model and the prior knowledge of the training data will affect the parameter matrix of each layer of neurons in the deep learning model (the parameter matrix of each layer of neurons includes the The weight corresponding to each neuron included in the layer neuron) is corrected and updated, so that after multiple iterations of training, the predicted value output by the output layer of the deep learning model i...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06N3/04G06N3/08
CPCG06N3/08G06N3/045G06N3/044
Inventor 周胜凯徐华周明耀
Owner HUAWEI TECH CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap