Device and method used for executing Adam gradient descent training algorithm

A gradient descent and training algorithm technology, applied in the application field of Adam algorithm, can solve the problems of insufficient computing performance of general-purpose processors, reduced memory access bandwidth, and high overhead of front-end decoding.

Active Publication Date: 2017-11-03
CAMBRICON TECH CO LTD
View PDF7 Cites 3 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0006] In view of this, the main purpose of the present invention is to provide a kind of device and method for carrying out Adam's gradient descending training algorithm, to solve the problem that

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Device and method used for executing Adam gradient descent training algorithm
  • Device and method used for executing Adam gradient descent training algorithm
  • Device and method used for executing Adam gradient descent training algorithm

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0048] Other aspects, advantages and salient features of the present invention will become apparent to those skilled in the art from the following detailed description of exemplary embodiments of the present invention according to the embodiments of the present invention in conjunction with the accompanying drawings.

[0049] In the present invention, the terms "include" and "comprising" and their derivatives mean to include but not limit; the term "or" is inclusive, meaning and / or.

[0050] In this specification, the various embodiments described below to describe the principles of the present invention are illustrative only and should not be construed as limiting the scope of the invention in any way. The following description with reference to the accompanying drawings is provided to assist in a comprehensive understanding of exemplary embodiments of the present invention as defined by the claims and their equivalents. The following description includes numerous specific de...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a device and a method used for executing an Adam gradient descent training algorithm. The device includes a direct memory access unit, an instruction cache unit, a controller unit, a data cache unit and a data processing module. The method includes: firstly, reading a gradient vector and a to-be-updated value vector, and initializing first moment and second moment vectors and corresponding exponential decay rates at the same time; during each iteration, utilizing the gradient vector to update the first moment and second moment vectors, respectively calculating a first biased moment estimation vector and a second biased moment estimation vector, and utilizing the first biased moment estimation vector and the second biased moment estimation vector to update to-be-updated parameters, and continuing training until the to-be-updated parameter vector converges. By utilizing the device and the method, application of the Adam gradient descent algorithm can be realized, and the efficiency of data processing can be greatly improved.

Description

technical field [0001] The present invention relates to the technical field of Adam algorithm application, in particular to a device and method for executing the Adam gradient descent training algorithm, and is a related application related to the hardware implementation of the Adam gradient descent optimization algorithm. Background technique [0002] The gradient descent optimization algorithm is widely used in the fields of function approximation, optimization calculation, pattern recognition and image processing. Adam algorithm is one of the gradient descent optimization algorithms. Features such as symmetric transformation invariance are widely used, and the use of special devices to implement the Adam algorithm can significantly increase its execution speed. [0003] Currently, one known way to implement the Adam gradient descent algorithm is to use a general-purpose processor. The method supports the above-mentioned algorithms by executing general-purpose instruction...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F9/30G06F9/38
CPCG06F9/3001G06F9/3814G06F9/3832G06F9/3887
Inventor 郭崎刘少礼陈天石陈云霁
Owner CAMBRICON TECH CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap