Device and method for executing reverse training of artificial neural network

Abstract

The invention provides a device for executing reverse training of an artificial neural network. The device comprises an instruction buffer memory unit, a controller unit, a direct memory accessing unit, an H-tree module, a main operation module and a plurality of auxiliary operation modules. The device can realize reverse training of a multilayer artificial neural network. For each layer, weighted summation is performed on an input gradient vector and an output gradient vector of the layer is calculated. An input gradient vector of a next layer can be obtained through multiplying the output gradient vector by a derivative value of an excited function in forward operation. The gradient of the weight at this layer is obtained through multiplying the input gradient vector by an input neuron in forward operation. Then the weight of this layer can be updated according to the gradient of the weight at this layer.

Description

technical field [0001] The present invention generally relates to artificial neural networks, and in particular to a device and method for performing reverse training of artificial neural networks. Background technique [0002] Multi-layer artificial neural networks are widely used in the fields of pattern recognition, image processing, function approximation, and optimization calculations. In recent years, multi-layer artificial networks have been favored by academic circles and researchers due to their high recognition accuracy and good parallelism. industry is getting more and more attention. [0003] One known method to support inverse training of multi-layer artificial neural networks is to use a general-purpose processor. The method supports the above-mentioned algorithms by executing general-purpose instructions using general-purpose register files and general-purpose functional units. One of the disadvantages of this method is that the computing performance of a si...

AI Technical Summary

Problems solved by technology

Since the GPU is a device specially used to perform graphics and image calculations and scientific calculations, without special support for multi-layer artificial neural network operations, it still requires a lot of front-end decoding work to perform multi-layer artificial neural network operations, which brings a lot of problems. additional cost

In addition, the GPU only has a small on-chip cache, and the model data (weights) of the multi-layer artificial neural network need to be repeatedly moved from off-chip, and the off-chip bandwidth has become the main performance bottleneck.

In addition, the GPU has only a small on-chip cache, and the model data (weights) of the multi-layer artificial neural network need to be repeatedly moved from off-chip. The off-chip bandwidth has become the main performance bottleneck, and it has brought huge power consumption overhead.

Images