Convolutional neural network weight gradient optimization method based on data stream

Abstract

The invention discloses a convolutional neural network weight gradient optimization method based on a data stream, and provides a configurable data stream architecture design for convolutional neural network weight gradient optimization, so that convolution operations of different sizes in weight gradient calculation can be supported, and the degree of parallelism is K * K (convolution kernel size) times that of serial input. The training performance of the whole convolutional neural network is improved, and the problem that convolution operations of different sizes are difficult to realize in weight gradient calculation is solved. Compared with the prior art, the method has the advantages that (1) the acceleration effect is obvious: for weight gradient calculation, the degree of parallelism is improved by K * K compared with that of an original serial scheme, and the transmission time of data input is remarkably reduced, so that the purpose of accelerating the whole network training is achieved, and 1-1 / (K * K)% of input storage can be reduced compared with a general matrix multiplication scheme; and 2) the applicability and the universality are simultaneously met.

Description

technical field [0001] The invention belongs to the field of information technology and the field of convolutional neural network training hardware acceleration, and in particular relates to convolutional neural network training based on low power consumption and high performance. Background technique [0002] Convolutional Neural Network (CNN), as a feed-forward neural network, is widely used in various fields such as computer vision and natural language processing. With the increasing network scale and training data set of CNN, the training of CNN requires huge computing power, storage space and power consumption. The training of CNN includes the calculation of forward propagation and back propagation of the convolution module. At present, the academic and industrial circles have proposed many solutions for the hardware implementation of forward propagation, but there is a lack of hardware implementation for back propagation. Backpropagation includes the calculation of t...

AI Technical Summary

Problems solved by technology

This scheme has two disadvantages: one is that it needs to read K×K times repeatedly, and the other is that it takes time to read data that does not participate in the calculation when calculating the corresponding gradient.

This method consumes a lot of storage resources and transmission time

Images