Softmax function hardware implementation method, module, chip and system

Abstract

The invention discloses a hardware implementation method, module, chip and system of a softmax function, and belongs to the technical field of hardware implementation of a neural network multi-task classification activation function. Based on the problems that in the prior art, e index operation and division operation are difficult to achieve in hardware of a softmax activation function, and the hardware implementation performance is not high, the softmax function is transformed; wherein a main controller module, a data input module, a calculation module and a data output module are constructed in a hardware system; in the calculation module, methods of comparison, subtraction, data segmentation, shift summation, mirror image search and the like are used to realize an E-to-2softmax function. According to the method, firstly, the feasibility of the E-to-2softmax function is theoretically analyzed, and then the hardware implementation part of the function is introduced in detail. Thus, e index operation and division operation of the softmax function are avoided in hardware implementation, so that hardware resources are saved, hardware friendliness is high, and the method has the advantages of being high in performance, low in complexity and the like.

Description

technical field [0001] The invention relates to the technical field of hardware implementation of neural network multi-task classification activation functions, and more specifically, relates to a hardware implementation method, module, chip and system of a softmax function. Background technique [0002] As a subset of machine learning, deep learning effectively extracts and transforms features in data by using multi-layer mathematical functions. One of the research hotspots of deep learning algorithms is the activation function. The activation function introduces nonlinear factors to neurons, so that the neural network can arbitrarily approximate any nonlinear function, so that the neural network can be applied to many nonlinear models. [0003] The softmax function (normalized exponential function) is usually used as the output layer activation function in multi-classification tasks, and as the last layer in deep learning for classification. The softmax function has a ver...

AI Technical Summary

Problems solved by technology

[0009] Aiming at the softmax function existing in the prior art because it has e index calculation and division, hardware implementation needs to consume too many resources or time, and the problem of high complexity, the present invention provides a hardware implementation method, module, chip and system of the softmax function, The softmax function is transformed, avoiding the exponent operation and the division operation in the hardware implementation, and realizing high-performance, low-complexity softmax function hardware implementation, the present invention does not need to iterate to reduce precision, saves calculation time, and can be used in the multi-classification task calculation of neural network high performance

Images