Softmax function calculation method based on piecewise lookup table and hardware system

Abstract

The invention discloses a softmax function calculation method based on a piecewise lookup table and a hardware system. According to the method, calculation of a softmax function is realized by adopting a segmented lookup table mode; data from-9.9999999 to-9.9900000 is represented by one -9.99; n sections are deduced in the same manner; in this way, the number of the storage data of the lookup table is N * 899; in practice, in order to facilitate searching, addresses of 900-1023, 1924-2047 and the like are vacated, the lookup table is expanded to N * 1024, the precision of input data can reach10 <-N-1>, the size of the lookup table is far smaller than that of a traditional lookup table, and storage resources of the lookup table on hardware are greatly reduced. A hardware system adopts an Xilinx ZYNQ platform. Compared with a traditional lookup table method, the method has the advantages that the size of the lookup table is reduced on the premise that the precision of input data and calculation results is guaranteed, hardware calculation in other non-lookup-table methods is not needed, and the area power consumption requirement of hardware is reduced.

Description

technical field [0001] The invention belongs to the field of hardware acceleration of neural networks, and relates to a softmax function calculation method and a hardware system based on a segmentation lookup table. Background technique [0002] Artificial neural networks are currently widely used in artificial intelligence applications, such as image recognition and voice assistants. With the gradual increase in the scale of neural networks, the amount of calculations has also increased sharply. Traditional general-purpose chips are limited by bandwidth and power consumption when processing complex neural networks. Therefore, dedicated neural network acceleration hardware has begun to receive attention, with lower energy consumption and higher performance. In the traditional artificial neural network, the softmax function is widely used in the last layer for the calculation of the output value. Therefore, the hardware implementation method of the softmax function is also ...

AI Technical Summary

Problems solved by technology

However, in the actual calculation process, polynomial calculations are also required. When the precision requirements are high, the required polynomial order is high, so there will be a large number of multiplication operations, which will not only consume hardware computing resources, but also increase calculation delays.

[0006] 3) CORDIC algorithm: This is a calculation method for coordinate rotation numbers, which has almost no hardware storage requirements, but the calculation process is more complicated, the calculation time is long, and the accuracy is not high enough

[0007] 4) The article "Efficient Hardware Architecture of Softmax Layer in DeepNeural Network", DSP 2018 proposed a method using Integral Stochastic Computing (Integral Stochastic Computing) to calculate the e index, which reduces the area and power consumption of the hardware, but the final calculation accuracy Only 96%, can not meet the exact requirements

Images