Softmax function design optimization and hardware realization method and system

Abstract

A Softmax function can complete the conversion from scalar to probability, and is widely used in the output layer of depth neural network classifier. Nowadays, as an important application of deep learning, multi-classification problem has more and more classification categories and higher precision requirements. The invention provides a Softmax function design optimization and hardware realizationmethod and system. According to a large number of input data, wide input range and high precision requirements, the invention calculates through two input modes to reduce on-chip storage resources, responds to a plurality of input pointing schemes through configurable lookup table, and determines output pointing schemes through hardware to improve precision.

Description

technical field [0001] The present invention designs the field of deep neural network classifiers, and specifically designs a Softmax function design optimization and hardware implementation method and system. Background technique [0002] The Softmax function can complete the conversion from scalar to probability, and is widely used in the output layer of deep neural network classifiers. Compared with other layers that only require simple addition and multiplication, such as convolutional layers, pooling layers, and fully connected layers, the Softmax layer requires more complex exponential operations and division operations, which brings corresponding difficulties to the hardware design of the Softmax function. According to the nature of the function, the present invention splits the lookup table of the exponent operation to reduce storage resources, and converts the division into one inversion operation and multiple multiplication operations to reduce the calculation amou...

AI Technical Summary

Problems solved by technology

Compared with other layers that only require simple addition and multiplication, such as convolutional layers, pooling layers, and fully connected layers, the Softmax layer requires more complex exponential operations and division operations, which brings corresponding difficulties to the hardware design of the Softmax function.

Images