Activation device and method for neural network processor
A technology for activating devices and neural networks, applied in the field of neural network processors, can solve problems such as consuming large hardware resources, achieve the effect of reducing hardware resources and improving resource utilization
Pending Publication Date: 2019-08-20
INST OF COMPUTING TECH CHINESE ACAD OF SCI
View PDF5 Cites 1 Cited by
- Summary
- Abstract
- Description
- Claims
- Application Information
AI Technical Summary
Problems solved by technology
In order to make the designed processor hardware compatible with various types of activation functions, a matching lookup table is often set for each activation function in the neural network processor, which consumes a lot of time for storage and search matching. hardware resources
Method used
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View moreImage
Smart Image Click on the blue labels to locate them in the text.
Smart ImageViewing Examples
Examples
Experimental program
Comparison scheme
Effect test
Embodiment Construction
[0051] After studying the nonlinear activation functions used in neural network processors, the inventors found that there are commonalities between different types of nonlinear activation functions, such as Figure 1a Shown are the sigmoid function curves and Figure 1b The tanh function curves shown have approximately the same profile. Therefore, the inventor proposes that different types of non-linear activation functions can share a set of lookup tables, through simple calculations on independent variables and / or on function values, for example, from a lookup table corresponding to a certain activation function. The function value is processed to obtain the function value corresponding to another activation function. Based on the above idea, the present invention proposes an activation device structure for a neural network processor, which can share a lookup table, and use a lookup table for one type of activation function to implement activation operations for multiple ac...
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More PUM
Login to View More Abstract
The invention provides an activation device for a neural network processor and a processor. The activation device comprises a lookup table unit, an activation control unit, a lookup preprocessing unitand a lookup post-processing unit, wherein the lookup table unit stores a corresponding relation between an independent variable value and a function value of an activation function type; the activation control unit is used for generating a control signal for the lookup table preprocessing unit and the lookup post-processing unit according to a currently selected activation function type; the lookup preprocessing unit is used for obtaining a to-be-searched value according to the control signal and the to-be-activated neuron value x; the lookup table unit is used for finding a function value matching the value to be looked up from a corresponding relation between an independent variable stored in the lookup table unit and an activation function value of the independent variable to serve asa lookup value; and the lookup post-processing unit is used for obtaining an activation result of the to-be-activated neuron value x according to the control signal and the lookup value.
Description
technical field [0001] The invention relates to the field of neural network processors, in particular to hardware acceleration for activation units in the neural network processor. Background technique [0002] Deep neural network is one of the perception models with the highest level of development in the field of artificial intelligence. This type of network simulates the neural connection structure of the human brain by building a model, and describes the data features layered through multiple transformation stages, providing image, video and audio Such large-scale data processing tasks have brought breakthroughs. The model structure is an operational model consisting of a large number of nodes interconnected through a mesh structure, and these nodes are called neurons. The strength of the connection between each two nodes represents the weighted value of the connection signal between the two nodes, called weight, which corresponds to the memory in the human neural netwo...
Claims
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More Application Information
Patent Timeline
Login to View More Patent Type & Authority Applications(China)
IPC IPC(8): G06N3/063G06F16/245
CPCG06F16/245G06N3/063
Inventor 韩银和闵丰许浩博王颖王郁杰
Owner INST OF COMPUTING TECH CHINESE ACAD OF SCI