A device for calculating nonlinear activation function based on Newton's iterative method
A Newton iteration method and activation function technology, which is applied in the calculation using the number system, the calculation using non-contact manufacturing equipment, the biological neural network model, etc. The effect of energy consumption
Active Publication Date: 2022-06-21
ZHEJIANG UNIV +1
View PDF16 Cites 0 Cited by
- Summary
- Abstract
- Description
- Claims
- Application Information
AI Technical Summary
Problems solved by technology
[0005] The purpose of the present invention is to provide a nonlinear activation function calculation device based on Newton's iterative method to solve the problem that the activation function in the neural network model is difficult to calculate efficiently and accurately
Method used
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View moreImage
Smart Image Click on the blue labels to locate them in the text.
Smart ImageViewing Examples
Examples
Experimental program
Comparison scheme
Effect test
Embodiment Construction
[0029] In order to make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention, and do not limit the protection scope of the present invention.
[0030] The nonlinear activation function calculation device provided by the embodiment includes a polynomial approximation calculation unit, an exponential function calculation unit, and a Newton iteration calculation unit.
[0031] The nonlinear activation function calculation device provided by the embodiment is mainly suitable for The computation of the nonlinear activation function of , where p 0 ,p 1 ,p 2 are constant parameters. E.g The images of these nonlinear activation functions are generally S-shaped and centrally symmetric, and the input variabl...
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More PUM
Login to View More Abstract
The invention discloses a nonlinear activation function calculation device based on the Newton iterative method, comprising: a polynomial approximation calculation unit for performing polynomial approximation calculation on the nonlinear activation function to obtain a first approximation result; an exponential function calculation unit for An approximate calculation of the exponential function is performed on the exponential part of the nonlinear activation function to obtain a second approximate result; the Newton iteration calculation unit uses the Newton iteration method to perform iterative calculation based on the first approximate result and the second approximate result to obtain an accurate calculation result. The non-linear activation function calculation device is suitable for fixed-point number calculations, and realizes high-precision calculations with less resource consumption, so as to meet the requirements of efficient and accurate activation functions and reduced calculation energy consumption in neural network models.
Description
technical field [0001] The invention relates to the field of computer artificial intelligence and the field of integrated circuit design, in particular to a nonlinear activation function calculation device based on Newton iteration method. Background technique [0002] In the field of artificial intelligence, the expression ability of the linear model is not enough, and the nonlinearity of the neural network model is often increased by the activation function to improve the expression ability of the neural network model. The data is often linearly inseparable. Common activation functions such as sigmoid and tanh involve exponential function operations and division operations. Software often tries to approximate the true value through iterative algorithms, but these functions are difficult to implement directly in hardware. [0003] With the widespread application of artificial intelligence, many deep learning hardware accelerators have emerged. In order to efficiently reali...
Claims
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More Application Information
Patent Timeline
Login to View More Patent Type & Authority Patents(China)
IPC IPC(8): G06F7/50G06F7/52G06N3/04
CPCG06F7/50G06F7/52G06N3/048G06N3/045
Inventor 姜晓红潘哲马德朱国权郝康利
Owner ZHEJIANG UNIV