MISH function implementation method based on improved CORDIC algorithm

Abstract

The invention belongs to the technical field of integrated circuit digital ICs, and particularly relates to an MISH function implementation method based on an improved CORDIC algorithm. Comprising the following steps: firstly, carrying out equivalent transformation on an MISH function expression; performing operation on the MISH function expression after the equivalent transformation to decompose the MISH function expression into five parts; according to the method, operation is achieved through a CORDIC algorithm module Rotation mode based on hyperbolic coordinates, a CORDIC algorithm module Rotation mode based on hyperbolic coordinates, a CORDIC algorithm module Rotation mode based on hyperbolic coordinates and a CORDIC algorithm module Vectoring mode based on a linear coordinate system, and finally the operation result of the MISH function is obtained. According to the method, on the basis of not reducing the calculation precision, the processing delay and the hardware resource consumption are reduced by 50%, and the calculation convergence range is expanded by 30%.

Description

Technical Field

[0001] The invention belongs to the technical field of integrated circuit digital IC, and in particular relates to a MISH function realization method based on an improved CORDIC algorithm. Background Art

[0002] In recent years, in the field of artificial intelligence, neural network models have developed rapidly as one of the core technologies. As an important component of "neurons" in neural networks, activation functions introduce nonlinear mapping learning capabilities, which plays a critical role in the training and generalization of neural networks. The MISH activation function is a smooth, continuous, self-regularized, non-monotonic activation function. On datasets such as CIFAR-10 and IMAGENet-1k, and in more than a dozen neural network models such as ResNet, WRN, SimpleNet, Xceptino Net, DenseNet, and MobileNet, the MISH activation function has achieved higher accuracy and more optimal results than activation functions such as SWISH and ReLU, and is a very high-quality activation function.

[0003] The expression of MISH activation function is It can be seen that the MISH activation function combines multiplication, tanh , ln and e Many special operations such as exponentiation are very complex when implemented in hardware circuits. tanh Trigonometric functions, etc. ln and e Traditional implementation methods of exponential complex function operations in hardware circuits include lookup tables, Taylor series expansion, CORDIC algorithm, etc. Due to the large number of complex operations of MISH activation functions, the use of multi-level lookup tables has the problem of occupying large on-chip storage resources, and the use of multi-level series expansion has the problem of truncation and rounding errors accumulating through multiple levels, resulting in low calculation accuracy. The CORDIC algorithm proposed in 1957 has three operation coordinate systems, each of which has two working modes. It can implement multiplication, division, trigonometric functions such as sinh / cosh / tanh, etc. based on simple shifts and additions. ln and e Exponential complex calculations are very suitable for application in hardware circuits.

[0004] The unified rotation iteration formula of the CORDIC algorithm is shown as follows: ; Among them: it includes three processing links of X / Y / Z; the coordinate system is m Select the value. When is a hyperbolic coordinate system, When is a linear coordinate system, In Rotation mode (RM), the direction of rotation of the rotation stage is d i The values ​​are: ; In Vectoring mode (VM), the direction of rotation of the rotating stage d i The values ​​are: (Hyperbolic / Linear Coordinate System); (circular coordinate system); Fixed rotation angles of CORDIC algorithm in different coordinate systems It is expressed as: ; When the hardware circuit is implemented, the fixed rotation angle is generally pre-stored in a small ROM table.

[0005] In particular, for rotations in hyperbolic coordinates, to ensure convergence, the rotation level i Yes(3* k +1) needs to be repeated once, a typical rotation sequence i is 1, 2, 3, 4, 4, 5, 6, 7, 7, 8, 9, 10, 10, 11, 12,13, 13. For the hyperbolic coordinate system Rotation mode, in this rotation range, the convergence interval of the CORDIC algorithm is (-1.113, 1.113); the rotation gain in the hyperbolic coordinate system is: ,in N is the total number of rotation stages, the typical value of the rotation gain It is 0.828.

[0006] From the expression of the MISH activation function, we can see that from the inner layer to the outer layer of the expression, the CORDIC algorithm module needs to be called four times when it is implemented based on the CORDIC algorithm. First, the CORDIC algorithm module Rotation mode under hyperbolic coordinates is used to calculate , and then use the CORDIC algorithm module Vectoring mode in hyperbolic coordinates to calculate Then use the CORDIC algorithm module Rotation mode in hyperbolic coordinates to calculate Finally, the CORDIC algorithm module Vectoring mode in the linear coordinate system is used to calculate , and get the final calculation result. The basic operation idea of ​​CORDIC algorithm is approximate calculation by iterative approximation. This four-stage serial cascade implementation method obviously has problems such as long processing cycle, large calculation error and large hardware resource consumption. Summary of the invention

[0007] The purpose of the present invention is to overcome the above-mentioned shortcomings of the prior art. In view of the problems that the large number of series-connected modules leads to long processing cycle, low calculation accuracy, high hardware consumption, etc. when the MISH function is implemented based on the CORDIC algorithm, an implementation method with low latency, high accuracy and low hardware consumption is provided. Based on the optimized function implementation structure and the improved CORDIC algorithm, the hardware implementation performance of the MISH function is improved.

[0008] In order to solve the above technical problems, the present invention provides a MISH function implementation method based on an improved CORDIC algorithm, comprising the following steps: Step S1: First, perform an equivalent transformation on the MISH function expression. Equivalent conversion to ; Step S2: Then, the equivalently converted MISH function expression is decomposed into , , , and division of five parts; in Part of the calculation is based on the CORDIC algorithm module Rotation mode in hyperbolic coordinates. The input of the CORDIC algorithm module is x in , y in , z in Both x , at this time the output of the CORDIC algorithm module x out for ,in K 0 is the pseudo rotation gain of the hyperbolic CORDIC algorithm module, which is a fixed constant when the total number of rotations is constant; Partly through x Multiply To achieve this; considering the CORDIC algorithm module Gain, will Partially converted to and with the above output x out Add together to get ; This is partly based on the CORDIC algorithm module Rotation mode under hyperbolic coordinates. The input of the CORDIC algorithm module is x in , y in All are 1, input z in for x When the output of the CORDIC algorithm module is x out That is , and add The operation of ; This is partly based on the CORDIC algorithm module Rotation mode under hyperbolic coordinates. The input of the CORDIC algorithm module is x in , y in All are 1, input z in for- x When y out That is , and multiply it by 2 to get , for the above output x out and output y out Add together to get ; The division part is implemented based on the Vectoring mode of the CORDIC algorithm module in the linear coordinate system. The input of the CORDIC algorithm module x in for ,enter y in for ,enter z in is 0, then the module output z out That is , which can be simplified to , which is the operation result of the MISH function.

[0009] Preferably, the CORDIC algorithm module Rotation mode based on hyperbolic coordinates is used to calculate Part, at this time, the CORDIC algorithm module iteration formula is improved to: ; in, ; ; Among them, the above i Indicates i Rotation stage; Indicates i The output of the X processing chain in the rotation stage; Indicates i The input of the X processing chain in the rotation stage; Indicates i The output of the Z processing chain in the rotation stage; Indicates i The input of the Z processing chain in the rotation stage; Indicates i The direction of rotation of the rotation stage; Indicates i Fixed rotation angle of each rotation stage.

[0010] Preferably, the CORDIC algorithm module Rotation mode based on hyperbolic coordinates is used to calculate and , then the CORDIC algorithm module iteration formula is improved to: ; in, ; ; Among them, the above i Indicates i Rotation stage; Indicates i The output of the X processing chain in the rotation stage; Indicates i The input of the X processing chain in the rotation stage; Indicates i The output of the Y processing chain in the rotation stage; Indicates i The input of the Y processing chain in the rotation stage; Indicates i The output of the Z processing chain in the rotation stage; Indicates i The input of the Z processing chain in the rotation stage; Indicates i The direction of rotation of the rotation stage; Indicates i Fixed rotation angle of each rotation stage.

[0011] Preferably, the CORDIC algorithm module Rotation mode based on hyperbolic coordinates is used to calculate as well as and ,enter z in Both x， Therefore only one Z processing chain is used during operation.

[0012] Preferably, the CORDIC algorithm module Vectoring mode based on the linear coordinate system is implemented, and the CORDIC algorithm module iteration formula is improved to: ; Among them, input x in must be greater than 0, so: ; ; Among them, the above i Indicates i Rotation stage; Indicates i The output of the Y processing chain in the rotation stage; Indicates i The input of the Y processing chain in the rotation stage; represents the input of the X processing chain of the CORDIC algorithm; Indicates i The output of the Z processing chain in the rotation stage; Indicates i The input of the Z processing chain in the rotation stage; Indicates i The direction of rotation of the rotation stage; Indicates i Fixed rotation angle of each rotation stage.

[0013] Preferably, the calculation structure of the MISH function adopts a cascaded first-level CORDIC algorithm module and a second-level CORDIC algorithm module, and a multiplication, addition and shift processing unit for integrating the input and output of the first-level CORDIC algorithm module; The first-level CORDIC algorithm module includes: three operation modules of the CORDIC algorithm in a hyperbolic coordinate system; the second-level CORDIC algorithm module includes: one operation module of the CORDIC algorithm in a linear coordinate system.

[0014] Preferably, the MISH function implementation method uses the CORDIC algorithm module Rotation mode, whose input x in and input y inare equal.

[0015] Compared with the prior art, the present invention has the following beneficial effects: The present invention proposes a MISH function implementation method based on an improved CORDIC algorithm, which can solve the problems of long processing cycle, low calculation accuracy, large hardware consumption, etc. of traditional methods by simplifying MISH expressions and improving CORDIC algorithm modules, wherein the processing cycle can be reduced from the traditional four serially connected CORDIC algorithm modules to two-stage serially connected CORDIC algorithm modules, and the processing cycle is reduced by half; when implemented in a hardware circuit, the four CORDIC algorithm modules of the traditional method require a total of 12 X / Y / Z processing links, while the method described in this article only requires 6 X / Y / Z processing links, and the hardware resource consumption is also reduced by half; simulation results show that in terms of calculation accuracy, under the same calculation bit width and number of rotations, the method described in this article can achieve a calculation accuracy of the order of 10^-5 within the convergence range of the CORDIC algorithm, and the calculation range of the method of the present invention is expanded by nearly 30% compared with the traditional implementation method. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 It is a structural block diagram of the MISH function implementation based on the two-stage cascade CORDIC algorithm module of the present invention.

[0017] Figure 2 It is a block diagram of the implementation of the Rotation mode of the improved CORDIC algorithm module of the present invention.

[0018] Figure 3 It is the simulation and comparison of the calculation accuracy of the MISH function of the present invention. DETAILED DESCRIPTION

[0019] The present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. The advantages and features of the present invention will become more apparent from the following description. It should be noted that the accompanying drawings are in very simplified form and in non-precise proportions, and are only used to conveniently and clearly assist in explaining the purpose of the embodiments of the present invention.

[0020] The embodiment of the present invention specifically provides a method for implementing a MISH function based on an improved CORDIC algorithm, comprising the following steps: (1) First, perform the following equivalent transformation on the expression of the MISH function: ; (2) Then the converted MISH function expression is decomposed. According to the converted MISH function expression, its calculation can be decomposed into the following five parts: , , , , and division, where Part of the calculation is based on the CORDIC algorithm module Rotation mode in hyperbolic coordinates. The input of the CORDIC algorithm module is x in , y in , z in Both x , at this time the output of the CORDIC algorithm module x out for ,in K 0 is the pseudo rotation gain of the hyperbolic CORDIC algorithm module, which is a fixed constant when the total number of rotations is constant; Partly through x Multiply Considering the CORDIC algorithm module Gain, will Partial conversion to achieve and with the above output x out Add together to get ; This is partly based on the CORDIC algorithm module Rotation mode under hyperbolic coordinates. The input of the CORDIC algorithm module is x in , y in All are 1, input z in for x When the output of the CORDIC algorithm module is x out That is , and add The operation of ; This is partly based on the CORDIC algorithm module Rotation mode under hyperbolic coordinates. The input of the CORDIC algorithm module is x in , y in All are 1, input z in for- x When y outThat is , and multiply it by 2 to get ; For the above output x out and output y out Add together to get ; The division part is implemented based on the Vectoring mode of the CORDIC algorithm module in the linear coordinate system. The input of the CORDIC algorithm module x in for ,enter y in for ,enter z in is 0, then the module output z out That is , which can be simplified to , which is the operation result of the MISH function.

[0021] In the embodiment of the present invention, , , Improvements in calculations such as division using the CORDIC algorithm module include: (1) Calculation based on the CORDIC algorithm module Rotation mode in hyperbolic coordinates , the input of the CORDIC algorithm module x in , y in , z in Both x , then the CORDIC algorithm module iteration formula is improved to: ; in, ; ; After the above improvements, the CORDIC algorithm module outputs x out That is .

[0022] (2) Calculation based on the CORDIC algorithm module Rotation mode in hyperbolic coordinates and , the input of the CORDIC algorithm module x in , y in All are 1,z in for x , then the CORDIC algorithm module iteration formula is improved to: ; in, ; ; After the above improvements, the CORDIC algorithm module outputs x out That is , y out That is .

[0023] (3) Calculation based on the CORDIC algorithm module Rotation mode in hyperbolic coordinates as well as and When input z in Both x , so only one Z processing link can be implemented during implementation; (4) The CORDIC algorithm module Vectoring mode is implemented based on the linear coordinate system. The input of the CORDIC algorithm module x in for ,enter y in for ,enter z in is 0, and the CORDIC algorithm module iteration formula is improved to: ; Among them, input x in must be greater than 0, so: ; ; After the above improvements, the CORDIC algorithm module outputs z out That is .

[0024] like Figure 1As shown in the figure, it is a structural block diagram of the MISH function implementation based on the improved CORDIC algorithm proposed in the present invention, which mainly includes two-stage cascaded CORDIC algorithm modules, and multiplication, addition and shift processing units for integrating the input and output of the first-stage CORDIC algorithm module, including three operation modules of the CORDIC algorithm in the hyperbolic coordinate system (first stage) and one operation module in the linear coordinate system (second stage). Among them, the Z processing links in the three implementations of the first-stage CORDIC algorithm module are all for input x Conducted (mainly involving the direction of rotation of each stage d i The Z processing link can be shared among the three operation modules to reduce the hardware resource consumption during circuit implementation. In addition, in the hyperbolic coordinate system mode, since the X / Y processing link is x in and y in At the same time, the processing operations are consistent, and one operation link can be optimized, so that the three CORDIC algorithm modules of the first level are reduced from the traditional three groups of X / Y / Z with a total of 9 processing links to X / Y and X / Z with a total of 4 processing links; the CORDIC algorithm module under the second-level linear coordinate system can also be omitted because the X processing link remains unchanged, and this level of CORDIC algorithm module is reduced to Y / Z with a total of 2 processing links. In this way, the MISH function implementation method based on the CORDIC algorithm described in the present invention only needs to implement 6 X / Y / Z processing links when implementing. Since there are only two Z processing links, only 2 fixed rotation angle lookup tables are required. Compared with the traditional cascade implementation of 4 CORDIC algorithm modules, the method described in the present invention reduces half of the X / Y / Z processing links and fixed rotation angle lookup table resource consumption. At the same time, since the method described in the present invention has only two levels of CORDIC algorithm modules cascaded, the processing delay is also reduced to half of the traditional method.

[0025] like Figure 2 As shown in FIG. 1 , the present invention improves the CORDIC algorithm module Rotation mode i The biggest difference from the traditional CORDIC module is that the CORDIC algorithm module Rotation mode used in the present invention has an input x in and input y in are equal, so the calculated values ​​of the X processing link and the Y processing link are exactly the same. When the circuit is implemented, the X processing link and the Y processing link do not need to cross-use each other's values. The corresponding processing function can be realized based on the independent X or Y processing link. Figure 2 As shown in xin and y in All are 1, input z in When 1, the X processing link can perform calculations K 0*e^ x , while the Y processing link can realize the calculation K 0*e^(- x ).

[0026] Figure 3 The figure shows the simulation of the calculation accuracy of the MISH activation function based on the method of the present invention, and the comparison of the calculation accuracy performance with the traditional 4-level cascade CORDIC algorithm module. The algorithm parameters during implementation are shown in Table 1. As can be seen from the figure, the calculation accuracy of the method of the present invention reaches 10^-5 in the full input range, while the traditional method only maintains this calculation accuracy in the middle (-1, 0.721) area, and the calculation results of other input ranges have large errors. Therefore, the method of the present invention maintains a high calculation accuracy in the full convergence range, and the calculation convergence range is expanded by 29.34% relative to the traditional method.

[0027] Table 1: List of simulation parameters for calculation accuracy of the method of the present invention In the above table, the three hyperbolic coordinate system RM mode CORDIC algorithm modules used in the method of the present invention are z in Both with input x Therefore, the relevant processing can be optimized by fusion, and only one Z processing link needs to be retained.

[0028] The above description is only a description of the preferred embodiments of the present invention, and is not intended to limit the scope of the present invention. Any changes or modifications made by a person skilled in the art in the field of the present invention based on the above disclosure shall fall within the scope of protection of the claims.

Claims

1. A method for implementing a MISH function based on an improved CORDIC algorithm, characterized in that: The steps include: Step S1: First, perform an equivalent transformation on the MISH function expression. Equivalent conversion to ; Step S2: Then, the equivalently converted MISH function expression is decomposed into , , , and division of five parts; in Part of the calculation is based on the CORDIC algorithm module Rotation mode under hyperbolic coordinates. The input of the CORDIC algorithm module is x in , y in , z in Both x , at this time the output of the CORDIC algorithm module x out for ,in K 0 is the pseudo rotation gain of the hyperbolic CORDIC algorithm module, which is a fixed constant when the total number of rotations is constant; Partly through x Multiply To achieve this; considering the CORDIC algorithm module Gain, will Partially converted to and with the above output x out Add together to get ; This is partly based on the CORDIC algorithm module Rotation mode under hyperbolic coordinates. The input of the CORDIC algorithm module is x in , y in All are 1, input z in for x When the output of the CORDIC algorithm module is x out That is , and add The operation of ; This is partly based on the CORDIC algorithm module Rotation mode under hyperbolic coordinates. The input of the CORDIC algorithm module is x in , y in All are 1, input z in for- x When y out That is , and multiply it by 2 to get , for the above output x out and output y out Add together to get ; The division part is implemented based on the Vectoring mode of the CORDIC algorithm module in the linear coordinate system. The input of the CORDIC algorithm module x in for ,enter y in for ,enter z in is 0, then the module output z out That is , which can be simplified to , which is the operation result of the MISH function.

2. A method for implementing a MISH function based on an improved CORDIC algorithm as claimed in claim 1, characterized in that: The CORDIC algorithm module Rotation mode based on hyperbolic coordinates is used to calculate Part, at this time, the CORDIC algorithm module iteration formula is improved to: ； in, ； ； Among them, the above i Indicates i Rotation stage; Indicates i The output of the X processing chain in the rotation stage; Indicates i The input of the X processing chain in the rotation stage; Indicates i The output of the Z processing chain in the rotation stage; Indicates i The input of the Z processing chain in the rotation stage; Indicates i The direction of rotation of the rotation stage; Indicates i Fixed rotation angle of each rotation stage.

3. A method for implementing a MISH function based on an improved CORDIC algorithm as claimed in claim 1, characterized in that: The CORDIC algorithm module Rotation mode based on hyperbolic coordinates is used to calculate and , then the CORDIC algorithm module iteration formula is improved to: ； in, ； ； Among them, the above i Indicates i Rotation stage; Indicates i The output of the X processing chain in the rotation stage; Indicates i The input of the X processing chain in the rotation stage; Indicates i The output of the Y processing chain in the rotation stage; Indicates i The input of the Y processing chain in the rotation stage; Indicates i The output of the Z processing chain in the rotation stage; Indicates i The input of the Z processing chain in the rotation stage; Indicates i The direction of rotation of the rotation stage; Indicates i Fixed rotation angle of each rotation stage.

4. The method for implementing a MISH function based on an improved CORDIC algorithm as claimed in claim 1, characterized in that: The CORDIC algorithm module Rotation mode based on hyperbolic coordinates is used to calculate as well as and ,enter z in Both x， Therefore only one Z processing chain is used during operation.

5. The method for implementing a MISH function based on an improved CORDIC algorithm as claimed in claim 1, characterized in that: The method is implemented based on the Vectoring mode of the CORDIC algorithm module in the linear coordinate system. At this time, the iteration formula of the CORDIC algorithm module is improved to: ； Among them, input x in must be greater than 0, so: ； ； Among them, the above i Indicates i Rotation stage; Indicates i The output of the Y processing chain in the rotation stage; Indicates i The input of the Y processing chain in the rotation stage; represents the input of the X processing chain of the CORDIC algorithm; Indicates i The output of the Z processing chain in the rotation stage; Indicates i The input of the Z processing chain in the rotation stage; Indicates i The direction of rotation of the rotation stage; Indicates i Fixed rotation angle of each rotation stage.

6. A method for implementing a MISH function based on an improved CORDIC algorithm as claimed in claim 1, characterized in that: The calculation structure of the MISH function adopts a cascaded first-level CORDIC algorithm module and a second-level CORDIC algorithm module, and a multiplication, addition and shift processing unit for integrating the input and output of the first-level CORDIC algorithm module; The first-level CORDIC algorithm module includes: three operation modules of the CORDIC algorithm in a hyperbolic coordinate system; the second-level CORDIC algorithm module includes: one operation module of the CORDIC algorithm in a linear coordinate system.

7. A method for implementing a MISH function based on an improved CORDIC algorithm as claimed in claim 1, characterized in that: The MISH function implementation method uses the CORDIC algorithm module Rotation mode, and its input x in and input y in are equal.

Images