Mamba nonlinear function operation circuit and operation method thereof
By converting the Mamba nonlinear function into a combination of logarithmic and exponential operations with base 2, and optimizing the computational pipeline using the HGHR-CORDIC and HGHV-CORDIC algorithms, the computational complexity of Sofplus and SiLU in the Mamba model is solved, achieving high-precision and low-complexity nonlinear function computation.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- SUN YAT SEN UNIV
- Filing Date
- 2026-02-11
- Publication Date
- 2026-06-12
AI Technical Summary
The nonlinear functions Sofplus and SiLU of the Mamba model are computationally complex, making it difficult to achieve a good trade-off between accuracy, computational speed, and complexity. Existing technical solutions have excessive hardware overhead in scenarios requiring high precision.
Design a Mamba nonlinear function operation circuit. By converting the Sofplus and SiLU functions into a combination of logarithmic and exponential operations with base 2, and optimizing the computation pipeline using HGHR-CORDIC and HGHV-CORDIC algorithms, the exponential and logarithmic calculations of the common part are realized, reducing the circuit complexity.
It achieves high-precision nonlinear function calculation, reduces circuit structure and computational complexity, improves computational efficiency, adapts to efficient hardware support, and meets the high-precision requirements of the large Mamba model.
Smart Images
Figure CN122195387A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of integrated circuit technology, and in particular to a Mamba nonlinear function operation circuit and its operation method. Background Technology
[0002] The Mamba architecture is a deep learning model designed for sequence modeling scenarios. Its core design combines the characteristics of selective state-space models with the advantages of gating mechanisms. Dedicated accelerator optimization for the Mamba model is in its early exploratory stages, and there is still considerable room for expansion in related technical solutions and performance improvement research. In particular, the computational complexity of the nonlinear functions Sofplus and SiLU in Mamba poses a significant challenge to its circuit design, making it difficult to achieve a good trade-off between accuracy, computational speed, and complexity.
[0003] In summary, the technical problems existing in the relevant technologies need to be improved. Summary of the Invention
[0004] The main objective of this application is to propose a Mamba nonlinear function calculation circuit and its calculation method, which can reduce circuit complexity while achieving high-precision and efficient nonlinear function calculation.
[0005] To achieve the above objectives, one aspect of this application provides a Mamba nonlinear function operation circuit, the circuit including a Sofplus operation module and a SiLU operation module, the Sofplus operation module including a first exponent calculation module and a first logarithm calculation module, and the SiLU operation module including a first exponent calculation module, a first logarithm calculation module, a second logarithm calculation module and a second exponent calculation module; The first exponent calculation module is connected to the first logarithm calculation module, the second logarithm calculation module is connected to the second exponent calculation module, and the first logarithm calculation module is connected between the second logarithm calculation module and the second exponent calculation module; The first exponent calculation module is used to respond to the Sofplus function calculation instruction, calculate the first exponent function in the Sofplus function according to the first input parameter, and obtain the first calculation result; The first logarithm calculation module is used to calculate the first logarithm function, which includes the first exponential function, in the Sofplus function based on the first calculation result, to obtain the Sofplus intermediate result; The first exponent calculation module is also used to respond to the SiLU function calculation instruction, calculate the second exponent function in the SiLU function according to the second input parameter, and obtain the second calculation result; The first logarithm calculation module is further configured to calculate the second logarithm function, which includes the second exponential function, in the SiLU function based on the second calculation result, to obtain a third calculation result; The second logarithm calculation module is used to calculate the third logarithm function in the SiLU function according to the second input parameters, and obtain the fourth calculation result; The second index calculation module is used to perform index calculation based on the third and fourth calculation results to obtain the SiLU target result.
[0006] In some embodiments, the circuit further includes a preprocessing module, a data selector, and a multiplier, wherein the preprocessing module is connected to the multiplier, the data selector is connected to the multiplier, and the multiplier is connected to the first exponent calculation module; The preprocessing module is used to transmit the first input parameter to the multiplier in response to the Sofplus function calculation instruction; The data selector is used to output a first constant to the multiplier in response to the Sofplus function calculation instruction; The multiplier is used to multiply the first input parameter by the first constant to obtain the exponential part of the first exponential function.
[0007] In some embodiments, the preprocessing module is further configured to transmit the second input parameter to the multiplier in response to the SiLU function calculation instruction; The data selector is used to output a second constant to the multiplier in response to the SiLU function calculation instruction; The multiplier is used to multiply the second input parameter by the second constant to obtain the exponential part of the second exponential function.
[0008] In some embodiments, the first index calculation module includes a separation module, an HGHR-CORDIC module, and a shift module; The separation module is connected to the HGHR-CORDIC module and the shift module respectively, and the HGHR-CORDIC module is connected to the shift module; The separation module is used to split the exponential part of the first exponential function to obtain the integer and decimal parts of the exponent; The HGHR-CORDIC module is used to calculate the exponential function with the decimal as the exponent using the HGHR-CORDIC algorithm to obtain the first intermediate result; The shift module is used to left-shift the first intermediate result using the integer as the displacement number to obtain the first calculation result.
[0009] In some embodiments, the first logarithmic calculation module includes a leader detector, an HGHV-CORDIC module, a first adder, and a second adder: The first exponent calculation module is connected to the first adder, the first adder is connected to the first leading detector, the first leading detector is connected to the HGHV-CORDIC module and the second adder respectively, and the HGHV-CORDIC module is connected to the second adder; The first adder is used to add the first calculation result to one to obtain the argument of the first logarithmic function; The leading detector is used to perform floating-point format conversion on the true number to obtain the exponent and mantissa bits of the floating-point representation of the true number; The HGHV-CORDIC module is used to calculate the logarithmic function with the mantissa as the argument to obtain a second intermediate result; The second adder is used to add the second intermediate result and the exponent to obtain the Sofplus intermediate result.
[0010] In some embodiments, the second logarithmic calculation module includes a leader detector, an HGHV-CORDIC module, and an adder; The preprocessing module is connected to the first preprocessor detector, the first preprocessor detector is connected to the HGHV-CORDIC module and the adder respectively, and the HGHV-CORDIC module is connected to the adder; The preprocessing module is also used to transmit the second input parameter to the leading detector in response to the SiLU function calculation instruction; The leading detector is used to perform floating-point number format conversion on the second input parameter to obtain the exponent and mantissa bits of the floating-point representation of the second input parameter; The HGHV-CORDIC module is used to calculate the logarithmic function with the mantissa as the argument to obtain a third intermediate result; The adder is used to add the third intermediate result and the exponent to obtain the fourth calculation result.
[0011] In some embodiments, the second index calculation module includes a separation module, an HGHR-CORDIC module, a shifting module, and a post-processing module; The preprocessing module is connected to the postprocessing module, the separation module is connected to the HGHR-CORDIC module and the shifting module respectively, the HGHR-CORDIC module is connected to the shifting module, and the shifting module is connected to the postprocessing module; The preprocessing module is also used to respond to the SiLU function calculation instruction by identifying the positive or negative number of the second input parameter to obtain a sign for a positive, zero, or negative number. The separation module is used to split the subtraction result of the fourth calculation result and the third calculation result to obtain the integer and decimal parts of the subtraction result; The HGHR-CORDIC module is used to calculate the exponential function with the decimal as the exponent using the HGHR-CORDIC algorithm to obtain the fourth intermediate result; The shift module is used to shift the fourth intermediate result to the left by an integer number of positions, using the integer as the shift factor, to obtain the fifth intermediate result; The post-processing module is used to positive, zero, or negative the fifth intermediate result according to the flag symbol to obtain the SiLU target result.
[0012] In some embodiments, the HGHR-CORDIC module includes a multiplier, a P branch, a 1 / K branch, and a W branch; The W branch is used to search in a preset lookup table based on the hyperbolic angle, determine the iteration step size, and calculate the hyperbolic angle of the next generation; The P branch is used to iteratively update the hyperbolic function value according to the iteration step size, so as to obtain the objective function value of the sum of hyperbolic sine and hyperbolic cosine. The 1 / K branch is used to iteratively update the proportional compensation factor according to the iteration step size to obtain the target compensation factor. The multiplier is used to multiply the objective function value with the objective compensation factor to obtain the output result of the exponential function, which includes the first intermediate result and the fourth intermediate result.
[0013] To achieve the above objectives, another aspect of this application proposes a Mamba nonlinear function operation method, which is applied to the Mamba nonlinear function operation circuit described in the above embodiments. The method includes the following steps: The Sofplus function in Mamba is converted into a first combined function comprising a first exponential function and a first logarithmic function, and the SiLU function in Mamba is converted into a second combined function comprising a second exponential function, a second logarithmic function, and a third logarithmic function; In response to the Sofplus function calculation instruction, the first exponential function is calculated based on the first input parameters to obtain the first calculation result; Based on the first calculation result, the first logarithmic function, which includes the first exponential function, is calculated to obtain the Sofplus intermediate result; In response to the SiLU function calculation instruction, the second exponential function is calculated according to the second input parameters to obtain the second calculation result; The second logarithmic function, which includes the second exponential function, is calculated based on the second calculation result to obtain the third calculation result; The third logarithmic function is calculated based on the second input parameters to obtain a fourth calculation result; The SiLU target result is obtained by performing an exponential calculation based on the third and fourth calculation results.
[0014] In some embodiments, the first combined function comprising the first exponential function and the first logarithmic function is represented as follows: ; in, This represents the first exponential function. Represents the first logarithmic function. This is represented as the first input parameter.
[0015] In some embodiments, the second combined function comprising the second exponential function, the second logarithmic function, and the third logarithmic function is represented as follows: When the second input parameter is greater than zero, the second combination function is expressed as: ; in, This represents the second exponential function. This represents the second logarithmic function. This represents the third logarithmic function. This is represented as the second input parameter; When the second input parameter is equal to zero, the second combination function is represented as zero; When the second input parameter is less than zero, the second combination function is expressed as: ; in, This represents the second exponential function. This represents the second logarithmic function. This represents the third logarithmic function.
[0016] To achieve the above objectives, another aspect of this application provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the method described above.
[0017] To achieve the above objectives, another aspect of the embodiments of this application proposes a computer-readable storage medium storing a computer program that, when executed by a processor, implements the methods described above.
[0018] To achieve the above objectives, another aspect of the embodiments of this application proposes a computer program product, including a computer program that, when executed by a processor, implements the aforementioned method.
[0019] The embodiments of this application include at least the following beneficial effects: This application provides a Mamba nonlinear function operation circuit and its operation method. This scheme uses the first exponent calculation module and the first logarithm calculation module as a common circuit for the Sofplus operation module and the SiLU operation module, processing the common part of the transformed Sofplus function and SiLU function, and completing the calculation of the exponent and logarithm of the common part. This helps reduce the complexity of the circuit structure and operation. The combination of exponent and logarithm after transforming the Sofplus function and SiLU function is more adaptable to the hardware's efficient support for operation, reducing computational costs and improving the efficiency of inference or training. This circuit implements a unified architecture for nonlinear functions in large-scale Mamba models, realizing the calculation of Sofplus and SiLU nonlinear functions in Mamba, meeting the requirements of high-precision scenarios, and maintaining high efficiency and low complexity in computation. Attached Figure Description
[0020] Figure 1 This is a schematic diagram of the structure of a Mamba nonlinear function operation circuit provided in an embodiment of this application; Figure 2 This is a schematic diagram of the operation of the Sofplus computing module provided in the embodiments of this application; Figure 3 This is a schematic diagram of the operation of a Mamba nonlinear function operation circuit provided in an embodiment of this application; Figure 4 This is a schematic diagram of the operation of the SiLU operation module provided in the embodiments of this application; Figure 5 This is a schematic diagram of the operation of the HGHR-CORDIC module provided in the embodiments of this application; Figure 6 This is a schematic diagram of the operation of the HGHV-CORDIC module provided in the embodiments of this application. Detailed Implementation
[0021] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit it. In the following description, when referring to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with those of this application; they are merely examples of apparatuses and methods consistent with some aspects of the embodiments of this application as detailed in the appended claims.
[0022] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.
[0023] The Mamba architecture is a deep learning model designed for sequence modeling scenarios. Its core design integrates the characteristics of selective state-space models with the advantages of gating mechanisms. The core objective of this architecture is to significantly improve the efficiency and effectiveness of modeling when processing long sequence data, while keeping computational complexity at a linear level. This effectively addresses the pain points of traditional models, which suffer from excessively high computational complexity and low efficiency in long sequence tasks. As a foundational model with strong generalization capabilities, Mamba has shown broad application prospects in multiple interdisciplinary fields such as computer vision, natural language processing, and medicine. However, from the current state of technological research, dedicated accelerator optimization for the Mamba model is still in the initial exploratory stage, and there is still considerable room for expansion in related technical solutions and performance improvement research. In particular, the computational complexity of the nonlinear functions Sofplus and SiLU in Mamba is a major challenge in the design of Mamba circuits, making it difficult to achieve a good trade-off between accuracy, computational speed, and complexity. Many unified architecture design methods for nonlinear functions in next-generation large-scale Mamba models are designed with low complexity or high-speed computation as the goal, which is insufficient for scenarios requiring high accuracy. Once their designs move towards higher precision, whether using lookup tables, piecewise approximations, or Taylor expansions, the resulting hardware overhead will be unbearable.
[0024] In summary, the technical problems existing in the relevant technologies need to be improved.
[0025] In view of this, this application provides a Mamba nonlinear function operation circuit and its operation method. The Mamba nonlinear function operation circuit realizes a unified architecture for nonlinear functions in the large Mamba model, realizes the calculation of Sofplus and SiLU nonlinear functions in Mamba, meets the requirements of high-precision scenarios, and maintains high efficiency and low complexity of calculation.
[0026] Figure 1 This is an optional structural diagram of the Mamba nonlinear function operation circuit provided in the embodiments of this application. The circuit includes a Sofplus operation module and a SiLU operation module. The Sofplus operation module includes a first exponent calculation module and a first logarithm calculation module. The SiLU operation module includes a first exponent calculation module, a first logarithm calculation module, a second logarithm calculation module and a second exponent calculation module. The first exponent calculation module is connected to the first logarithm calculation module, the second logarithm calculation module is connected to the second exponent calculation module, and the first logarithm calculation module is connected between the second logarithm calculation module and the second exponent calculation module. The first exponent calculation module is used to respond to the Sofplus function calculation instruction, calculate the first exponent function in the Sofplus function according to the first input parameter, and obtain the first calculation result; The first logarithm calculation module is used to calculate the first logarithm function, which includes the first exponential function, in the Sofplus function based on the first calculation result, and obtain the intermediate Sofplus result; The first exponent calculation module is also used to respond to the SiLU function calculation instruction, calculate the second exponent function in the SiLU function according to the second input parameter, and obtain the second calculation result; The first logarithm calculation module is also used to calculate the second logarithm function, which includes the second exponential function, in the SiLU function based on the second calculation result, to obtain the third calculation result; The second logarithm calculation module is used to calculate the third logarithm function in the SiLU function based on the second input parameter, and obtain the fourth calculation result; The second index calculation module is used to perform index calculation based on the third and fourth calculation results to obtain the SiLU target result.
[0027] Specifically, the formulas for the nonlinear functions Sofplus and SiLU in Mamba are shown in equations (1) and (2): (1); (2); Since exponentiation and division with base e are not hardware-friendly mathematical operations, the original mathematical formulas (1) and (2) are transformed. For Sofplus, we have: (3); in, Represents the first exponential function, Represents the first logarithmic function, This is represented as the first input parameter.
[0028] For SiLU, when x > 0, then: (4); in, This represents the second exponential function. This represents the second logarithmic function. Represents the third logarithmic function, This is represented as the second input parameter; When x=0, then: (5); When x < 0, then: (6); in, This represents the second exponential function. This represents the second logarithmic function. This represents the third logarithmic function.
[0029] As shown in equations (3), (4), (5), and (6), the division and exponentiation operations in equations (1) and (2) are converted into a combination of base-2 logarithms and base-2 exponents. This transformation adapts to the hardware's efficient support for binary operations, reduces computational costs, and improves the efficiency of inference or training. Based on base-2 exponentiation and base-2 logarithmic operations, plus a small number of constant multiplications and additions / subtractions, the Mamba nonlinear function operation circuit can realize the calculation of these two nonlinear functions in Mamba. In addition, since the two mathematical transformations have common mathematical operations, such as base-2 exponentiation and logarithmic operations, by using the first exponentiation calculation module and the first logarithm calculation module as the common circuit of the Sofplus operation module and the SiLU operation module, the common part of the transformed Sofplus function and SiLU function is processed, and the exponent and logarithm calculation of the common part is completed, which helps to reduce the complexity of the circuit structure and operation.
[0030] In some embodiments, the circuit further includes a preprocessing module, a data selector, and a multiplier, wherein the preprocessing module is connected to the multiplier, the data selector is connected to the multiplier, and the multiplier is connected to the first exponent calculation module. The preprocessing module is used to transmit the first input parameter to the multiplier in response to the Sofplus function calculation instruction; The data selector is used to respond to the Sofplus function calculation instruction and output the first constant to the multiplier; The multiplier is used to multiply the first input parameter by the first constant to obtain the exponential part of the first exponential function.
[0031] Specifically, please refer to Figure 2Before the Sofplus and SiLU operation modules, the circuit also includes a preprocessing module, a data selector, and a multiplier. The preprocessing module receives sel and x, where x can be represented as either a first or second input parameter. The data format of x is INT16, including one sign bit, three integer bits, and twelve decimal bits. When sel=1, it indicates a SiLU function calculation instruction, outputting the second input parameter x to the multiplier; when sel=0, it indicates a Sofplus function calculation instruction, outputting the first input parameter x to the multiplier. The data selector receives the same sel control signal. When sel=0, it executes the Sofplus function calculation, according to equation (3). , the first constant The output is sent to the multiplier. The multiplier multiplies the first input parameter x with... Multiply to get That is, the exponential part of the first exponential function. Similarly, when sel=1, that is, when the SiLU function is executed, as shown in equations (4), (5) and (6), the multiplier can receive the second input parameter x and the second constant. Multiplying the two together gives the exponential part of the second exponential function. .
[0032] In some embodiments, the first exponent calculation module includes a separation module, an HGHR-CORDIC module, and a shift module; The separate modules are connected to the HGHR-CORDIC module and the shift module respectively, and the HGHR-CORDIC module is connected to the shift module; The splitting module is used to split the exponential part of the first exponential function to obtain the integer and decimal parts of the exponent; The HGHR-CORDIC module is used to calculate the first intermediate result by using the HGHR-CORDIC algorithm to calculate the exponential function with decimal exponents. The shift module is used to left-shift the first intermediate result with an integer displacement to obtain the first calculation result.
[0033] Specifically, please refer to Figure 2 and Figure 3 For the exponential part of an exponential function, such as , The separation module can separate the integer and decimal parts of the exponent. The exponent calculation of the integer part can be performed by binary shifting, while the exponent calculation of the decimal part is performed accurately by another algorithm. For example, for the input D of the exponent calculation, D is a fixed-point number, which is decomposed into an integer I and a decimal F, as shown in equation (7): (7); Then the exponent is calculated as 2. D It can be represented as: (8); Through the Shift left The bits can ultimately be calculated. .
[0034] In Sofplus's calculations, the Separation Module executes equation (7) to separate the integer part I1 and the fractional part F1 of the fixed-point number D1. The separated fractional part F1 is then input into the HGHR-CORDIC Module (HGHR-CORDIC module) designed based on the HGHR-CORDIC algorithm for 2... F1 The calculation yields the first intermediate result, I1 is input to the shift module (Shift), and Shift then operates on 2... F1 Shifting left by I1 bits yields 2. D1 That is, the first calculation result To ensure data alignment, the transmission of I1 is delayed through pipelined timing.
[0035] HGHR-CORDIC refers to the rotation mode used to calculate the exponent in HGH-CORDIC (High-radix Generalized Hyperbolic Coordinate Rotation Digital Computer).
[0036] In some embodiments, the first logarithmic calculation module includes a leader detector, an HGHV-CORDIC module, a first adder, and a second adder: The first exponent calculation module is connected to the first adder, the first adder is connected to the first preamble detector, the first preamble detector is connected to the HGHV-CORDIC module and the second adder respectively, and the HGHV-CORDIC module is connected to the second adder. The first adder is used to add the first calculation result to one to obtain the argument of the first logarithmic function; A leading detector is used to convert the true number to a floating-point format, obtaining the exponent and mantissa bits of the floating-point representation of the true number; The HGHV-CORDIC module is used to calculate the logarithmic function with the mantissa as the argument to obtain a second intermediate result; The second adder is used to add the second intermediate result and the exponent to obtain the Sofplus intermediate result.
[0037] Specifically, please refer to Figure 2 and Figure 3 In Sofplus's calculations, the first exponent calculation module calculates... The first intermediate result is obtained. The first adder adds one to the first intermediate result to obtain the result. (The argument of the first logarithmic function), for the input T, i.e. the argument, of the logarithmic function calculation, where T is non-negative, we can let (9); in, Equation (9) is the floating-point representation of T, where E represents the exponent and M represents the number of bits. This processing simplifies the logarithmic calculation. Therefore, the logarithmic calculation of T can be expressed as: (10); Simply perform a logarithmic calculation on the mantissa, and then add the exponent to the logarithmic result of the mantissa to obtain the final logarithmic result.
[0038] The leading one detector (LOD) receives T1 (i.e., The data format is fixed-point. T1 is converted to a floating-point number, yielding its floating-point exponent E1 and mantissa M1. The HGHV-CORDIC module, designed based on the HGHV-CORDIC algorithm, is used to calculate log2M1. Finally, log2M1 is added to E1 to obtain the Sofplus intermediate result. Both the HGHV-CORDIC Module and the HGHV-CORDIC Module convert the algorithm's iteration into a pipelined design. For data alignment, the transmission of E1 is also delayed through pipeline timing.
[0039] HGHV-CORDIC refers to the vector mode used to calculate the exponent in HGH-CORDIC (High-radix Generalized Hyperbolic Coordinate Rotation Digital Computer).
[0040] In some embodiments, the Mamba nonlinear function operation circuit further includes a multiplier connected to the first logarithm calculation module. According to equation (3), the multiplier is used to process the intermediate results of Sofplus. With constant Multiplying these results yields the final Sofplus target result.
[0041] In some embodiments, the second logarithmic calculation module includes a leader detector, an HGHV-CORDIC module, and an adder; The preprocessing module is connected to the first preprocessor detector, which is connected to the HGHV-CORDIC module and the adder, respectively. The HGHV-CORDIC module is connected to the adder. The preprocessing module is also used to transmit the second input parameter to the lead detector in response to the SiLU function calculation instruction; The leading detector is used to perform floating-point format conversion on the second input parameter to obtain the exponent and mantissa bits of the floating-point representation of the second input parameter; The HGHV-CORDIC module is used to calculate the logarithmic function with the mantissa as the argument and obtain a third intermediate result; The adder is used to add the third intermediate result and the exponent to obtain the fourth calculation result.
[0042] Specifically, please refer to Figure 3 and Figure 4 According to equations (4), (5), and (6), when sel=1, the SiLU function is calculated. The preprocessing module performs a sign judgment on the second input parameter x. If x is greater than zero, equation (4) is executed, and the positive number flag 2'b01 is used as the ctrl signal to be pipelined and passed to the postprocessing module. The calculation of the SiLU function is divided into two paths, in which the first exponential calculation module and the first logarithmic calculation module calculate... The second input parameter is multiplied by the constant log2(1 / e), then exponentially calculated, 1 is added, and then logarithmically calculated to obtain... This process is related to the above. The calculation process is similar. Alternatively, the preprocessing module transmits the second input parameter x to the second logarithm calculation module. The second logarithm calculation module has a similar structure and operation process to the first logarithm calculation module. The pre-detector of the second logarithm calculation module performs floating-point format conversion on the second input parameter x (T2) to obtain the exponent (E2) and mantissa (M2). The HGHV-CORDIC module then... 2. After performing the calculation, add it to E2 to obtain the result. That is, the fourth calculation result.
[0043] In some embodiments, the second index calculation module includes a separation module, an HGHR-CORDIC module, a shift module, and a post-processing module; The preprocessing module connects to the postprocessing module; the separation module connects to the HGHR-CORDIC module and the shift module respectively; the HGHR-CORDIC module connects to the shift module; and the shift module connects to the postprocessing module. The preprocessing module is also used to respond to the SiLU function calculation instruction, identify the positive or negative number of the second input parameter, and obtain the sign of positive, zero or negative number; The separation module is used to split the result of subtracting the fourth calculation result from the third calculation result, and obtain the integer and decimal parts of the subtraction result; The HGHR-CORDIC module is used to calculate the exponential function with decimal exponents using the HGHR-CORDIC algorithm, and obtain the fourth intermediate result; The shift module is used to shift the fourth intermediate result to the left by an integer number of bits to obtain the fifth intermediate result; The post-processing module is used to positive, zero, or negative the fifth intermediate result according to the flag symbol to obtain the SiLU target result.
[0044] Specifically, please refer to Figure 3 and Figure 4 The circuit also includes a subtractor and a pipeline module. The second logarithmic calculation module is connected to the pipeline module. The inputs of the pipeline module and the first exponential calculation module are connected to the subtractor. The pipeline module delays the third calculation result and aligns it with the fourth intermediate result. The subtractor subtracts the fourth calculation result from the third calculation result to obtain the subtracted result. The separation module of the second exponent calculation module separates the subtraction result into an integer I² and a decimal F². The HGHR-CORDIC module then calculates 2. F2 The shift module will 2 F2 After shifting left by I2 bits, we get Finally, based on the positive number flag, the SiLU target result is output, i.e. This completes the SiLU calculation when the second input parameter x is greater than zero.
[0045] If x equals zero, perform the calculation using formula (5), because the circuit cannot input 0 for calculation (e.g. The function requires inputs to be greater than zero; therefore, the data for the other two input paths must be consistent with the input data of the previous cycle. This consistency is meaningless; it's only to ensure the timing of the pipelined computation, guaranteeing the correct input-output sequence and reducing register flips to lower power consumption. During output, the meaningless output data is set to 0 based on the Ctrl signal indicating the zero flag, resulting in the correct output of 0.
[0046] If x is less than zero, the calculation of formula (6) is executed. In the preprocessing module, the negative number flag 2'b10 is used as the ctrl signal for pipeline step-by-step transmission. The input x is multiplied by the constant log2(1 / e) and the opposite number is taken. Then, the exponent is calculated, 1 is added, and then the logarithm is performed to obtain the result. Alternatively, take the opposite of x to get |x|, then perform a logarithmic operation to obtain... After a delay by the pipeline module, it is then compared with the calculation. Aligning all the lines and subtracting them yields the result. Next, the subtraction result is subjected to exponentiation. Since x is less than zero, the post-processing module receives the negative number flag 2'b10 ctrl signal, takes the inverse, and outputs the result of SiLU calculation. .
[0047] The logarithms and exponents mentioned above are calculated using the HGH-CORDIC algorithm. The HGH-CORDIC algorithm can calculate the exponent and logarithm of any integer base through multiple iterations; the higher the number of iterations, the higher the accuracy. Furthermore, compared to the traditional generalized hyperbolic CORDIC algorithm, the HGH-CORDIC algorithm iterates much faster, requiring only half the iteration time to achieve the same accuracy. Therefore, the HGH-CORDIC algorithm can achieve fast and high-precision calculations.
[0048] The rotation mode used to calculate the exponent in HGH-CORDIC is shown in the following iterative formula (with a base of 4): (11); in, , , .
[0049] Similarly, the iterative formula for the vector mode used to calculate the logarithm in HGH-CORDIC, HGHV-CORDIC (with a base of 4), is as follows: (12); iteration step size The method for determining the value is as follows: Rotation mode: (13); Vector mode: (14); For rotation mode, input ( , , Initialize to: (15); And the range of values for M is: (16); Therefore, the input for the exponent calculation in this embodiment is between [0, 1), satisfying the range of values for M. The output will then be: (17); When b=2, the calculation yields 2. M The scaling factor K is obtained by the following formula: (18); Among them, and .
[0050] Similarly, for vector patterns, if their initial input ( , , )for: (19); The range of values for M is ; The input for the logarithmic calculation in this embodiment is between (1, 2), which also satisfies the range of values for M.
[0051] The output will be: (20); This application embodiment uses a simple left shift in hardware to... Shift one position to the left to get the log. b M. By simply setting b=2, we can obtain 2 through multiple iterations of the radix-4 HGH-CORDIC. M The calculation results are shown in log2M. Compared with the traditional generalized hyperbolic CORDIC algorithm, the radix-4 high-radix generalized hyperbolic CORDIC algorithm can reduce the number of iterations by half while maintaining the same accuracy, significantly shortening the iteration time and achieving high-precision and high-efficiency computation.
[0052] Furthermore, the HGHR-CORDIC algorithm in equation (11) is optimized to reduce hardware resource overhead while maintaining computational accuracy. and By combining the results, we obtain equation (21). (twenty one); For the initial value, the input will be ( , Initialize to (twenty two); The output will be: (twenty three); The rest remains unchanged. After the original HGHR-CORDIC algorithm completes its iterations, the outputs of the X and Y branches are added together and then divided by K to obtain the exponent calculation result. The optimized algorithm, however, directly divides the final result by K after the P branch outputs the result to obtain the exponent calculation result. With the same number of iterations, the computational accuracy of the P branch of the improved HGHR-CORDIC algorithm is equivalent to the computational accuracy of the sum of the outputs of the X and Y branches of the original HGHR-CORDIC algorithm. This optimization of the exponent calculation module not only reduces the hardware overhead of one branch but also eliminates the need for an adder at the output stage. In scenarios involving large-scale data input, assuming the hardware is N times parallel, the Sofplus function has N exponent calculations, and the Silu function has 2N exponent calculations, then for a unified architecture hardware, with 2N parallel exponent calculation modules, each designed in a pipelined manner with I iterations, this optimization can save hardware costs equivalent to 4*N*I branches and 2N adders at the final output stage. Specifically, this reduces the number of 4*N*I 3-to-1 multiplexers, 4*N*I 2-to-1 multiplexers, and 4*N*I+2N adders. As N and I increase, the advantage of reduced hardware overhead becomes even more pronounced. Therefore, the optimized HGHR-CORDIC algorithm further reduces hardware costs while maintaining its high accuracy and efficiency.
[0053] Please refer to Figure 5 and Figure 6The HGHR-CORDIC and HGHV-CORDIC modules are designed as pipelined structures. After inputting an initial value, the final output is obtained after i iterations. The iteration number i also specifies the specific shift value for each branch across different iterations. The choice of different shifts within each branch is determined by the absolute value of di, whose absolute value and sign are calculated by the comparator, and the judgment criteria are shown in equations (13) and (14). The sign (±) of the output of di controls whether the adder performs addition or subtraction. The circuit design of the HGHR-CORDIC and HGHV-CORDIC modules follows a "rule of thumb" that "if the output precision requires n bits, the internal register should have log2n additional bits at the least significant bit," requiring 4 (log212≈4) additional bits to extend the 12-bit fractional part. After adding 4 redundant bits, the bit error rate of the first 12 bits is greatly reduced because the redundant bits can resist the propagation of errors. Therefore, the inputs of the HGHR-CORDIC and HGHV-CORDIC modules are both 16 fractional bits. This embodiment can remove the last 4 bits from the output of HGHR-CORDIC and HGHV-CORDIC, retaining 12 decimal places and achieving higher precision calculations. The 1 / K and W branches in the HGHR-CORDIC module and the Z branch in the HGHV-CORDIC module can be optimized through integrated LUTs, thereby simplifying hardware design and reducing hardware overhead.
[0054] In some embodiments, the HGHR-CORDIC module includes a multiplier, a P branch, a 1 / K branch, and a W branch; The W branch is used to search in a pre-defined lookup table based on the hyperbolic angle, determine the iteration step size, and calculate the hyperbolic angle of the next generation. The P branch is used to iteratively update the hyperbolic function value according to the iteration step size, and obtain the objective function value of the sum of hyperbolic sine and hyperbolic cosine. The 1 / K branch is used to iteratively update the proportional compensation factor according to the iteration step size to obtain the target compensation factor; The multiplier is used to multiply the objective function value with the objective compensation factor to obtain the output result of the exponential function, which includes the first intermediate result and the fourth intermediate result.
[0055] Specifically, please refer to Figure 5For the HGHR-CORDIC module, the W branch has a bit width of 21, including 1 sign bit, 4 integer bits, and 16 fractional bits, because LUT(R) requires 4 integer bits. The final output P / K is the result of the exponent calculation in the range [1, 2], represented as an unsigned fixed-point number with 1 integer bit and 16 fractional bits. If the final P / K has 16 fractional bits and meets the precision requirements, then 1 / K and P do not need too many fractional bits. According to MATLAB simulations, 1 / K is positive during the iteration process, remaining below 1.9. Therefore, 1 / K is represented as an unsigned fixed-point number with 1 integer bit and 13 fractional bits. The P branch has a bit width of 13, including 1 sign bit, 2 integer bits, and 10 fractional bits.
[0056] For the W branch in HGHR-CORDIC and the Z branch in HGHV-CORDIC, calculations are required. Since direct computation is complex, lookup tables (LUTs) are typically used. However, the required LUT size grows exponentially with the number of iterations, leading to significant hardware overhead. To address this issue, a Taylor series expansion is used to approximate the formula while minimizing accuracy loss, as shown in the following formula: (twenty four); When the decimal places are greater than or equal to 16 and b=2, for iterations of i≥3, It can be approximated as d i *4 -i / ln2. Therefore, for the HGHR-CORDIC module, the update formula for the W branch can be simplified to: (25); For the 1 / K branch in HGHR-CORDIC, the scaling factor K is not a constant, and d i The value of K changes with each iteration. Therefore, in each iteration... i -1 The numerical values may not be the same.
[0057] ; Where K1=1. We execute K in each iteration. i -1 The values are multiplied together, and then the final output is scaled using the product. However, calculating K is relatively complex, making a direct hardware implementation resource-intensive. To address this issue, a method is used... express Taylor series expansion: (26); It can be observed that as the iteration exponent i increases, ui The value gradually approaches 1. The values of u1 to u4 are shown in Table 2. Based on 1 / K3 = 1 / (K1*u1*u2), the value of 1 / K3 is stored in the lookup table LUT(1 / K), as shown in Table 3. For subsequent calculations, 1 / K4 = 1 / (K3*u3) and 1 / K5 = 1 / (K4*u4), multiplication can be eliminated through shifting and addition operations. Starting from 1 / K5, u... i The value of remains almost constant at 1, which means 1 / K i It also remains unchanged. At this point, P is aligned by pipeline delay, and the result of the P branch calculation is output in the same cycle. Finally, 1 / K and P are multiplied to obtain the required exponential calculation result.
[0058] In some embodiments, please refer to Figure 6 For the HGHV-CORDIC module, the X and W branches are 20 bits, consisting of 1 sign bit, 3 integer bits, and 16 decimal bits. The Z branch outputs the logarithmic calculation result within [0, 1], therefore it is an unsigned 16-bit fixed-point number, with all 16 bits representing decimals. The P branch of HGHV-CORDIC and the X and W branches of HGHV-CORDIC directly correspond to their respective algorithmic mathematical formulas. The Z branch in the HGHV-CORDIC module can be optimized by integrating a LUT. According to equation (24), the update formula for the Z branch in the HGHV-CORDIC module is simplified to: (27); The values of |d in equations (25) and (27) i The result of | / ln2 is 1.44269504 or 2.88539008. The binary representation of 1.44269504 is 1.0111000101010010b. Other constants such as 2.88539008 can be derived by performing a one-bit left shift. When i is less than 3, it's like 4... i+1 tanh -1 2(d) i 4 -i ) or tanh -1 2(d) i 4 -i Such terms are relatively complex and can be pre-computed and retrieved using a LUT. In d i If the constants are not zero, only a small number of constants need to be stored in the LUT, and these constants are all greater than 0. These constants are listed in Table 1.
[0059] Table 1 shows the pre-computed data stored by LUT(R) and LUT(V) for different iterations.
[0060] Table 2 u i binary value
[0061] Table 3 Pre-computed data stored in LUT (1 / K)
[0062] In some embodiments, please refer to Figure 3 The Mamba nonlinear function operation circuit also includes a data selector. The input of the data selector is connected to the output of the Sofplus operation module and the SiLU operation module, respectively. The data selector can select to output either the Sofplus target result or the SiLU target result based on the sel signal. This application also provides an optional flowchart of a Mamba nonlinear function operation method, which may include, but is not limited to, steps S101 to S107.
[0063] Step S101: Convert the Sofplus function in Mamba into a first combined function including a first exponential function and a first logarithmic function, and convert the SiLU function in Mamba into a second combined function including a second exponential function, a second logarithmic function and a third logarithmic function; Step S102: In response to the Sofplus function calculation instruction, the first exponential function is calculated according to the first input parameters to obtain the first calculation result; Step S103: Calculate the first logarithmic function, which includes the first exponential function, based on the first calculation result to obtain the Sofplus intermediate result; Step S104: In response to the SiLU function calculation instruction, the second exponential function is calculated according to the second input parameters to obtain the second calculation result; Step S105: Calculate the second logarithmic function, which includes the second exponential function, based on the second calculation result to obtain the third calculation result; Step S106: Calculate the third logarithmic function based on the second input parameters to obtain the fourth calculation result; Step S107: Perform exponential calculation based on the third and fourth calculation results to obtain the SiLU target result.
[0064] It is understood that the contents of the above circuit embodiments are all applicable to the present method embodiments. The specific functions implemented in the present method embodiments are the same as those in the above circuit embodiments, and the beneficial effects achieved are also the same as those achieved in the above circuit embodiments.
[0065] The circuit and method proposed in this embodiment optimize the HGH-CORDIC (High-radix Generalized Hyperbolic Coordinate Rotation Digital Computer) algorithm, which can efficiently achieve high-precision exponential and logarithmic calculations, further reducing hardware complexity. Based on this optimized algorithm, a unified architecture is designed to realize a high-precision, high-efficiency, and low-complexity nonlinear function hardware. This solution can meet the application scenarios requiring high precision by increasing the number of iterations, and the pipelined architecture can reduce output latency, allowing the hardware to output in each cycle. Furthermore, hardware and software co-optimization reduces hardware overhead, enabling the calculation of various functions. This embodiment can be used for high-precision scenarios involving large Mamba models, such as financial computing and scientific computing, where these scenarios have stringent requirements for computational efficiency and hardware resources.
[0066] This application also provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the above-described method. This electronic device can be any smart terminal, including tablet computers, in-vehicle computers, etc.
[0067] It is understood that the content of the above method embodiments is applicable to this device embodiment. The specific functions implemented by this device embodiment are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.
[0068] To achieve the above objectives, another aspect of this application provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the method described above.
[0069] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method.
[0070] It is understood that the content of the above method embodiments is applicable to this storage medium embodiment. The specific functions implemented in this storage medium embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.
[0071] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.
[0072] It is understood that the content of the above method embodiments is applicable to the embodiments of this program product. The specific functions implemented by the embodiments of this program product are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.
[0073] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.
[0074] The Mamba nonlinear function operation circuit, operation method, device, electronic device, storage medium, and program product provided in this application have at least the following beneficial effects: 1) By deriving the mathematical formulas of the Mamba nonlinear functions Sofplus and SiLU, the bottlenecks in the implementation of exponential and logarithmic calculations were identified, and the HGH-CORDIC algorithm, which can achieve exponential and logarithmic calculations with high accuracy and efficiency, was combined for calculation implementation.
[0075] 2) Based on 1), since the mathematical formulas have common parts, a unified architecture can be designed to realize the calculation of two nonlinear functions, Sofplus and SiLU.
[0076] 3) Based on 1), optimize the HGHR-CORDIC algorithm in the existing HGH-CORDIC algorithm to achieve low-complexity computation while maintaining high accuracy and high efficiency.
[0077] 4) Based on 1), the nonlinear function calculation is mapped to a circuit through software and hardware co-optimization, realizing the hardware design of a unified nonlinear function calculation architecture, which has the characteristics of high precision, high efficiency and complex calculation.
[0078] In summary, this embodiment designs a unified architecture for nonlinear functions that is high-precision, high-efficiency, and low-complexity for the next-generation large model Mamba, in order to meet the requirements of high-precision scenarios while maintaining high efficiency and low complexity.
[0079] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.
[0080] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this application, and may include more or fewer steps than shown, or combine certain steps, or different steps.
[0081] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.
[0082] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.
[0083] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.
[0084] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.
[0085] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.
[0086] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.
[0087] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.
[0088] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0089] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.
Claims
1. A Mamba nonlinear function operation circuit, characterized in that, The circuit includes a Sofplus operation module and a SiLU operation module. The Sofplus operation module includes a first exponent calculation module and a first logarithm calculation module. The SiLU operation module includes a first exponent calculation module, a first logarithm calculation module, a second logarithm calculation module, and a second exponent calculation module. The first exponent calculation module is connected to the first logarithm calculation module, the second logarithm calculation module is connected to the second exponent calculation module, and the first logarithm calculation module is connected between the second logarithm calculation module and the second exponent calculation module; The first exponent calculation module is used to respond to the Sofplus function calculation instruction, calculate the first exponent function in the Sofplus function according to the first input parameter, and obtain the first calculation result; The first logarithm calculation module is used to calculate the first logarithm function, which includes the first exponential function, in the Sofplus function based on the first calculation result, to obtain the Sofplus intermediate result; The first exponent calculation module is also used to respond to the SiLU function calculation instruction, calculate the second exponent function in the SiLU function according to the second input parameter, and obtain the second calculation result; The first logarithm calculation module is further configured to calculate the second logarithm function, which includes the second exponential function, in the SiLU function based on the second calculation result, to obtain a third calculation result; The second logarithm calculation module is used to calculate the third logarithm function in the SiLU function according to the second input parameters, and obtain the fourth calculation result; The second index calculation module is used to perform index calculation based on the third and fourth calculation results to obtain the SiLU target result.
2. The circuit according to claim 1, characterized in that, The circuit also includes a preprocessing module, a data selector, and a multiplier. The preprocessing module is connected to the multiplier, the data selector is connected to the multiplier, and the multiplier is connected to the first exponent calculation module. The preprocessing module is used to transmit the first input parameter to the multiplier in response to the Sofplus function calculation instruction; The data selector is used to output a first constant to the multiplier in response to the Sofplus function calculation instruction; The multiplier is used to multiply the first input parameter by the first constant to obtain the exponential part of the first exponential function.
3. The circuit according to claim 1, characterized in that, The first index calculation module includes a separation module, an HGHR-CORDIC module, and a shift module; The separation module is connected to the HGHR-CORDIC module and the shift module respectively, and the HGHR-CORDIC module is connected to the shift module; The separation module is used to split the exponential part of the first exponential function to obtain the integer and decimal parts of the exponent; The HGHR-CORDIC module is used to calculate the exponential function with the decimal as the exponent using the HGHR-CORDIC algorithm to obtain the first intermediate result; The shift module is used to left-shift the first intermediate result using the integer as the displacement number to obtain the first calculation result.
4. The circuit according to claim 1, characterized in that, The first logarithmic calculation module includes a leader detector, an HGHV-CORDIC module, a first adder, and a second adder. The first exponent calculation module is connected to the first adder, the first adder is connected to the first leading detector, the first leading detector is connected to the HGHV-CORDIC module and the second adder respectively, and the HGHV-CORDIC module is connected to the second adder; The first adder is used to add the first calculation result to one to obtain the argument of the first logarithmic function; The leading detector is used to perform floating-point format conversion on the true number to obtain the exponent and mantissa bits of the floating-point representation of the true number; The HGHV-CORDIC module is used to calculate the logarithmic function with the mantissa as the argument to obtain a second intermediate result; The second adder is used to add the second intermediate result and the exponent to obtain the Sofplus intermediate result.
5. The circuit according to claim 2, characterized in that, The second logarithmic calculation module includes a leader detector, an HGHV-CORDIC module, and an adder; The preprocessing module is connected to the first preprocessor detector, the first preprocessor detector is connected to the HGHV-CORDIC module and the adder respectively, and the HGHV-CORDIC module is connected to the adder; The preprocessing module is also used to transmit the second input parameter to the leading detector in response to the SiLU function calculation instruction; The leading detector is used to perform floating-point number format conversion on the second input parameter to obtain the exponent and mantissa bits of the floating-point representation of the second input parameter; The HGHV-CORDIC module is used to calculate the logarithmic function with the mantissa as the argument to obtain a third intermediate result; The adder is used to add the third intermediate result and the exponent to obtain the fourth calculation result.
6. The circuit according to claim 2, characterized in that, The second index calculation module includes a separation module, an HGHR-CORDIC module, a shift module, and a post-processing module; The preprocessing module is connected to the postprocessing module, the separation module is connected to both the HGHR-CORDIC module and the shifting module, the HGHR-CORDIC module is connected to the shifting module, and the shifting module is connected to the postprocessing module. The preprocessing module is also used to respond to the SiLU function calculation instruction by identifying the positive or negative number of the second input parameter to obtain a sign for a positive, zero, or negative number. The separation module is used to split the subtraction result of the fourth calculation result and the third calculation result to obtain the integer and decimal parts of the subtraction result; The HGHR-CORDIC module is used to calculate the exponential function with the decimal as the exponent using the HGHR-CORDIC algorithm to obtain the fourth intermediate result; The shift module is used to shift the fourth intermediate result to the left by an integer number of positions, using the integer as the displacement, to obtain the fifth intermediate result; The post-processing module is used to positive, zero, or negative the fifth intermediate result according to the flag symbol to obtain the SiLU target result.
7. The circuit according to any one of claims 1 to 6, characterized in that, The HGHR-CORDIC module includes a multiplier, a P branch, a 1 / K branch, and a W branch; The W branch is used to search in a preset lookup table based on the hyperbolic angle, determine the iteration step size, and calculate the hyperbolic angle of the next generation; The P branch is used to iteratively update the hyperbolic function value according to the iteration step size, so as to obtain the objective function value of the sum of hyperbolic sine and hyperbolic cosine. The 1 / K branch is used to iteratively update the proportional compensation factor according to the iteration step size to obtain the target compensation factor. The multiplier is used to multiply the objective function value with the objective compensation factor to obtain the output result of the exponential function, which includes the first intermediate result and the fourth intermediate result.
8. A method for calculating Mamba nonlinear functions, characterized in that, The method is applied to the Mamba nonlinear function operation circuit as described in any one of claims 1 to 7, and the method includes the following steps: The Sofplus function in Mamba is converted into a first combined function comprising a first exponential function and a first logarithmic function, and the SiLU function in Mamba is converted into a second combined function comprising a second exponential function, a second logarithmic function, and a third logarithmic function; In response to the Sofplus function calculation instruction, the first exponential function is calculated based on the first input parameters to obtain the first calculation result; The first logarithmic function, which includes the first exponential function, is calculated based on the first calculation result to obtain the Sofplus intermediate result; In response to the SiLU function calculation instruction, the second exponential function is calculated according to the second input parameters to obtain the second calculation result; The second logarithmic function, which includes the second exponential function, is calculated based on the second calculation result to obtain the third calculation result; The third logarithmic function is calculated based on the second input parameters to obtain a fourth calculation result; The SiLU target result is obtained by performing an exponential calculation based on the third and fourth calculation results.
9. The method according to claim 8, characterized in that, The first combined function, which includes the first exponential function and the first logarithmic function, is expressed as follows: ； in, This represents the first exponential function. Represents the first logarithmic function. This is represented as the first input parameter.
10. The method according to claim 8, characterized in that, The second combined function, which includes the second exponential function, the second logarithmic function, and the third logarithmic function, is expressed as follows: When the second input parameter is greater than zero, the second combination function is expressed as: ； in, This represents the second exponential function. This represents the second logarithmic function. This represents the third logarithmic function. This is represented as the second input parameter; When the second input parameter is equal to zero, the second combination function is represented as zero; When the second input parameter is less than zero, the second combination function is expressed as: ； in, This represents the second exponential function. This represents the second logarithmic function. This represents the third logarithmic function.