Computing circuits, methods, apparatuses, chips, and electronic devices

Abstract

This application discloses a computing circuit, method, apparatus, chip, and electronic device, relating to the field of data processing technology. The specific technical solution is as follows: The computing circuit includes: an input processing unit, used to determine the absolute value of the input independent variable and determine a mapped independent variable based on the absolute value, wherein the mapped independent variable belongs to a target interval with a single symbol; a piecewise linear approximation unit, connected to the input processing unit, used to output a mapped piecewise linear approximation value based on the piecewise linear parameter set corresponding to the mapped independent variable and the target activation function, wherein the domain configured for the piecewise linear parameter set is the target interval; and a reconstruction compensation unit, connected to the piecewise linear approximation unit, used to reconstruct and compensate the mapped piecewise linear approximation value based on the reconstruction compensation configuration parameter set corresponding to the target activation function, and output the function value of the target activation function under the input independent variable.

Description

Technical Field

[0001] This application belongs to the field of data processing technology, specifically relating to a computing circuit, method, apparatus, chip, and electronic device. Background Technology

[0002] Currently, piecewise linear approximation (PWL) can discretize the activation function into several linear segments, with each segment corresponding to a slope and an intercept. In related technologies, the piecewise linear computation unit needs to segment the entire domain of the activation function (i.e., including both the positive and negative semi-axes) to obtain a piecewise linear parameter set (i.e., each linear segment and its corresponding slope and intercept information). This piecewise linear parameter set is then stored in the chip, allowing the piecewise linear computation unit to load the piecewise linear parameter set of any activation function and switch between approximators for different activation functions.

[0003] However, due to the problem that the piecewise linear computing unit stores too many parameters in the above technology, the size of the piecewise linear computing unit is large, which in turn leads to a large chip size and affects chip deployment. Summary of the Invention

[0004] The purpose of this application is to provide a computing circuit, method, apparatus, chip, and electronic device that can solve the problem of large chip size affecting chip deployment.

[0005] In a first aspect, embodiments of this application provide a computational circuit comprising: an input processing unit, configured to determine the absolute value of an input independent variable and determine a mapped independent variable based on the absolute value of the input independent variable, the mapped independent variable belonging to a target interval with a single symbol; a piecewise linear approximation unit, connected to the input processing unit, configured to output a mapped piecewise linear approximation value based on a piecewise linear parameter set corresponding to the mapped independent variable and the target activation function, the domain configured for the piecewise linear parameter set being the target interval; and a reconstruction compensation unit, connected to the piecewise linear approximation unit, configured to reconstruct and compensate the mapped piecewise linear approximation value based on a reconstruction compensation configuration parameter set corresponding to the target activation function, and output the function value of the target activation function under the input independent variable.

[0006] Secondly, embodiments of this application provide a chip that includes the computing circuitry described in the first aspect.

[0007] Thirdly, embodiments of this application provide a calculation method applied to a calculation circuit as described in the first aspect. The method includes: determining the absolute value of an input independent variable through an input processing unit of the calculation circuit, and determining a mapping independent variable based on the absolute value of the input independent variable, wherein the mapping independent variable belongs to a target interval with a single symbol; outputting a mapped piecewise linear approximation value based on a piecewise linear parameter set corresponding to the mapped independent variable and the target activation function through a piecewise linear approximation unit of the calculation circuit, wherein the domain configured for the piecewise linear parameter set is the target interval; and reconstructing and compensating the mapped piecewise linear approximation value based on a reconstruction compensation configuration parameter set corresponding to the target activation function through a reconstruction compensation unit of the calculation circuit, and outputting the function value of the target activation function under the input independent variable.

[0008] Fourthly, embodiments of this application provide a computing device comprising: a processing module, configured to determine the absolute value of an input independent variable, and determine a mapping independent variable based on the absolute value of the input independent variable, wherein the mapping independent variable belongs to a target interval with a single symbol; output a mapped piecewise linear approximation value based on a piecewise linear parameter set corresponding to the mapped independent variable and the target activation function, wherein the domain configured for the piecewise linear parameter set is the target interval; and reconstruct and compensate the mapped piecewise linear approximation value based on a reconstruction compensation configuration parameter set corresponding to the target activation function, and output the function value of the target activation function under the input independent variable.

[0009] Fifthly, embodiments of this application provide an electronic device that includes a computing circuit as described in the first aspect, or a chip as described in the second aspect.

[0010] In a sixth aspect, embodiments of this application provide a readable storage medium on which a program or instructions are stored, which, when executed by a processor, implement the steps of the method as described in the third aspect.

[0011] In a seventh aspect, embodiments of this application provide a computer program product stored in a storage medium, which is executed by at least one processor to implement the steps of the method as described in the third aspect.

[0012] Eighthly, embodiments of this application provide a chip including a processor and a communication interface coupled to the processor, the processor being used to run programs or instructions to implement the steps of the method as described in the third aspect.

[0013] In this embodiment, the computational circuit may include an input processing unit, a piecewise linear approximation unit, and a reconstruction compensation unit. The piecewise linear approximation unit is connected to the input processing unit, and the reconstruction compensation unit is connected to the piecewise linear approximation unit. The input processing unit is used to determine the absolute value of the input independent variable and determine the mapped independent variable based on the absolute value of the independent variable. The mapped independent variable belongs to a target interval with a single symbol. The piecewise linear approximation unit is used to output a mapped piecewise linear approximation value based on the piecewise linear parameter set corresponding to the mapped independent variable and the target activation function. The domain configured by the piecewise linear parameter set is the target interval. The reconstruction compensation unit is used to reconstruct and compensate the mapped piecewise linear approximation value based on the reconstruction compensation configuration parameter set corresponding to the target activation function and output the function value of the target activation function under the input independent variable. Because the input processing unit can determine the absolute value of the input independent variable and determine the mapping independent variable based on the absolute value, thus mapping the input independent variable belonging to a region of arbitrary signs to a target interval belonging to a single sign, instead of directly using the input independent variable belonging to an interval including both positive and negative signs, the piecewise linear approximation unit can output a mapped piecewise linear approximation value based on the mapped independent variable and the piecewise linear parameter set corresponding to the target activation function (the domain of which is configured is the target interval of a single sign). However, when the configured domain is the target interval of a single sign, the number of parameters in the piecewise linear parameter set is less than when the configured domain includes both positive and negative signs. The number of parameters in the linear parameter set means that the piecewise linear approximation unit can output a mapped piecewise linear approximation value based on the mapped independent variable and a piecewise linear parameter set with a smaller number of parameters. Moreover, the reconstruction compensation unit can reconstruct and compensate the mapped piecewise linear approximation value based on the reconstruction compensation configuration parameter set corresponding to the target activation function, and output the function value of the target activation function under the input independent variable. That is, the reconstruction compensation unit can output accurate function values. Therefore, while ensuring the accuracy of the output function value, the number of parameters in the piecewise linear parameter set used for calculation can be reduced, thereby reducing the number of parameters stored in the computing circuit, reducing the size of the computing circuit, and further reducing the size of the chip containing the computing circuit, thus facilitating chip deployment. Attached Figure Description

[0014] Figure 1 This is one of the structural schematic diagrams of the computing circuit provided in the embodiments of this application;

[0015] Figure 2 This is a schematic diagram of the structure of the piecewise linear approximation unit of the computing circuit provided in the embodiments of this application;

[0016] Figure 3 This is a schematic diagram of the reconfiguration compensation unit of the computing circuit provided in the embodiments of this application;

[0017] Figure 4 This is a second schematic diagram of the computing circuit provided in the embodiments of this application;

[0018] Figure 5 This is a schematic diagram of the chip structure provided in the embodiments of this application;

[0019] Figure 6 This is one of the flowcharts illustrating the calculation method provided in the embodiments of this application;

[0020] Figure 7 This is a second schematic flowchart of the calculation method provided in the embodiments of this application;

[0021] Figure 8 This is a schematic diagram of the structure of the computing device provided in the embodiments of this application;

[0022] Figure 9 This is one of the hardware structure diagrams of the electronic device provided in the embodiments of this application;

[0023] Figure 10 This is the second schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0024] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0025] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such terms can be used interchangeably where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0026] The terms "at least one," "at least one of," etc., used in the specification and claims of this application refer to any one, any two, or a combination of two or more of the included items. For example, at least one of a, b, and c can mean: "a," "b," "c," "a and b," "a and c," "b and c," and "a, b, and c," where a, b, and c can be single or multiple. Similarly, "at least two" refers to two or more items, and its meaning is similar to that of "at least one."

[0027] In deep neural networks, activation functions are core operators that introduce nonlinearity and enhance the expressive power of the model. In mainstream network architectures such as Transformer, activation functions (such as GELU, SiLU, Sigmoid, Tanh, Softplus, etc.) appear before and after the encoder, decoder, and normalization layer, and are key components of the non-multiply-accumulate path in the model's inference computation graph.

[0028] With the increasing scale of large models and the growing demand for edge deployment, achieving high throughput in neural network accelerators under power and area constraints has become one of the core challenges in hardware design. The quality of the hardware implementation of activation functions directly affects inference latency and energy efficiency. Traditional neural network accelerators typically configure dedicated arithmetic circuits for each type of activation function: for operators based on exponential functions such as Sigmoid and Tanh, dedicated exponentiation calculation units or multi-order Taylor expansion circuits are required; for operators containing normally distributed cumulative functions or Sigmoid products such as GELU and SiLU, multiple nonlinear sub-modules need to be further combined; for operators containing logarithmic operations such as Softplus, dedicated logarithmic calculation circuits are required. This kind of dedicated hardware scales linearly with the number of supported operators, and the hardware area and power consumption overhead increases accordingly.

[0029] Piecewise Linear (PWL) approximation, as a mainstream hardware-friendly alternative, discretizes the activation function into several linear segments, approximating the function value within each segment using a linear expression (slope and intercept). Since PWL approximation only requires a few breakpoint comparisons, a few shifts and additions, the hardware implementation relies solely on comparators, lookup tables, and simple arithmetic units, avoiding highly complex circuitry for exponentiation, division, and square root operations. Building on this, the industry has further proposed reconfigurable PWL computation units: by storing the breakpoints, slopes, and intercepts of each segment in configurable memory, a single piece of hardware can switch to approximators for different activation functions at runtime by loading different parameter sets, allowing a single hardware unit to support multiple activation functions.

[0030] However, the above solution has the following drawbacks:

[0031] The resource overhead of the aforementioned traditional neural network accelerator schemes is directly proportional to the number of operators. Constructing dedicated arithmetic circuits (including exponential, logarithmic, square root, division, etc.) for each type of activation function leads to a linear increase in hardware area and power consumption with the number of supported operators; the redundancy of dedicated circuits is particularly significant in accelerators that need to support multiple activation functions.

[0032] In the PWL scheme described above, the computational circuitry will perform calculations for each activation function within its complete domain. The independent configuration of piecewise linear parameter sets (i.e., linear segments, slope information, and intercept information) has the following consequences: the piecewise linear parameter sets corresponding to each activation function are independent of each other, and the storage size of the piecewise linear computation unit increases with the number of supported operators, resulting in an excessive amount of parameters stored in the piecewise linear computation unit, which leads to a large size of the piecewise linear computation unit and thus a large chip size, affecting chip deployment.

[0033] The computing circuits, methods, devices, chips, and electronic devices provided in this application will be described in detail below with reference to the accompanying drawings and through specific embodiments and application scenarios.

[0034] Figure 1 A schematic diagram of the computing circuit provided in an embodiment of this application is shown. Figure 1 As shown, the computational circuit provided in this application embodiment includes: an input processing unit, used to determine the absolute value of the input independent variable and determine the mapping independent variable based on the absolute value of the independent variable, wherein the mapping independent variable belongs to a target interval with a single symbol; a piecewise linear approximation unit, connected to the input processing unit, used to output a piecewise linear approximation value based on the piecewise linear parameter set corresponding to the mapping independent variable and the target activation function, wherein the domain configured for the piecewise linear parameter set is the target interval; and a reconstruction compensation unit, connected to the piecewise linear approximation unit, used to reconstruct and compensate the piecewise linear approximation value based on the reconstruction compensation configuration parameter set corresponding to the target activation function, and output the function value of the target activation function under the input independent variable.

[0035] In some embodiments of this application, the aforementioned input independent variable can be an independent variable input from other circuits. The input terminal of the input processing unit can be connected to these other circuits and receive the input independent variable from them, thereby enabling the input processing unit to determine the absolute value of the input independent variable.

[0036] It is understandable that, since the input independent variable may belong to a range with positive or negative signs, the absolute value of the input independent variable can be determined first to remove the sign of the input independent variable, which makes it easier to determine the mapping independent variable that belongs to the target range with a single sign later.

[0037] In some embodiments of this application, the aforementioned single symbol may include, but is not limited to, any of the following: a positive symbol or a negative symbol.

[0038] For example, if the single sign is positive, the target interval can be [0, n), where n is any real number greater than 0. If the single sign is negative, the target interval can be (m, 0], where m is any real number less than 0.

[0039] In some embodiments of this application, when the single sign is positive, the input processing unit can directly determine the absolute value of the independent variable as the mapped independent variable. When the single sign is negative, the input processing unit can invert the absolute value of the independent variable to obtain the mapped independent variable.

[0040] In some embodiments of this application, the target activation function may include, but is not limited to, at least one of the following: GELU, SiLU, Sigmoid, Tanh, Softplus, etc.

[0041] In some embodiments of this application, the aforementioned piecewise linear parameter set may include at least one set of parameters, each set of parameters including piecewise linear interval information, slope information, and intercept information. The piecewise linear interval information is used to indicate the segmented sub-intervals within the target interval.

[0042] In some examples, the above-mentioned piecewise linear interval information may include breakpoint information. It can be understood that the piecewise linear interval information in the above-mentioned at least one set of parameters may be the information of at least one breakpoint. Two adjacent breakpoints in the at least one breakpoint may form a linear segment to form at least one linear segment. The at least one breakpoint may divide the target interval into at least one piecewise sub-interval, and each piecewise sub-interval corresponds to a linear segment.

[0043] Among them, the slope information corresponding to the linear segment with the smallest coordinate value of the corresponding breakpoint in at least one of the above linear segments is 0, so that at least one linear segment approximately takes a horizontal asymptote at the far end.

[0044] In some embodiments of this application, the computing circuit can first determine the target activation function and provide the piecewise linear parameter set corresponding to the target activation function to the piecewise linear approximation unit. In this way, the piecewise linear approximation unit can output the mapped piecewise linear approximation value according to the mapping independent variable and the piecewise linear parameter set corresponding to the target activation function.

[0045] In some embodiments of this application, the piecewise linear approximation unit described above can be the only physical module in the computing circuit that undertakes nonlinear approximation calculations.

[0046] In some embodiments of this application, the aforementioned piecewise linear parameter set includes at least one set of parameters, each set of parameters including piecewise linear interval information, slope information, and intercept information. The piecewise linear interval information is used to indicate the segmented sub-intervals within the target interval. Combined with... Figure 1 ,like Figure 2 As shown, the piecewise linear approximation unit includes: a breakpoint comparator array connected to the input processing unit, used to determine the target piecewise sub-interval where the mapping independent variable is located from the piecewise sub-intervals indicated by the piecewise linear interval information in at least one set of parameters, and output the sub-interval identifier of the target piecewise sub-interval; a parameter selector connected to the breakpoint comparator array, used to determine the target slope information and target intercept information corresponding to the target piecewise sub-interval from the slope information and intercept information in at least one set of parameters according to the sub-interval identifier, and output the target slope information and target intercept information; and a linear synthesizer connected to the input processing unit and the parameter selector respectively, used to calculate the product value of the mapping independent variable and the target slope information, add the product value and the target intercept information, and output the result as the mapped piecewise linear approximation value.

[0047] In some embodiments of this application, the above-mentioned sub-range identifier may include, but is not limited to, at least one of the following: name, ID, index.

[0048] In some embodiments of this application, the breakpoint comparator array described above may include at least one comparator, each comparator corresponding to a segmented sub-interval, and each comparator is used to determine whether the mapping independent variable belongs to the segmented sub-interval corresponding to each comparator, so as to determine the target segmented sub-interval where the mapping independent variable is located from the segmented sub-interval indicated by the segmented linear interval information in at least one set of parameters.

[0049] In some embodiments of this application, the breakpoint comparator array includes at least two endpoint comparators, which can be connected in parallel. This allows multiple mapping variables to be judged in parallel by at least two breakpoint comparators when the number of input independent variables is at least two, i.e. when the number of mapped independent variables is at least two.

[0050] Combination Figure 2 The breakpoint comparator array described above can be connected to the register group in the following embodiments, so that the breakpoint comparator array can obtain at least one set of parameters from the register group.

[0051] In this embodiment of the application, the target slope information and target intercept information mentioned above are the same set of parameters as the piecewise linear interval information indicating the segmented sub-intervals of the target.

[0052] In some embodiments of this application, the parameter selector may include, but is not limited to, at least one of the following: multiplexer (MUX), look-up table (LUT), etc.

[0053] In some embodiments of this application, the parameter selector may be configured with at least one correspondence, each correspondence being a correspondence between an identifier, a slope information, and an intercept information. Thus, the parameter selector can determine an identifier that is the same as the sub-interval identifier from at least one identifier in at least one correspondence, and determine the slope information and intercept information corresponding to that identifier as the target slope information and the target intercept information.

[0054] Combination Figure 2 The parameter selector described above can be connected to the register group in the following embodiments, so that the parameter selector can obtain at least one set of parameters from the register group.

[0055] In some embodiments of this application, combined with Figure 2 The linear synthesizer described above includes a multiplier and an adder connected together. The multiplier can be used to calculate the product of the mapping independent variable and the target slope information, and the adder can be used to calculate the sum of the product and the target intercept information, so as to obtain and output the piecewise linear approximation of the mapping.

[0056] It is understandable that the linear synthesizer can obtain and output a piecewise linear approximation of the mapping using the first algorithm, which is as follows:

[0057] ;

[0058] in, For the piecewise linear approximation of the mapping, For target slope information, For target intercept information, The independent variable is the mapping variable. It should be noted that the above formula uses the single symbol as the negative sign for illustration.

[0059] Thus, it can be seen that since the piecewise linear approximation unit can include a breakpoint comparator array, a parameter selector, and a linear synthesizer, the breakpoint comparator array can determine the target piecewise sub-interval where the mapping independent variable is located, the parameter selector can determine the target slope information and target intercept information corresponding to the target piecewise sub-interval based on the sub-interval identifier, and the linear synthesizer can calculate and output the mapped piecewise linear approximation value. In other words, the mapped piecewise linear approximation value can be calculated and output using only the breakpoint comparator array, the parameter selector, and the linear synthesizer, without setting up other modules. Therefore, the size of the piecewise linear approximation unit can be reduced.

[0060] In some embodiments of this application, the set of reconstruction compensation configuration parameters corresponding to the above-mentioned target activation function includes symbol configuration bits, compensation configuration bits, and compensation parameters. Combined with... Figure 1 ,like Figure 3 As shown, the reconstruction compensation unit includes: a conditional inverter connected to the piecewise linear approximation unit; a ReLU compensator connected to the conditional inverter; and a first adder connected to the ReLU compensator. The conditional inverter is used to reconstruct the mapped piecewise linear approximation value according to the sign configuration bit, outputting a first intermediate result. The ReLU compensator is used to compensate the first intermediate result according to the compensation configuration bit, outputting a second intermediate result. The first adder is used to add the compensation parameter and the second intermediate result, and output the result as the function value.

[0061] In some examples, the above-mentioned reconstruction may include, but is not limited to, symbol processing, which may include, but is not limited to, at least one of the following: symbol preservation, symbol inversion.

[0062] Here, "sign preservation" can be understood as not processing the sign of the piecewise linear approximation value, and directly outputting the piecewise linear approximation value as the first intermediate result. "Sign inversion" can be understood as determining the sign of the piecewise linear approximation value to be the opposite sign.

[0063] In some examples, combined Figure 3 The aforementioned input processing unit is further configured to obtain symbol information from the input independent variables; the aforementioned conditional inverter includes: a first judge connected to the piecewise linear approximation unit and the ReLU compensator; a symbol processor including an inverting unit and a controlled inverter, the inverting unit being connected to the input processing unit, and the controlled inverter being connected to the inverting unit, the first judge, and the ReLU compensator; wherein, the first judge is configured to output the mapped piecewise linear approximation value as a first intermediate result when the symbol configuration bit is a first value; or, when the symbol configuration bit is a second value, output the mapped piecewise linear approximation value to the controlled inverter; the inverting unit is configured to invert the symbol information; the controlled inverter is configured to output the mapped piecewise linear approximation value as a first intermediate result when the symbol information after inversion indicates that the symbol is the same as the symbol of the target interval; or, when the symbol information after inversion indicates that the symbol is different from the symbol of the target interval, perform a second symbol inversion on the mapped piecewise linear approximation value, and output the second symbol inverted mapped piecewise linear approximation value as the first intermediate result.

[0064] It should be noted that, Figure 3 The diagram only shows the symbol processor receiving the output sign(x) from the preceding module, where the preceding module can be understood as the aforementioned input processing unit.

[0065] In some examples, the aforementioned sign information is used to indicate whether the sign of the input independent variable is positive or negative. The input processing unit can directly extract the sign information from the input independent variable. For example, in an embodiment that represents a signed fixed-point number using two's complement, the input processing unit can directly take the information of the highest-order bit as the sign information.

[0066] In some examples, the first value can be 0, and the second value can be 1. It can be understood that when the symbol configuration bit is set to the first value, the channel between the first determiner and the symbol processor is disconnected, and the first determiner directly outputs the mapped piecewise linear approximation, for example, directly outputting the mapped piecewise linear approximation to the ReLU compensator. When the symbol configuration bit is set to the second value, the channel between the first determiner and the symbol processor is connected, and the symbol processor can perform symbol processing on the mapped piecewise linear approximation and output the reconstructed mapped piecewise linear approximation as the first intermediate result.

[0067] It is understandable that, for a conditional negator, the first decision unit of the conditional negator can directly output the piecewise linear approximation of the mapping as the first intermediate result; or, the symbol processor of the conditional negator can output the piecewise linear approximation of the mapping after symbol processing as the first intermediate result.

[0068] Optionally, the above-mentioned inversion unit may include an input XOR / inverter and a MUX.

[0069] It is understandable that if the symbol configuration bit takes the second value, it can be considered that the symbol information indicates that the symbol is different from the symbol in the target interval. Therefore, the inversion unit can directly invert the symbol information.

[0070] Optionally, the controlled inverter described above may include a quantity input XOR / inverter and a MUX selected by a control signal.

[0071] It is understandable that the controlled inverter can determine whether the sign information indicator sign after sign inversion is the same as the sign of the target interval before outputting the first intermediate result, and take different actions to make the sign of the first intermediate result the same as the sign of the input independent variable.

[0072] Thus, since the conditional inverter can include a first determiner, an inverting unit, and a controlled inverter, when the sign configuration bit is set to a first value (i.e., when the sign information indicates that the sign is the same as the sign of the target interval), the first determiner can directly output the mapped piecewise linear approximation as the first intermediate result. This ensures that the sign of the mapped piecewise linear approximation is the same as the sign of the input independent variable, avoiding redundant calculations. Alternatively, when the sign configuration bit is set to a second value (i.e., when the sign information indicates that the sign is different from the sign of the target interval), the first determiner can input the mapped piecewise linear approximation into the controlled inverter, and the inverting unit will invert the sign information. The controlled inverter can then employ different behaviors based on the inverted sign information to ensure that the sign of the first intermediate result is the same as the sign of the input independent variable. Therefore, the accuracy of subsequent calculations can be guaranteed.

[0073] In some examples, combined Figure 3 The aforementioned ReLU compensator includes: a second determiner connected to a conditional negator and a first adder; a compensation processor including a ReLU generation unit and a second adder, the ReLU generation unit being connected to the second adder, and the second adder being connected to the second determiner and the first adder; wherein, the second determiner is used to output a first intermediate result as a second intermediate result when the compensation configuration bit is a third value; or, when the compensation configuration bit is a fourth value, to output a first intermediate result to the second adder; the ReLU generation unit is used to generate ReLU compensation parameters based on the input independent variable; the second adder is used to add the first intermediate result and the ReLU compensation parameters, and output the result as the second intermediate result.

[0074] Among them, such as Figure 3 As shown, the ReLU compensator can be connected to the other circuits mentioned above and receive the input argument from the other circuits; the ReLU compensator can also be connected to the input processing unit and the reconstruction compensation unit.

[0075] Optionally, the second determiner can be connected to the first determiner of the conditional negator and the sign processor, wherein the second determiner can be connected to the controlled negator of the sign processor. In this way, the second determiner can obtain the first intermediate result from the conditional negator.

[0076] Optionally, the compensation processor described above can be connected to the other circuits described above to obtain input variables from those other circuits.

[0077] Further optionally, the ReLU generation unit can be connected to the other circuits mentioned above to obtain the input arguments from those other circuits. Specifically, the ReLU generation unit can be connected to the other circuits mentioned above ( Figure 3 The diagram illustrates the connection between a ReLU generator unit and a global input (x, i.e., other circuits) to obtain input arguments from other circuits.

[0078] Optionally, the third value can be 0, and the fourth value can be 1. It can be understood that when the compensation configuration bit is set to the third value, the channel between the second judge and the compensation processor is disconnected, and the second judge directly outputs the first intermediate result as the second intermediate result; when the compensation configuration bit is set to the fourth value, the channel between the second judge and the compensation processor is connected, and the compensation processor can compensate for the first intermediate result and output the compensated first intermediate result as the second intermediate result.

[0079] It is understandable that, for a ReLU compensator, the second decision unit of the ReLU compensator can output the mapped piecewise linear approximation as the second intermediate result; or, the second adder of the ReLU compensator can output the compensated mapped piecewise linear approximation as the second intermediate result.

[0080] Further optionally, when the compensation configuration bit takes the fourth value, the second judge can input the first intermediate result into the compensation processor so that the ReLU generation unit of the compensation processor can perform calculations based on the input independent variable and the predetermined parameter value to obtain the ReLU compensation parameter.

[0081] The operation may include, but is not limited to, at least one of the following: maximum value operation, addition operation, multiplication operation, etc.

[0082] Therefore, since the calculation circuit also includes a ReLU compensator, and the ReLU compensator includes a second decision unit and a compensation processor, which includes a ReLU generation unit and a second adder, the first intermediate result can be directly output as the second intermediate result through the second decision unit of the ReLU compensator, or the ReLU compensation parameters can be generated by the ReLU generation unit of the compensation processor of the ReLU compensator, and the first intermediate result can be compensated by the ReLU compensation parameters through the second adder of the compensation processor of the ReLU compensator, and the compensated result can be output as the second intermediate result. Therefore, the accuracy of the calculated function value can be further improved, thus improving the calculation accuracy of the calculation circuit.

[0083] As shown above, since the reconstruction compensation configuration parameter set corresponding to the target activation function includes sign configuration bits, compensation configuration bits, and compensation parameters, the conditional inverter of the reconstruction compensation unit can reconstruct the piecewise linear approximation of the mapping based on the sign configuration bits, outputting a first intermediate result. The sign of this first intermediate result is the same as the sign of the input independent variable, thus ensuring the accuracy of the sign. Furthermore, the ReLU compensator of the reconstruction compensation unit can compensate for the first intermediate result based on the compensation configuration bits, outputting a second intermediate result. The first adder of the reconstruction compensation unit can then compensate for the second intermediate result again based on the compensation parameters, thereby ensuring the accuracy of the result value. Therefore, the accuracy of the output function value can be ensured, and consequently, the accuracy of the calculation can be ensured.

[0084] This application provides a computing circuit that may include an input processing unit, a piecewise linear approximation unit, and a reconstruction compensation unit. The piecewise linear approximation unit is connected to the input processing unit, and the reconstruction compensation unit is connected to the piecewise linear approximation unit. The input processing unit is used to determine the absolute value of the input independent variable and determine the mapped independent variable based on the absolute value of the independent variable. The mapped independent variable belongs to a target interval with a single symbol. The piecewise linear approximation unit is used to output a mapped piecewise linear approximation value based on the piecewise linear parameter set corresponding to the mapped independent variable and the target activation function. The domain configured for the piecewise linear parameter set is the target interval. The reconstruction compensation unit is used to reconstruct and compensate the mapped piecewise linear approximation value based on the reconstruction compensation configuration parameter set corresponding to the target activation function and output the function value of the target activation function under the input independent variable. Because the input processing unit can determine the absolute value of the input independent variable and determine the mapping independent variable based on the absolute value, thus mapping the input independent variable belonging to a region of arbitrary signs to a target interval belonging to a single sign, instead of directly using the input independent variable belonging to an interval including both positive and negative signs, the piecewise linear approximation unit can output a mapped piecewise linear approximation value based on the mapped independent variable and the piecewise linear parameter set corresponding to the target activation function (the domain of which is configured is the target interval of a single sign). However, when the configured domain is the target interval of a single sign, the number of parameters in the piecewise linear parameter set is less than when the configured domain includes both positive and negative signs. The number of parameters in the linear parameter set means that the piecewise linear approximation unit can output a mapped piecewise linear approximation value based on the mapped independent variable and a piecewise linear parameter set with a smaller number of parameters. Moreover, the reconstruction compensation unit can reconstruct and compensate the mapped piecewise linear approximation value based on the reconstruction compensation configuration parameter set corresponding to the target activation function, and output the function value of the target activation function under the input independent variable. That is, the reconstruction compensation unit can output accurate function values. Therefore, while ensuring the accuracy of the output function value, the number of parameters in the piecewise linear parameter set used for calculation can be reduced, thereby reducing the number of parameters stored in the computing circuit, reducing the size of the computing circuit, and further reducing the size of the chip containing the computing circuit, thus facilitating chip deployment.

[0085] In some embodiments of this application, such as Figure 4 As shown, the above-mentioned computing circuit further includes: a register group connected to the piecewise linear approximation unit and the reconstruction compensation unit. The register group stores a piecewise linear parameter set and a reconstruction compensation configuration parameter set corresponding to at least one activation function. The register group is used to provide the piecewise linear approximation unit with the piecewise linear parameter set corresponding to the target activation function of at least one activation function, and to provide the reconstruction compensation unit with the reconstruction compensation configuration parameter set corresponding to the target activation function.

[0086] In some examples, the above register group may include at least one register.

[0087] In some examples, the target activation function described above can be determined by the computational requirements of the computing circuit.

[0088] It can be understood that the piecewise linear parameter set corresponding to each of the above at least one activation function belongs to the above piecewise linear parameter set.

[0089] In some examples, at least one piecewise linear parameter set and a reconstruction compensation configuration parameter set corresponding to an activation function can be pre-stored in a register set, so that the register set can provide the piecewise linear approximation unit with the piecewise linear parameter set corresponding to the target activation function and the reconstruction compensation unit with the reconstruction compensation configuration parameter set corresponding to the target activation function.

[0090] Therefore, since the computational circuit also includes a register set, which stores at least one piecewise linear parameter set and reconstruction compensation configuration parameter set corresponding to an activation function, and the domain configured for this piecewise linear parameter set is the aforementioned target interval, the number of parameters in the piecewise linear parameter set corresponding to the at least one activation function is less when the configured domain includes both positive and negative signs. Thus, while ensuring the computational circuit supports at least one activation function, the number of parameters stored in the register set can be reduced, thereby reducing the size of the register set.

[0091] Figure 5 A schematic diagram of the chip structure provided in an embodiment of this application is shown. Figure 5 As shown, the chip provided in this application embodiment includes the computing circuit in the above embodiment.

[0092] In some embodiments of this application, the above-described computing circuitry can be deployed in a neural network accelerator.

[0093] In some embodiments of this application, the chip may include, but is not limited to, at least one of the following: a neural network processing unit, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a central processing unit (CPU), or a system on a chip (SoC). This application does not specifically limit the chip to any of these.

[0094] It should be noted that the descriptions of each unit in the computing circuit can be found in the specific descriptions in the above embodiments, and will not be repeated here in the embodiments of this application.

[0095] This application provides a chip that includes the computing circuit described in the above embodiments. Because the input processing unit of the chip's computing circuit can determine the absolute value of the input independent variable and determine the mapping independent variable based on the absolute value, it maps the input independent variable belonging to a region of arbitrary signs to a target interval belonging to a single sign, instead of directly using the input independent variable belonging to an interval including both positive and negative signs. Thus, the piecewise linear approximation unit of the computing circuit can output a mapped piecewise linear approximation value based on the mapped independent variable and the piecewise linear parameter set corresponding to the target activation function (the domain of which is the target interval with a single sign). Furthermore, when the configured domain is the target interval with a single sign, the number of parameters in the piecewise linear parameter set is less than when the configured domain includes both positive and negative signs. The piecewise linear approximation unit can output a mapped piecewise linear approximation value based on the mapped independent variable and the piecewise linear parameter set with a smaller parameter size. Furthermore, the reconstruction compensation unit of the computational circuit can reconstruct and compensate the mapped piecewise linear approximation value based on the reconstruction compensation configuration parameter set corresponding to the target activation function, outputting the function value of the target activation function under the input independent variable. Thus, the reconstruction compensation unit can output an accurate function value. Therefore, while ensuring the accuracy of the output function value, the number of parameters in the piecewise linear parameter set used for computation can be reduced, thereby reducing the number of parameters stored in the computational circuit, reducing the size of the computational circuit, and consequently reducing the size of the chip containing the computational circuit, thus facilitating chip deployment.

[0096] Figure 6 A flowchart illustrating the calculation method provided in an embodiment of this application is shown, applied to the calculation circuit in the above embodiment. For example... Figure 6 As shown, the calculation method provided in this application embodiment may include the following steps 101 to 103.

[0097] Step 101: The calculation circuit determines the absolute value of the input independent variable through its input processing unit, and determines the mapping independent variable based on the absolute value of the independent variable.

[0098] In this embodiment of the application, the above-mentioned mapping independent variable belongs to the target interval of a single symbol.

[0099] Step 102: The computational circuit outputs the mapped piecewise linear approximation value based on the piecewise linear parameter set corresponding to the mapped independent variable and the target activation function through the piecewise linear approximation unit of the computational circuit.

[0100] In this embodiment of the application, the domain configured for the above-mentioned piecewise linear parameter set is the target interval.

[0101] In some embodiments of this application, the aforementioned piecewise linear parameter set includes at least one set of parameters, each set of parameters including piecewise linear interval information, slope information, and intercept information. The piecewise linear interval information is used to indicate the segmented sub-intervals within the target interval. Step 102 can be specifically implemented through steps 102a to 102c described below.

[0102] Step 102a: The calculation circuit determines the target segmented sub-interval where the mapping independent variable is located from the segmented sub-intervals indicated by the segmented linear interval information in at least one set of parameters through the breakpoint comparator array of the segmented linear approximation unit, and outputs the sub-interval identifier of the target segmented sub-interval.

[0103] Step 102b: The calculation circuit uses the parameter selector of the piecewise linear approximation unit to determine the target slope information and target intercept information corresponding to the target piecewise sub-interval from the slope information and intercept information in at least one set of parameters according to the sub-interval identifier, and outputs the target slope information and target intercept information.

[0104] Step 102c: The calculation circuit calculates the product of the mapping independent variable and the target slope information through the linear synthesizer of the piecewise linear approximation unit, adds the product value and the target intercept information, and outputs the result as the mapping piecewise linear approximation value.

[0105] Thus, it can be seen that since the calculation circuit can determine the target segmented sub-interval where the mapped independent variable is located through the breakpoint comparator array, and can determine the target slope information and target intercept information corresponding to the target segmented sub-interval based on the sub-interval identifier through the parameter selector, and the calculation circuit can calculate and output the mapped segmented linear approximation value through the linear synthesizer, that is, the mapped segmented linear approximation value can be calculated and output through only the breakpoint comparator array, parameter selector and linear synthesizer, without setting up other modules, the size of the calculation circuit can be reduced.

[0106] Step 103: The calculation circuit uses the reconstruction compensation unit of the calculation circuit to reconstruct and compensate the piecewise linear approximation value of the mapping according to the reconstruction compensation configuration parameter set corresponding to the target activation function, and outputs the function value of the target activation function under the input independent variable.

[0107] In some embodiments of this application, the reconstruction compensation configuration parameter set corresponding to the target activation function includes symbol configuration bits, compensation configuration bits, and compensation parameters. Step 103 can be specifically implemented through steps 103a to 103c as described below.

[0108] Step 103a: The calculation circuit reconstructs the piecewise linear approximation value of the mapping by reconstructing the conditional inverter of the reconstruction compensation unit according to the sign configuration bit, and outputs the first intermediate result.

[0109] In some examples, before step 103a above, the calculation method provided in this application embodiment may also include step 201 below, and step 103a above can be specifically implemented by step 103a1 or step 103a2 below.

[0110] Step 201: The calculation circuit obtains symbolic information from the input independent variables through the input processing unit.

[0111] Step 103a1: The calculation circuit outputs the mapped piecewise linear approximation as the first intermediate result when the sign configuration bit is set to the first value by the first judgment unit of the conditional inverter.

[0112] Step 103a2: The calculation circuit, through the first judge, outputs the mapped piecewise linear approximation value to the controlled inverter of the symbol processor of the conditional inverter when the symbol configuration bit is the second value. The symbol information is inverted through the inverting unit of the symbol processor, and the first intermediate result is output through the controlled inverter based on the symbol information after inversion and the mapped piecewise linear approximation value.

[0113] Thus, when the sign configuration bit is set to the first value (i.e., when the sign information indicates the same sign as the target interval), the calculation circuit can directly output the mapped piecewise linear approximation as the first intermediate result through the first judge. This ensures that the sign of the mapped piecewise linear approximation is the same as the sign of the input independent variable, avoiding redundant calculations. Alternatively, when the sign configuration bit is set to the second value (i.e., when the sign information indicates a different sign than the target interval), the calculation circuit can input the mapped piecewise linear approximation into the inversion unit through the first judge. The inversion unit inverts the sign information, and the controlled inverter can use different behaviors based on the inverted sign information to ensure that the sign of the first intermediate result is the same as the sign of the input independent variable. Therefore, the accuracy of subsequent calculations can be guaranteed.

[0114] In some examples, step 103a2 above can be implemented by step 103a2a or step 103a2b below.

[0115] Step 103a2a: The calculation circuit, through the first judge, outputs a mapped piecewise linear approximation value to the controlled inverter of the symbol processor of the conditional inverter when the symbol configuration bit is the second value. The symbol information is inverted through the inverting unit of the symbol processor. And when the symbol information after inversion indicates that the symbol is the same as the symbol in the target interval, the mapped piecewise linear approximation value is output as the first intermediate result through the controlled inverter.

[0116] Step 103a2b: The calculation circuit, through the first judge, outputs a mapped piecewise linear approximation value to the controlled inverter of the symbol processor of the conditional inverter when the symbol configuration bit is the second value. The symbol information is inverted through the inverting unit of the symbol processor. When the symbol information after inversion indicates that the symbol is different from the symbol in the target interval, the mapped piecewise linear approximation value is inverted again through the controlled inverter. The mapped piecewise linear approximation value after inversion is output as the first intermediate result.

[0117] Thus, it can be seen that, since the computing circuit can also use a controlled inverter to employ different behaviors based on the inverted sign information to ensure that the sign of the first intermediate result is the same as the sign of the input independent variable, the accuracy of subsequent calculations can be guaranteed.

[0118] Step 103b: The calculation circuit compensates the first intermediate result according to the compensation configuration bit by using the ReLU compensator of the reconstruction compensation unit, and outputs the second intermediate result.

[0119] In some examples, step 103b above can be implemented by step 103b1 or step 103b2 as described below.

[0120] Step 103b1: The calculation circuit, through the second judge of the ReLU compensator, outputs the first intermediate result as the second intermediate result when the value of the compensation configuration bit is the third value.

[0121] Step 103b2: The calculation circuit, through the second judge, outputs the first intermediate result to the second adder of the ReLU compensator when the value of the compensation configuration bit is the fourth value. The ReLU generation unit of the compensation processor of the ReLU compensator generates the ReLU compensation parameter according to the input independent variable. The second adder of the compensation processor adds the first intermediate result and the ReLU compensation parameter, and outputs the result as the second intermediate result.

[0122] Thus, it can be seen that since the calculation circuit can directly output the first intermediate result as the second intermediate result through the second judgment unit of the ReLU compensator, or generate ReLU compensation parameters through the ReLU generation unit of the compensation processor of the ReLU compensator, and use the ReLU compensation parameters to compensate the first intermediate result through the second adder of the ReLU compensator, and output the compensated result as the second intermediate result, the accuracy of the calculated function value can be further improved, thereby improving the calculation accuracy of the calculation circuit.

[0123] Step 103c: The calculation circuit adds the compensation parameters and the second intermediate result by reconstructing the first adder of the compensation unit, and outputs the result as the function value.

[0124] Thus, since the reconstruction compensation configuration parameter set corresponding to the aforementioned target activation function includes sign configuration bits, compensation configuration bits, and compensation parameters, the computational circuit can reconstruct the piecewise linear approximation of the mapping based on the sign configuration bits using the conditional inverter of the reconstruction compensation unit, outputting a first intermediate result. The sign of this first intermediate result is the same as the sign of the input independent variable, thereby ensuring the accuracy of the sign. Furthermore, the computational circuit can compensate the first intermediate result based on the compensation configuration bits using the ReLU compensator of the reconstruction compensation unit, outputting a second intermediate result. The computational circuit can then compensate the second intermediate result again based on the compensation parameters using the first adder of the reconstruction compensation unit, thereby ensuring the accuracy of the result value. Therefore, the accuracy of the output function value can be ensured, thus ensuring the accuracy of the calculation.

[0125] This application provides a calculation method in which a calculation circuit can determine the absolute value of an input independent variable through its input processing unit, and determine a mapping independent variable based on the absolute value of the input independent variable. This mapping independent variable belongs to a target interval with a single symbol. Furthermore, through the piecewise linear approximation unit of the calculation circuit, a piecewise linear approximation value is output based on the piecewise linear parameter set corresponding to the mapping independent variable and the target activation function. The domain configured for this piecewise linear parameter set is the target interval. Thus, the calculation circuit can reconstruct and compensate the piecewise linear approximation value based on the reconstruction compensation configuration parameter set corresponding to the target activation function through its reconstruction compensation unit, and output the function value of the target activation function under the input independent variable. Because the computational circuit can determine the absolute value of the input independent variable through the input processing unit and determine the mapping independent variable based on the absolute value, thus mapping the input independent variable belonging to a region of arbitrary signs to a target interval belonging to a single sign, instead of directly using the input independent variable belonging to an interval including both positive and negative signs, the computational circuit can output the mapped piecewise linear approximation value through the piecewise linear parameter set corresponding to the mapped independent variable and the target activation function (the domain of which is configured is the target interval of a single sign). Furthermore, when the configured domain is the target interval of a single sign, the number of parameters in this piecewise linear parameter set is less than when the configured domain includes both positive and negative signs. The piecewise linear parameter set has a small number of parameters. In other words, the piecewise linear approximation unit can output a mapped piecewise linear approximation value based on the mapped independent variable and the piecewise linear parameter set with a smaller number of parameters. Moreover, the computing circuit can reconstruct and compensate the mapped piecewise linear approximation value through the reconstruction compensation configuration parameter set corresponding to the target activation function, and output the function value of the target activation function under the input independent variable. That is, the reconstruction compensation unit can output an accurate function value. Therefore, while ensuring the accuracy of the output function value, the number of parameters in the piecewise linear parameter set used for calculation can be reduced, thereby reducing the number of parameters stored in the computing circuit, reducing the size of the computing circuit, and further reducing the size of the chip containing the computing circuit, thus facilitating chip deployment.

[0126] In some embodiments of this application, before step 102 above, the calculation method provided in the embodiments of this application may further include step 202 below.

[0127] Step 202: The calculation circuit provides the piecewise linear parameter set corresponding to the target activation function in at least one activation function to the piecewise linear approximation unit through the register group, and provides the reconstruction compensation configuration parameter set corresponding to the target activation function to the reconstruction compensation unit.

[0128] In this embodiment of the application, the above-mentioned register group stores at least one piecewise linear parameter set and reconstruction compensation configuration parameter set corresponding to an activation function.

[0129] Thus, since the register group stores at least one piecewise linear parameter set and reconstruction compensation configuration parameter set corresponding to an activation function, and the domain configured for the piecewise linear parameter set is the aforementioned target interval, the number of parameters in the piecewise linear parameter set corresponding to the at least one activation function is less when the configured domain is a target interval with a single symbol than when the configured domain includes both positive and negative symbols. Therefore, while ensuring that the computing circuit supports at least one activation function, the number of parameters stored in the register group can be reduced, thereby reducing the size of the register group and thus reducing the size of the chip.

[0130] In some embodiments of this application, before step 202 above, the calculation method provided in the embodiments of this application may further include steps 203 to 206 as described below.

[0131] Step 203: The calculation circuit determines the set of reconstruction compensation configuration parameters corresponding to the first activation function based on the first activation function and the first residual function.

[0132] In this embodiment of the application, the first activation function is any one of at least one activation function; the first cofunction is odd-symmetric or even-symmetric about the zero point.

[0133] In some examples, the first cofunction mentioned above can be a predefined cofunction, a preconfigured cofunction, or a cofunction defined according to the first activation function, etc.

[0134] In some examples, the computational circuit can use an enumeration method to determine the set of reconstruction compensation configuration parameters corresponding to the first activation function based on the first activation function and the first cofactor.

[0135] Step 204: The calculation circuit performs numerical sampling on the first cofunction in the target interval to obtain at least one target data point.

[0136] Step 205: The calculation circuit performs piecewise linear approximation based on at least one target data point and the first cofunction to obtain the piecewise linear parameter set corresponding to the first activation function.

[0137] In some examples, the computational circuit can perform piecewise linear approximation based on at least one target data point and a first cofunction using at least one of the following algorithms, which may include, but are not limited to, at least one of the following: genetic algorithm, minimum mean square error optimal segmentation, dynamic programming algorithm, segmentation method based on other optimization objectives, etc.

[0138] Step 206: The calculation circuit stores the piecewise linear parameter set and the reconstruction compensation configuration parameter set corresponding to the first activation function into the register group.

[0139] It is understood that the piecewise linear parameter set and the reconstruction compensation configuration parameter set corresponding to at least one activation function stored in the register group can be generated and stored through the above steps 203 to 206.

[0140] Thus, it can be seen that since the computing circuit can determine the reconstruction compensation configuration parameter set corresponding to the first activation function based on the first activation function and the first cofunction, and the first cofunction is odd-symmetric or even-symmetric about the zero point, the first cofunction can be numerically sampled in the target interval to obtain at least one target data point. Then, based on at least one target data point and the first cofunction, piecewise linear approximation processing is performed to obtain a piecewise linear parameter set whose domain is the domain of the single symbol mentioned above. Compared with the piecewise linear parameter set covering both positive and negative symbols, the size is directly halved. In this way, the computing circuit can store the piecewise linear parameter set corresponding to the first activation function and the reconstruction compensation configuration parameter set in the register group, so that less data is stored in the register group instead of more data. Therefore, the storage overhead of the computing circuit can be reduced, thereby reducing the area of ​​the computing circuit configuration storage and thus reducing the size of the computing circuit.

[0141] The following will use complete examples to illustrate the computing circuits and computing methods provided in the embodiments of this application.

[0142] In the following embodiments, Implementation Method 1 provides a general symmetric decomposition and half-domain piecewise linear approximation method; Implementation Method 2 provides a hardware circuit for a general activation function calculation unit that implements this method (i.e., the calculation circuit in the above embodiments). The two can be used in combination or independently as needed: Implementation Method 1 can be used as an offline analysis and parameter generation process on the software side, while Implementation Method 2 can be used independently as a hardware implementation of the general activation function to access third-party processes.

[0143] The following example uses the single symbol mentioned above as the negative sign to illustrate the point.

[0144] Implementation Method 1: General Symmetric Decomposition and Half-Domain Piecewise Linear Approximation Method

[0145] This application provides a symmetric decomposition method for uniformly mapping multiple activation functions to a piecewise linear approximation hardware in the same half-domain. This method is based on the symmetry of the activation functions themselves, and... The approximate domain extends from the complete real number line. Specification to negative half shaft And reconstructed on the hardware side through simple symbolic mirroring and ReLU compensation. The global value of .

[0146] like Figure 7 As shown in the figure, this application provides a calculation method, including the following steps:

[0147] Step 1: Determine the symmetry type and configure parameters.

[0148] Specifically, regarding the activation functions to be supported Determine a pair of configuration bits With a constant bias This makes the remainder function defined under this configuration... (See the unified calculation formula below) It exhibits odd or even symmetry near the zero point.

[0149] The activation function to be paid can be the first activation function in the above embodiment, and the configuration bit... This can be the symbol configuration bit in the above embodiments, the configuration bit The above can be the compensation configuration bit in the above embodiment, the above constant bias c can be the compensation parameter in the above embodiment, and the above cofunction can be the first cofunction in the above embodiment.

[0150] Step 2: Construct the objective function for the negative half-axis.

[0151] Specifically, in coherence function on interval Numerical sampling is performed to obtain target data points (i.e., at least one target data point in the above embodiments) for subsequent piecewise linear approximation.

[0152] Where x is the coordinate value.

[0153] Step 3: Solve using piecewise linear approximation.

[0154] Specifically, for On A piecewise linear approximation is performed to obtain a set of breakpoints (i.e., piecewise linear interval information in the above embodiments), slope information, and intercept information (i.e., at least one set of parameters in the above embodiments). This method is not limited to specific methods for solving the piecewise linear parameters: genetic algorithms, minimum mean square error optimal segmentation, dynamic programming algorithms, or segmentation methods based on other optimization objectives are all applicable. In hardware implementation, the slope of the leftmost segment can be set to zero, so that the approximation takes a horizontal asymptote at the far end.

[0155] Step 4: Parameter loading.

[0156] Specifically, the obtained piecewise linear parameters (i.e., at least one set of parameters in the above embodiments) and constants are... With configuration bits , Loaded into the configuration register group (i.e., the register group in the above embodiment) of the hardware computing unit (i.e., the computing circuit in the above embodiment) of Embodiment 2.

[0157] Specifically:

[0158] (1) This method uses the following unified calculation formula to express and reconstruct various activation functions:

[0159] ;

[0160] in, Indicates in upper covariance The function obtained by piecewise linear approximation always has a non-positive real number input domain; because For any real number All are non-positive real numbers. A good definition is given for all inputs across the entire domain.

[0161] The symbol reconstruction factor takes values ​​of or , configured by 1 bit Select this configuration bit. The symbol configuration bit in the above embodiments can be configured when When =1, (Enable symbolic mirroring), used for global reconstruction of odd symmetric types; when When =0, (Bypass symbol mirroring), used for global reconstruction of even symmetric types.

[0162] The ReLU compensation factor is configured by 1 bit. Select this configuration bit. This can be the compensation configuration bit in the above embodiments, when When =1, (Enable ReLU compensation); when When = 0, the entire ReLU compensation link is bypassed; this term is taken as... .

[0163] It is a constant bias, determined by a constant entry in the configuration register (e.g., the register group in the above embodiment).

[0164] thus, and Adopt a unified polarity convention: Take This indicates that the corresponding branch (symmetric mirroring / ReLU compensation) is enabled, and the value is taken as follows: This indicates that the branch road is bypassed.

[0165] The above formula shows that the expression of various activation functions in this method depends only on a single set of half-domain piecewise linear parameters (used for implementation). A constant bias and two 1-bit configuration bits , .

[0166] (2) Two types of symmetric decomposition

[0167] Define the complement function This method modifies the supported activation functions by their modulo functions. Symmetry types are divided into two categories:

[0168] Type 1: Odd symmetric type, corresponding to , Under this type, satisfy .because For any The output of piecewise linear approximation always holds true. It is approaching ;Utilizing odd symmetry, for any have ,thereby Representative operators include Tanh, Sigmoid, etc.

[0169] Type 2: Even-symmetric type, corresponding to =0、 =1. Under this type, satisfy Since even functions are symmetric about the vertical axis, for any have ,thereby Representative operators include GELU, SiLU, Softplus, etc.

[0170] The following uses GELU as an example to illustrate the mathematical basis of even-symmetric decomposition. GELU is defined as follows: , Let be the cumulative distribution function of the standard normal distribution. Take . =1, c=0, then the cofunction is Using identities , can be obtained .exist Within the interval, For any ,Compare and It can be known ,Right now It is an even function. Similarly, it can be verified that the cofunctions of operators such as SiLU and Softplus also satisfy even symmetry.

[0171] (3) Configuration examples of the five core activation functions

[0172] Based on the above method, the configuration parameters for the five currently mainstream activation functions are determined as follows:

[0173] For Tanh: odd symmetric type, No ReLU compensation. constant bias The objective function of the half-domain piecewise linear approximation is: , This function is an odd function.

[0174] For Sigmoid: odd symmetric type, No ReLU compensation. constant bias The objective function of the half-domain piecewise linear approximation is: , The function is an odd function; constant bias From the configuration register The term is added to the summation path.

[0175] For GELU: even-symmetric type, Includes ReLU compensation. constant bias The objective function of the half-domain piecewise linear approximation is: , ,Right now .

[0176] For SiLU: even-symmetric type, Includes ReLU compensation. constant bias The objective function of the half-domain piecewise linear approximation is: , .

[0177] For Softplus: an even-symmetric type, Includes ReLU compensation. constant bias The objective function of the half-domain piecewise linear approximation is: , .

[0178] The applicability of this method is not limited to the five core activation functions mentioned above. In another embodiment, for piecewise linear variants or fast approximations based on the above functions, such as HardSigmoid, HardTanh, Hardswish, QuickGELU, and Softsign, their symmetry type and configuration parameters can be determined using the same process, and the unified calculation formula and hardware configuration interface of this method are equally applicable to them.

[0179] The beneficial effects of Embodiment 1 of this application are as follows:

[0180] 1. The semi-domain approximation halves the size of the piecewise linear parameters. This method only requires... upper covariance By performing piecewise linear approximation, compared to the global piecewise linear scheme that covers the entire real number axis, the scale of parameters (breakpoints, slopes, intercepts) is directly halved, reducing storage overhead and on-chip storage area.

[0181] 2. Unified access for multiple activation functions. A single hardware interface (one set of half-domain piecewise linear parameters + 1 constant bias + 2 configuration bits) can support multiple activation functions. When a new activation function that satisfies the mathematical property of "exhibiting odd or even symmetry after compensation" appears, it is only necessary to resolve its half-domain piecewise linear parameters and configuration bits, and it can be directly connected to the hardware computing unit of this method without modifying the hardware itself.

[0182] 3. Covers current mainstream activation functions. This method is applicable to activation functions used in current mainstream Transformers and edge models, such as Tanh, Sigmoid, GELU, SiLU, and Softplus; it is also applicable to their piecewise linear variants (HardSigmoid, HardTanh, etc.) and fast approximation forms (Hardswish, QuickGELU, etc.).

[0183] Implementation Method 2: Hardware Circuit of General Activation Function Calculation Unit

[0184] This embodiment discloses a general activation function calculation unit hardware circuit (i.e., the calculation circuit in the above embodiments) for implementing the method of Embodiment 1. The circuit is basically composed of a piecewise linear approximation unit, a ReLU compensator, a register set, and a reconstruction compensation unit. It can switch to approximators of various activation functions during runtime according to the loaded configuration parameters (i.e., the piecewise linear parameter set and the reconstruction compensation configuration parameter set in the above embodiments).

[0185] Combination Figure 4 The aforementioned computing circuit includes:

[0186] Input processing unit: from input arguments Extracting symbolic information with the absolute value of the independent variable And generate the inverted non-positive signal. For downstream use. In the embodiment of representing signed fixed-point numbers using two's complement, Take it directly from the highest bit. and It can be obtained through bitwise operations and inversion plus one, with extremely low hardware overhead.

[0187] Piecewise linear approximation unit: As input, calculate based on the piecewise linear parameter set corresponding to the target activation function loaded in the register set. This unit is the only physical module in this computing unit that undertakes nonlinear approximate calculations.

[0188] ReLU compensator: and Input, Output And subject to configuration bits Gating: When The entire module is bypassed (either via clock gating or data gating) and does not perform any calculations. In an embodiment where the sign bit is used as the selection signal, the module is implemented using a MUX: when Indicates non-negative selection Otherwise choose .

[0189] Reconstruction compensation unit: based on configuration bits , With constant register Synthesized final output .

[0190] Register set: Stores the piecewise linear parameter set and constant bias corresponding to the current activation function (i.e., the target activation function in the above embodiment). With configuration bits , When switching activation functions, only the parameters in the register set need to be reloaded; the hardware structure itself remains unchanged.

[0191] For example, hardware configuration examples for the five activation functions.

[0192] For the five core activation functions given in Implementation Method 1, the hardware configuration and runtime data path of this computing unit are as follows:

[0193] For Tanh, configuration bits , constant bias Piecewise linear parameters are exist The semi-domain approximation parameters. Runtime data path: via conditional inverter, according to Inversion or identity pass; ReLU compensation link is Close; the summator adds the output of the inverted unit to the zero constant bias, resulting in... The approximate output.

[0194] For Sigmoid, the configuration bit , constant bias Piecewise linear parameters are exist The semi-domain approximation parameters are on the [top / bottom]. The runtime data path is similar to Tanh, the difference being that an additional constant is added to the summator and adder. This yields an approximate output of the Sigmoid function.

[0195] For GELU, configuration bits , constant bias Piecewise linear parameters are exist The semi-domain approximation parameters. Runtime data path: After bypassing the conditional inverter (identical pass), the ReLU generation unit operates and outputs. The product enters the summation path via the ReLU compensation link; the summator adds the two products together to obtain an approximate output of GELU.

[0196] For SiLU and Softplus, the configuration parameters are the same as for GELU. , , Only piecewise linear parameters are replaced with the corresponding function in The semi-domain approximation parameters are on the same surface; the runtime data path is completely consistent with GELU.

[0197] Switching between the five activation functions only requires updating the piecewise linear parameter and constant bias in the configuration register set. The hardware structure remains unchanged, except for the two 1-bit configuration bits. In another embodiment, the configuration register set can be designed as a double-buffered structure, allowing the switching process to be completed within a single clock cycle, which is suitable for network structures that require the mixed use of multiple activation functions in the same computational process.

[0198] The beneficial effects of Embodiment 2 of this application:

[0199] 1. A single physical circuit supports multiple activation functions. Compared to the approach of configuring a dedicated arithmetic circuit for each activation function, this computational unit uses a half-domain piecewise linear core unit, a ReLU generation unit, and a reconstruction compensation logic to handle the hardware implementation of all supported activation functions, significantly reducing hardware area and power consumption.

[0200] 2. The size of the piecewise linear approximation unit is halved compared to the global approach. Directly benefiting from the symmetric decomposition in Implementation Method 1, the piecewise linear approximation core unit only needs to be stored... The breakpoints, slopes, and intercepts on the chip are reduced by half the storage size compared to a global piecewise linear scheme that covers the entire real number axis, further reducing the area and power consumption of on-chip configured storage.

[0201] 3. The reconstruction and compensation module has extremely low overhead. (Sign bit extraction, ...) and Both ReLU generation and ReLU generation are implemented using bit-level operations or a single MUX, with zero-floating-point arithmetic; the conditional inverter is implemented using a 1-bit controlled inverter; the ReLU link gate is implemented using an AND gate or a MUX; constant bias is used. It is introduced by a single adder. The hardware overhead of the entire reconstruction and compensation path is negligible compared to traditional multipliers, dividers, and exponentiation units.

[0202] 4. Can be embedded as a standard SFU unit in the accelerator mainstream waterline. The interface configuration of this computing unit (input...) Output The configuration parameter package has the same interface form as the Special Function Unit (SFU) commonly found in neural network accelerators. It can be used as a nonlinear post-processing path for standard SFU embedded accelerators and can be executed in parallel with the main paths such as matrix multiplication and convolution.

[0203] 5. Operator switching incurs no hardware cost. Switching from one activation function to another only requires reloading the piecewise linear parameters, constant biases, and configuration bits in the configuration register set; the hardware structure itself remains unchanged. In embodiments where the configuration register set employs a double-buffered structure, the switching process can be completed within a single clock cycle, making it suitable for network structures that require the mixed use of multiple activation functions in the same computational process.

[0204] Furthermore, the computing circuit and computing method provided in the embodiments of this application also satisfy the following:

[0205] 1. New activation functions that satisfy the corresponding mathematical properties naturally fall within the applicability of this computational circuit and method. The types of activation functions supported by this computational circuit and method are characterized by their inherent symmetry: after appropriate compensation, they exhibit odd or even symmetry. When a new activation function that satisfies this property appears, it is only necessary to resolve its half-domain piecewise linear parameters and configuration bits on the algorithm side, and it can be directly loaded into this computational circuit without modifying the hardware itself.

[0206] 2. This computing circuit can be used as a nonlinear post-processing path for embedded edge AI accelerators as a general-purpose SFU unit, and can be executed in parallel with main paths such as matrix multiplication and convolution; it can also be used as an independent nonlinear computing IP module and integrated into various computing architectures.

[0207] 3. The integer implementation of this computing circuit is compatible with the low-bit quantization deployment of neural networks. In another embodiment, piecewise linear parameters and constant biases can be stored in low-bit-ratio point form, connecting with the mainstream of quantized matrix multiplication, further reducing the overall overhead of edge inference.

[0208] It should be noted that each of the above method embodiments, or various possible implementations of each method embodiment, can be executed individually or in combination of any two or more. The specific implementation can be determined according to actual usage requirements, and this application embodiment does not impose any restrictions on this.

[0209] The calculation method provided in this application can be executed by a computing device. This application uses the execution of the calculation method by a computing device as an example to illustrate the computing device provided in this application.

[0210] Figure 8 This is a schematic diagram of the structure of a computing device provided in an embodiment of this application. Figure 8 As shown, the computing device 300 includes a processing module 301.

[0211] The processing module 301 is used to determine the absolute value of the input independent variable, and determine the mapping independent variable based on the absolute value of the independent variable. The mapping independent variable belongs to the target interval with a single symbol. It outputs the mapped piecewise linear approximation value based on the piecewise linear parameter set corresponding to the mapping independent variable and the target activation function. The domain configured by the piecewise linear parameter set is the target interval. It also performs reconstruction compensation on the mapped piecewise linear approximation value based on the reconstruction compensation configuration parameter set corresponding to the target activation function, and outputs the function value of the target activation function under the input independent variable.

[0212] This application provides a computing device. Because the computing device can determine the absolute value of the input independent variable through an input processing unit and determine the mapping independent variable based on the absolute value, it maps the input independent variable belonging to a region of arbitrary signs to a target interval belonging to a single sign, instead of directly using the input independent variable belonging to an interval including both positive and negative signs. Thus, the computing device can output a mapped piecewise linear approximation value through a piecewise linear approximation unit based on the mapped independent variable and the piecewise linear parameter set corresponding to the target activation function (the domain of which is the target interval with a single sign). Furthermore, when the configured domain is the target interval with a single sign, the number of parameters in the piecewise linear parameter set is less than when the configured domain includes both positive and negative signs. In this case, the number of parameters in the piecewise linear parameter set is reduced. That is, the piecewise linear approximation unit can output a mapped piecewise linear approximation value based on the mapped independent variable and the piecewise linear parameter set with a smaller number of parameters. Moreover, the computing device can reconstruct and compensate the mapped piecewise linear approximation value based on the reconstruction compensation configuration parameter set corresponding to the target activation function through the reconstruction compensation unit, and output the function value of the target activation function under the input independent variable. In other words, the reconstruction compensation unit can output an accurate function value. Therefore, while ensuring the accuracy of the output function value, the number of parameters in the piecewise linear parameter set used for calculation can be reduced, thereby reducing the number of parameters stored in the computing circuit, reducing the size of the computing circuit, and further reducing the size of the chip containing the computing circuit. This facilitates the deployment of the computing device.

[0213] In some embodiments of this application, the processing module 301 is further configured to, before the piecewise linear approximation unit of the computing circuit outputs the mapped piecewise linear approximation value based on the piecewise linear parameter set corresponding to the target activation function in at least one activation function, and to provide the reconstruction compensation configuration parameter set corresponding to the target activation function to the piecewise linear approximation unit through the register group of the computing circuit; wherein, the register group stores the piecewise linear parameter set corresponding to at least one activation function and the reconstruction compensation configuration parameter set.

[0214] In some embodiments of this application, the processing module 301 is further configured to: determine a reconstruction compensation configuration parameter set corresponding to a first activation function based on a first activation function and a first cofunction, wherein the first cofunction is odd-symmetric or even-symmetric about the zero point, before providing the piecewise linear parameter set corresponding to the target activation function of at least one activation function to the piecewise linear approximation unit through the register group of the computing circuit; sample the first cofunction numerically in the target interval to obtain at least one target data point; perform piecewise linear approximation processing based on the at least one target data point and the first cofunction to obtain a piecewise linear parameter set corresponding to the first activation function; and store the piecewise linear parameter set corresponding to the first activation function and the reconstruction compensation configuration parameter set in the register group; wherein the first activation function is any one of at least one activation function.

[0215] In some embodiments of this application, the aforementioned piecewise linear parameter set includes at least one set of parameters, each set of parameters including piecewise linear interval information, slope information, and intercept information. The piecewise linear interval information is used to indicate the piecewise sub-intervals within the target interval. Specifically, the processing module 301 is used to determine the target piecewise sub-interval where the mapping independent variable is located from the piecewise sub-intervals indicated by the piecewise linear interval information in the at least one set of parameters through the breakpoint comparator array of the piecewise linear approximation unit, and output the sub-interval identifier of the target piecewise sub-interval; and to determine the target slope information and target intercept information corresponding to the target piecewise sub-interval from the slope information and intercept information in the at least one set of parameters through the parameter selector of the piecewise linear approximation unit, based on the sub-interval identifier, and output the target slope information and target intercept information; and to calculate the product value of the mapping independent variable and the target slope information through the linear synthesizer of the piecewise linear approximation unit, add the product value and the target intercept information, and output the result as the mapped piecewise linear approximation value.

[0216] In some embodiments of this application, the reconstruction compensation configuration parameter set corresponding to the target activation function includes a sign configuration bit, a compensation configuration bit, and a compensation parameter. Specifically, the processing module 301 is used to reconstruct the piecewise linear approximation of the mapping based on the sign configuration bit using the conditional inverter of the reconstruction compensation unit, and output a first intermediate result; and to compensate the first intermediate result based on the compensation configuration bit using the ReLU compensator of the reconstruction compensation unit, and output a second intermediate result; and to add the compensation parameter and the second intermediate result using the first adder of the reconstruction compensation unit, and output the result as the function value.

[0217] In some embodiments of this application, the processing module 301 is further configured to obtain symbol information from the input independent variables through the input processing unit before reconstructing the piecewise linear approximation value based on the symbol configuration bit through the conditional inverter of the reconstruction compensation unit and outputting the first intermediate result. Specifically, the processing module 301 is configured to output the piecewise linear approximation value as the first intermediate result when the symbol configuration bit is a first value, through the first judge of the conditional inverter; or, when the symbol configuration bit is a second value, through the first judge, output the piecewise linear approximation value to the controlled inverter of the symbol processor of the conditional inverter, and invert the symbol information through the inverting unit of the symbol processor, and output the first intermediate result based on the inverted symbol information and the piecewise linear approximation value through the controlled inverter.

[0218] In some embodiments of this application, the processing module 301 is specifically used to output the mapped piecewise linear approximation value as a first intermediate result when the symbol information after symbol inversion is the same as the symbol of the target interval; or, when the symbol information after symbol inversion is different from the symbol of the target interval, to invert the symbol of the mapped piecewise linear approximation value again and output the mapped piecewise linear approximation value after inversion as a first intermediate result.

[0219] In some embodiments of this application, the processing module 301 is specifically used to output the first intermediate result as the second intermediate result when the compensation configuration bit is a third value, through the second judge of the ReLU compensator; or, through the second judge when the compensation configuration bit is a fourth value, output the first intermediate result to the second adder of the ReLU compensator, and generate ReLU compensation parameters according to the input independent variables through the ReLU generation unit of the compensation processor of the ReLU compensator, and add the first intermediate result and the ReLU compensation parameters through the second adder of the compensation processor, and output the result of the addition as the second intermediate result.

[0220] The computing device in this application embodiment can be an electronic device or a component within an electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal or other devices besides a terminal. For example, the electronic device can be a mobile phone, tablet computer, laptop computer, PDA, in-vehicle electronic device, mobile internet device (MID), augmented reality (AR) / virtual reality (VR) device, robot, wearable device, ultra-mobile personal computer (UMPC), netbook, or personal digital assistant (PDA), etc. It can also be a server, network attached storage (NAS), personal computer (PC), television set (TV), ATM, or self-service machine, etc. This application embodiment does not specifically limit the scope of the device.

[0221] The computing device in this application embodiment can be a device with an operating system. The operating system can be the Android operating system, the iOS operating system, or other possible operating systems; this application embodiment does not specifically limit the specific operating system.

[0222] The computing device provided in this application embodiment is capable of realizing Figures 6 to 7 The various processes implemented in the method implementation examples will not be described again here to avoid repetition.

[0223] In some embodiments of this application, such as Figure 9 As shown, this application embodiment also provides an electronic device 400, including a processor 401 and a memory 402. The memory 402 stores a program or instructions that can run on the processor 401. When the program or instructions are executed by the processor 401, they implement the various steps of the above-described calculation method embodiment and can achieve the same technical effect. To avoid repetition, they will not be described again here.

[0224] It should be noted that the electronic devices in the embodiments of this application include the aforementioned mobile electronic devices and non-mobile electronic devices.

[0225] Figure 10 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of this application.

[0226] The electronic device 500 includes, but is not limited to, components such as: radio frequency unit 501, network module 502, audio output unit 503, input unit 504, sensor 505, display unit 506, user input unit 507, interface unit 508, memory 509, and processor 510.

[0227] Those skilled in the art will understand that the electronic device 500 may also include a power supply (such as a battery) for supplying power to various components. The power supply may be logically connected to the processor 510 through a power management system, thereby enabling functions such as managing charging, discharging, and power consumption through the power management system. Figure 10 The electronic device structure shown does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown, or combine certain components, or have different component arrangements, which will not be elaborated here.

[0228] The processor 510 is used to determine the absolute value of the input independent variable, and determine the mapping independent variable based on the absolute value of the independent variable. This mapping independent variable belongs to the target interval with a single symbol. It also outputs the mapped piecewise linear approximation value based on the piecewise linear parameter set corresponding to the mapped independent variable and the target activation function. The domain configured by this piecewise linear parameter set is the target interval. Furthermore, it performs reconstruction compensation on the mapped piecewise linear approximation value based on the reconstruction compensation configuration parameter set corresponding to the target activation function, and outputs the function value of the target activation function under the input independent variable.

[0229] This application provides an electronic device. The electronic device's computing circuit can determine the absolute value of the input independent variable through an input processing unit and determine the mapping independent variable based on the absolute value. This maps input independent variables belonging to regions with arbitrary signs to target intervals belonging to a single sign, instead of directly using input independent variables belonging to intervals including both positive and negative signs. In this way, the computing circuit can output a mapped piecewise linear approximation value through a piecewise linear approximation unit based on the mapped independent variable and the piecewise linear parameter set corresponding to the target activation function (the domain of this piecewise linear parameter set is the target interval with a single sign). Furthermore, when the configured domain is the target interval with a single sign, the number of parameters in this piecewise linear parameter set is less than when the configured domain includes both positive and negative signs. Given the target interval, the piecewise linear parameter set has a smaller number of parameters. In other words, the piecewise linear approximation unit can output a mapped piecewise linear approximation value based on the mapped independent variable and the piecewise linear parameter set with a smaller number of parameters. Moreover, the computational circuit can reconstruct and compensate the mapped piecewise linear approximation value through the reconstruction compensation configuration parameter set corresponding to the target activation function, and output the function value of the target activation function under the input independent variable. That is to say, the reconstruction compensation unit can output an accurate function value. Therefore, while ensuring the accuracy of the output function value, the number of parameters in the piecewise linear parameter set used for computation can be reduced, thereby reducing the number of parameters stored in the computational circuit, reducing the size of the computational circuit, and further reducing the size of the chip containing the computational circuit, thus facilitating chip deployment.

[0230] In some embodiments of this application, the processor 510 is further configured to, before the piecewise linear approximation unit of the computing circuit outputs the mapped piecewise linear approximation value based on the piecewise linear parameter set corresponding to the target activation function in at least one activation function, and to provide the reconstruction compensation configuration parameter set corresponding to the target activation function to the piecewise linear approximation unit through the register group of the computing circuit; wherein the register group stores the piecewise linear parameter set corresponding to at least one activation function and the reconstruction compensation configuration parameter set.

[0231] In some embodiments of this application, the processor 510 is further configured to: determine a reconstruction compensation configuration parameter set corresponding to a first activation function based on a first activation function and a first cofunction, wherein the first cofunction is odd-symmetric or even-symmetric about zeros, before providing the piecewise linear parameter set corresponding to a target activation function to the piecewise linear approximation unit through the register group of the computing circuit; sample the first cofunction numerically in the target interval to obtain at least one target data point; perform piecewise linear approximation processing based on the at least one target data point and the first cofunction to obtain a piecewise linear parameter set corresponding to the first activation function; and store the piecewise linear parameter set corresponding to the first activation function and the reconstruction compensation configuration parameter set in the register group; wherein the first activation function is any one of at least one activation function.

[0232] In some embodiments of this application, the above-mentioned piecewise linear parameter set includes at least one set of parameters, each set of parameters including piecewise linear interval information, slope information and intercept information, wherein the piecewise linear interval information is used to indicate the piecewise sub-intervals within the target interval.

[0233] The processor 510 is specifically configured to: determine the target segmented sub-interval where the mapping independent variable is located from the segmented sub-intervals indicated by the segmented linear interval information in at least one set of parameters using the breakpoint comparator array of the segmented linear approximation unit, and output the sub-interval identifier of the target segmented sub-interval; determine the target slope information and target intercept information corresponding to the target segmented sub-interval from the slope information and intercept information in at least one set of parameters using the parameter selector of the segmented linear approximation unit according to the sub-interval identifier, and output the target slope information and target intercept information; and calculate the product value of the mapping independent variable and the target slope information using the linear synthesizer of the segmented linear approximation unit, add the product value and the target intercept information, and output the result as the mapped segmented linear approximation value.

[0234] In some embodiments of this application, the set of reconstruction compensation configuration parameters corresponding to the target activation function includes symbol configuration bits, compensation configuration bits, and compensation parameters.

[0235] The processor 510 described above is specifically used to reconstruct the piecewise linear approximation of the mapping according to the sign configuration bit by using the conditional inverter of the reconstruction compensation unit, and output a first intermediate result; and to compensate the first intermediate result according to the compensation configuration bit by using the ReLU compensator of the reconstruction compensation unit, and output a second intermediate result; and to add the compensation parameter and the second intermediate result by using the first adder of the reconstruction compensation unit, and output the result of the addition as a function value.

[0236] In some embodiments of this application, the processor 510 is further configured to obtain sign information from the input independent variables through the input processing unit before reconstructing the piecewise linear approximation of the mapping according to the sign configuration bit by the conditional inverter of the reconstruction compensation unit and outputting the first intermediate result.

[0237] The processor 510 is specifically configured to, through the first determiner of the conditional inverter, output the piecewise linear approximation of the mapping as a first intermediate result when the value of the symbol configuration bit is a first value; or, through the first determiner, when the value of the symbol configuration bit is a second value, output the piecewise linear approximation of the mapping to the controlled inverter of the symbol processor of the conditional inverter, and through the inverting unit of the symbol processor, invert the symbol information, and through the controlled inverter, output the first intermediate result based on the inverted symbol information and the piecewise linear approximation of the mapping.

[0238] In some embodiments of this application, the processor 510 is specifically configured to output the mapped piecewise linear approximation as a first intermediate result when the symbol information after symbol inversion is the same as the symbol of the target interval; or, when the symbol information after symbol inversion is different from the symbol of the target interval, to invert the symbol of the mapped piecewise linear approximation again and output the mapped piecewise linear approximation after inversion as the first intermediate result.

[0239] In some embodiments of this application, the processor 510 is specifically configured to, through the second determiner of the ReLU compensator, output the first intermediate result as the second intermediate result when the compensation configuration bit is a third value; or, through the second determiner, output the first intermediate result to the second adder of the ReLU compensator when the compensation configuration bit is a fourth value, and generate ReLU compensation parameters according to the input independent variables through the ReLU generation unit of the compensation processor of the ReLU compensator, and add the first intermediate result and the ReLU compensation parameters through the second adder of the compensation processor, and output the result of the addition as the second intermediate result.

[0240] It should be understood that, in this embodiment, the input unit 504 may include a graphics processing unit (GPU) 5041 and a microphone 5042. The GPU 5041 processes image data of still images or videos obtained by an image capture device (such as a camera) in video capture mode or image capture mode. The display unit 506 may include a display panel 5061, which may be configured in the form of a liquid crystal display, an organic light-emitting diode, or the like. The user input unit 507 includes at least one of a touch panel 5071 and other input devices 5072. The touch panel 5071 is also called a touch screen. The touch panel 5071 may include a touch detection device and a touch controller. Other input devices 5072 may include, but are not limited to, physical keyboards, function keys (such as volume control buttons, power buttons, etc.), trackballs, mice, and joysticks, which will not be described in detail here.

[0241] The memory 509 can be used to store software programs and various data. The memory 509 may primarily include a first storage area for storing programs or instructions and a second storage area for storing data. The first storage area may store the operating system, application programs or instructions required for at least one function (such as sound playback, image playback, etc.). Furthermore, the memory 509 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct memory bus RAM (DRRAM). The memory 509 in this embodiment includes, but is not limited to, these and any other suitable types of memory.

[0242] Processor 510 may include one or more processing units; optionally, processor 510 integrates an application processor and a modem processor, wherein the application processor mainly handles operations involving the operating system, user interface, and applications, and the modem processor mainly handles wireless communication signals, such as a baseband processor. It is understood that the aforementioned modem processor may also not be integrated into processor 510.

[0243] This application also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the above-described calculation method embodiments and achieve the same technical effect. To avoid repetition, they will not be described again here.

[0244] The processor mentioned above is the processor in the electronic device described in the above embodiments. The readable storage medium mentioned above includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0245] This application embodiment also provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run programs or instructions to implement the various processes of the above-described calculation method embodiments and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0246] It should be understood that the chips mentioned in the embodiments of this application may include, but are not limited to, at least one of the following: neural network processing unit, application-specific integrated circuit, field-programmable gate array, central processing unit, system-on-a-chip, etc., and this application does not specifically limit them.

[0247] This application provides a computer program product that is stored in a storage medium and executed by at least one processor to implement the various processes of the above-described calculation method embodiments, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0248] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0249] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods of the various embodiments of this application.

[0250] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. A computing circuit, characterized in that, include: An input processing unit is used to determine the absolute value of the input independent variable and determine the mapping independent variable based on the absolute value of the independent variable, wherein the mapping independent variable belongs to a target interval with a single symbol. A piecewise linear approximation unit, connected to the input processing unit, is used to output a mapped piecewise linear approximation value based on the piecewise linear parameter set corresponding to the mapping independent variable and the target activation function, wherein the domain configured for the piecewise linear parameter set is the target interval; The reconstruction compensation unit, connected to the piecewise linear approximation unit, is used to reconstruct and compensate the mapped piecewise linear approximation value according to the reconstruction compensation configuration parameter set corresponding to the target activation function, and output the function value of the target activation function under the input independent variable.

2. The computing circuit according to claim 1, characterized in that, The computing circuit also includes: A register group is connected to the piecewise linear approximation unit and the reconstruction compensation unit. The register group stores at least one piecewise linear parameter set and reconstruction compensation configuration parameter set corresponding to an activation function. The register group is used to provide the piecewise linear parameter set corresponding to the target activation function in the at least one activation function to the piecewise linear approximation unit, and to provide the reconstruction compensation configuration parameter set corresponding to the target activation function to the reconstruction compensation unit.

3. The computing circuit according to claim 1 or 2, characterized in that, The piecewise linear parameter set includes at least one set of parameters, each set of parameters including piecewise linear interval information, slope information and intercept information, wherein the piecewise linear interval information is used to indicate the piecewise sub-intervals within the target interval; The piecewise linear approximation unit includes: A breakpoint comparator array, connected to the input processing unit, is used to determine the target segmented sub-interval where the mapping independent variable is located from the segmented sub-intervals indicated by the segmented linear interval information in the at least one set of parameters, and output the sub-interval identifier of the target segmented sub-interval; A parameter selector, connected to the breakpoint comparator array, is used to determine, based on the sub-interval identifier, the target slope information and target intercept information corresponding to the target segmented sub-interval from the slope information and intercept information in the at least one set of parameters, and output the target slope information and the target intercept information; A linear synthesizer, connected to the input processing unit and the parameter selector respectively, is used to calculate the product of the mapping independent variable and the target slope information, add the product value and the target intercept information, and output the result as the mapping piecewise linear approximation value.

4. The computing circuit according to claim 1 or 2, characterized in that, The set of reconstruction compensation configuration parameters corresponding to the target activation function includes symbol configuration bits, compensation configuration bits, and compensation parameters; The reconstruction compensation unit includes: The conditional inverter is connected to the piecewise linear approximation unit; A ReLU compensator is connected to the conditional inverter; The first adder is connected to the ReLU compensator; The conditional inverter is used to reconstruct the piecewise linear approximation of the mapping based on the sign configuration bit and output a first intermediate result. The ReLU compensator is used to compensate the first intermediate result according to the compensation configuration bit and output the second intermediate result; The first adder is used to add the compensation parameter and the second intermediate result, and output the result as the function value.

5. The computing circuit according to claim 4, characterized in that, The input processing unit is also used to obtain symbol information from the input independent variables; The conditional inverter includes: The first judge is connected to the piecewise linear approximation unit and the ReLU compensator; The symbol processor includes an inversion unit and a controlled inverter. The inversion unit is connected to the input processing unit, and the controlled inverter is connected to the inversion unit, the first determiner, and the ReLU compensator. Wherein, the first determiner is used to output the mapped piecewise linear approximation as the first intermediate result when the value of the sign configuration bit is a first value; or, when the value of the sign configuration bit is a second value, to output the mapped piecewise linear approximation to the controlled inverter. The inverting unit is used to invert the symbol information; The controlled inverter is used to output the mapped piecewise linear approximation as the first intermediate result when the symbol information indicator after inverting the symbol is the same as the symbol of the target interval; or, when the symbol information indicator after inverting the symbol is different from the symbol of the target interval, to invert the symbol of the mapped piecewise linear approximation again and output the mapped piecewise linear approximation after inverting the symbol again as the first intermediate result.

6. The computing circuit according to claim 4, characterized in that, The ReLU compensator includes: The second judgment unit is connected to the condition inverter and the first adder. The compensation processor includes a ReLU generation unit and a second adder, wherein the ReLU generation unit is connected to the second adder, and the second adder is connected to a second judge and a first adder. Wherein, the second judge is used to output the first intermediate result as the second intermediate result when the value of the compensation configuration bit is the third value; or, when the value of the compensation configuration bit is the fourth value, to output the first intermediate result to the second adder. The ReLU generation unit is used to generate ReLU compensation parameters based on the input independent variables; The second adder is used to add the first intermediate result and the ReLU compensation parameter, and output the result as the second intermediate result.

7. A chip, characterized in that, It includes the computing circuit as described in any one of claims 1 to 6.

8. A calculation method, characterized in that, The method, applied to a computing circuit as described in any one of claims 1 to 6, comprises: The absolute value of the input independent variable is determined by the input processing unit of the computing circuit, and the mapping independent variable is determined based on the absolute value of the independent variable, wherein the mapping independent variable belongs to the target interval of a single symbol. The piecewise linear approximation unit of the computing circuit outputs a mapped piecewise linear approximation value based on the piecewise linear parameter set corresponding to the mapped independent variable and the target activation function. The domain configured for the piecewise linear parameter set is the target interval. The reconstruction compensation unit of the computing circuit reconstructs and compensates the piecewise linear approximation of the mapping according to the reconstruction compensation configuration parameter set corresponding to the target activation function, and outputs the function value of the target activation function under the input independent variable.

9. The method according to claim 8, characterized in that, Before the piecewise linear approximation unit of the computational circuit outputs the mapped piecewise linear approximation value based on the piecewise linear parameter set corresponding to the mapped independent variable and the target activation function, the method further includes: The register group of the computing circuit provides the piecewise linear parameter set corresponding to the target activation function in at least one activation function to the piecewise linear approximation unit, and provides the reconstruction compensation configuration parameter set corresponding to the target activation function to the reconstruction compensation unit. The register group stores the piecewise linear parameter set and the reconstruction compensation configuration parameter set corresponding to the at least one activation function.

10. The method according to claim 9, characterized in that, Before providing the piecewise linear parameter set corresponding to the target activation function in at least one activation function to the piecewise linear approximation unit via the register group of the computation circuit, the method further includes: Based on the first activation function and the first cofactor, determine the set of reconstruction compensation configuration parameters corresponding to the first activation function, wherein the first cofactor is odd-symmetric or even-symmetric about the zero point; Numerical sampling is performed on the first cofunction in the target interval to obtain at least one target data point; Based on the at least one target data point and the first cofunction, a piecewise linear approximation process is performed to obtain the piecewise linear parameter set corresponding to the first activation function; Store the piecewise linear parameter set and the reconstruction compensation configuration parameter set corresponding to the first activation function into the register group; Wherein, the first activation function is any one of the at least one activation function.

11. The method according to any one of claims 8 to 10, characterized in that, The piecewise linear parameter set includes at least one set of parameters, each set of parameters including piecewise linear interval information, slope information and intercept information, wherein the piecewise linear interval information is used to indicate the piecewise sub-intervals within the target interval; The piecewise linear approximation unit of the computing circuit outputs a mapped piecewise linear approximation value based on the piecewise linear parameter set corresponding to the mapped independent variable and the target activation function, including: Using the breakpoint comparator array of the piecewise linear approximation unit, the target piecewise sub-interval where the mapping independent variable is located is determined from the piecewise sub-intervals indicated by the piecewise linear interval information in the at least one set of parameters, and the sub-interval identifier of the target piecewise sub-interval is output. Using the parameter selector of the piecewise linear approximation unit, the target slope information and target intercept information corresponding to the target piecewise sub-interval are determined from the slope information and intercept information in the at least one set of parameters according to the sub-interval identifier, and the target slope information and target intercept information are output. The linear synthesizer of the piecewise linear approximation unit calculates the product of the mapping independent variable and the target slope information, adds the product value to the target intercept information, and outputs the result as the mapping piecewise linear approximation value.

12. The method according to any one of claims 8 to 10, characterized in that, The set of reconstruction compensation configuration parameters corresponding to the target activation function includes symbol configuration bits, compensation configuration bits, and compensation parameters; The reconstruction compensation unit of the computing circuit reconstructs and compensates the piecewise linear approximation of the mapping according to the reconstruction compensation configuration parameter set corresponding to the target activation function, and outputs the function value of the target activation function under the input independent variable, including: The conditional inverter of the reconstruction compensation unit reconstructs the piecewise linear approximation of the mapping based on the sign configuration bit, and outputs the first intermediate result. The ReLU compensator of the reconstruction compensation unit compensates the first intermediate result according to the compensation configuration bit, and outputs the second intermediate result. The first adder of the reconstruction compensation unit adds the compensation parameter and the second intermediate result, and outputs the result as the function value.

13. The method according to claim 12, characterized in that, Before reconstructing the piecewise linear approximation of the mapping based on the sign configuration bit using the conditional inverter of the reconstruction compensation unit and outputting the first intermediate result, the method further includes: The input processing unit obtains symbolic information from the input independent variables. The conditional inverter of the reconstruction compensation unit reconstructs the piecewise linear approximation of the mapping based on the sign configuration bit, and outputs a first intermediate result, including: The first determiner of the conditional inverter outputs the piecewise linear approximation of the mapping as the first intermediate result when the sign configuration bit is set to a first value; or, When the symbol configuration bit is set to the second value, the first determiner outputs the mapping piecewise linear approximation value to the controlled inverter of the symbol processor of the conditional inverter, and performs symbol inversion on the symbol information through the inversion unit of the symbol processor. The first intermediate result is then output through the controlled inverter based on the symbol information after symbol inversion and the mapping piecewise linear approximation value.

14. The method according to claim 13, characterized in that, The step of outputting the first intermediate result based on the symbol information after symbol inversion and the piecewise linear approximation of the mapping includes: If the symbol information after symbol inversion indicates that the symbol is the same as the symbol of the target interval, the piecewise linear approximation of the mapping is output as the first intermediate result; or, If the symbol information after symbol inversion is different from the symbol of the target interval, the symbol of the mapped piecewise linear approximation is inverted again, and the mapped piecewise linear approximation after inversion is output as the first intermediate result.

15. The method according to claim 12, characterized in that, The ReLU compensator of the reconstruction compensation unit compensates the first intermediate result according to the compensation configuration bits and outputs a second intermediate result, including: The second determiner of the ReLU compensator outputs the first intermediate result as the second intermediate result when the compensation configuration bit takes a third value; or, When the compensation configuration bit is set to the fourth value, the second judge outputs the first intermediate result to the second adder of the ReLU compensator. The ReLU generation unit of the compensation processor of the ReLU compensator generates ReLU compensation parameters based on the input variables. The second adder of the compensation processor adds the first intermediate result and the ReLU compensation parameters, and outputs the result as the second intermediate result.

16. A computing device, characterized in that, include: The processing module is used to determine the absolute value of the input independent variable and determine the mapping independent variable based on the absolute value of the independent variable, wherein the mapping independent variable belongs to the target interval of a single symbol. The system outputs a piecewise linear approximation value based on the piecewise linear parameter set corresponding to the mapping independent variable and the target activation function, wherein the domain of the piecewise linear parameter set is the target interval; and it performs reconstruction compensation on the piecewise linear approximation value based on the reconstruction compensation configuration parameter set corresponding to the target activation function, and outputs the function value of the target activation function under the input independent variable.

17. An electronic device, characterized in that, It includes the computing circuitry as described in any one of claims 1 to 6, or the chip as described in claim 7.

Images