Random computing processing unit and method based on adaptive compensation mechanism

Abstract

The present application relates to a random computing processing unit and method based on an adaptive compensation mechanism, which replaces the traditional fixed compensation rule with an adaptive compensation core solidified by a lightweight neural network, and the adaptive compensation core can predict an optimal compensation parameter in real time and dynamically according to two operands participating in multiplication operation to perform compensation. Through the data-driven mode, the scheme realizes accurate, continuous and personalized compensation of the calculation error of random calculation, and through the constant coefficient multiplier technology, the adaptive compensation core has extremely low area and power consumption overhead in hardware. Compared with the prior art, the fidelity of random calculation can be greatly improved without significantly increasing the hardware cost, and a new effective way is provided for constructing a high-performance and high-energy-efficiency random computing neural network accelerator.

Description

Technical Field

[0001] The present invention relates to the technical field of semiconductor integrated circuits, and in particular to a random calculation processing unit and method based on an adaptive compensation mechanism. Background Art

[0002] Stochastic computing (SC) is an emerging computing paradigm whose core advantage lies in its ability to implement multiplication operations using minimalist logic circuits, such as a single AND gate. This holds great potential for building low-power, small-area neural network accelerators. However, the accuracy of stochastic computing has long been limited by inherent computational errors. These errors primarily include: inherent precision loss, whereby high-bit-width product information is compressed into low-bit-width count values ​​during the computation process, resulting in irreversible information loss; and scaling error, whereby the conversion between stochastic bit streams and binary values ​​requires scaling operations. Improper handling of the scaling factor can introduce significant nonlinear distortion.

[0003] To address scaling error, a piecewise-linear scaling compensation (PWSC) method has been proposed. This method divides the input into different intervals using a preset threshold and assigns a fixed compensation parameter to each interval. Although existing methods such as PWSC attempt to mitigate scaling error, they suffer from three major drawbacks and are unable to address the inherent loss of precision mentioned above. First, the compensation model is imprecise. Based on fixed rules, it cannot accurately fit complex nonlinear errors that are highly correlated with the input values. Second, the compensation mechanism is discontinuous, with the compensation parameter "jumping" at segment boundaries, introducing new quantization errors. Third, the compensation method is non-personalized, using the same compensation parameter for all input combinations within a region, making optimal compensation impossible. Therefore, a new technology solution is urgently needed that can simultaneously and more effectively address the two major issues of inherent loss of precision and scaling error in stochastic computing, while also enabling accurate, continuous, and adaptive compensation. Summary of the Invention

[0004] Based on this, it is necessary to provide a random computing processing unit based on an adaptive compensation mechanism and a random computing method based on an adaptive compensation mechanism, which can simultaneously and more effectively solve the two major problems of inherent precision loss and scaling error in random computing, and achieve accurate, continuous and adaptive compensation.

[0005] To achieve the above objectives, the embodiments of the present invention adopt the following technical solutions:

[0006] On the one hand, a random computing processing unit based on an adaptive compensation mechanism is provided, comprising a parallel main data path and a side data path:

[0007] The main data path includes a random number generator, a multiplier array, an adder tree circuit, and an accumulator. The random number generator is used to convert the 7-bit unsigned values ​​of the two input operands into a 128-bit random bit stream. The multiplier array is used to perform parallel multiplication operations on the random bit stream and output a product bit stream. The adder tree circuit is used to count the bits with a value of 1 in the product bit stream and output an initial result. The accumulator is used to accumulate the compensated initial results and output a final multiplication result.

[0008] The side data path includes a decoder, an adaptive compensation core, a compensation correction unit, a sign multiplication logic and a complement converter. The decoder is used to decode the two input operands into a 1-bit sign bit and a 7-bit unsigned value respectively. The adaptive compensation core is used to calculate and output a 4-bit compensation parameter in real time based on the 7-bit unsigned values ​​of the two operands. The compensation correction unit is used to perform a left shift operation on the initial result according to the compensation parameter to obtain the compensated initial result. The sign multiplication logic is used to generate the product sign based on the 1-bit sign bit of the two operands and then send it to the accumulator through the complement converter. The adaptive compensation core is a hardware logic circuit solidified from a pre-trained feedforward fully connected neural network, and is pre-trained using a training data set consisting of operand pairs and ideal compensation parameters.

[0009] In one embodiment, the adaptive compensation kernel includes an input layer, a hidden layer, and an output layer, the input layer including a first input node and a second input node for respectively receiving 7-bit unsigned numerical values ​​of two operands, the hidden layer including at least one hidden neuron node for performing neuronal inference based on the 7-bit unsigned numerical values ​​of the two operands, and the output layer including an output neuron node for outputting compensation parameters.

[0010] In one embodiment, the hidden layer includes four hidden neuron nodes, and the activation function of the hidden layer is a rectified linear unit.

[0011] In one embodiment, all multiplication operations of variables by constants in the adaptive compensation core are implemented using constant coefficient multipliers.

[0012] On the other hand, a random calculation method based on an adaptive compensation mechanism is further provided. The random calculation processing unit based on the adaptive compensation mechanism according to any of the above items is used, and the random calculation method comprises the following steps:

[0013] Get the first and second binary operands;

[0014] Inputting the first operand and the second operand into the main data path to perform a random multiplication operation to obtain an initial result;

[0015] After calculating compensation parameters for the first operand and the second operand input side data path, compensation correction is performed on the initial result according to the compensation parameters to obtain a compensated initial result;

[0016] The compensated initial results are accumulated through the main data path to obtain the final multiplication result.

[0017] One of the above technical solutions has the following advantages and beneficial effects:

[0018] The above-mentioned random computing processing unit and method based on the adaptive compensation mechanism can generate a continuous and smooth compensation response surface by adopting a data-driven neural network prediction model to replace the compensation mechanism based on fixed rules in the existing technology. This not only fundamentally avoids the boundary jump error introduced by segmented processing, but also provides a "tailor-made" refined compensation scheme for each specific pair of input operands; this refined compensation capability enables the random computing processing unit to correct the scaling error while also fitting and correcting the inherent precision loss caused by information compression to a certain extent, ultimately achieving high fidelity of the calculation results, greatly improving the overall performance of random computing based on the adaptive compensation mechanism. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The drawings constituting a part of this application are used to provide a further understanding of the present invention. The illustrative embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0021] Figure 1 Schematic diagram of the structure of a random calculation processing unit based on an adaptive compensation mechanism in one embodiment;

[0022] Figure 2 A schematic diagram of the internal logic structure of an adaptive compensation core in one embodiment;

[0023] Figure 3 Schematic diagram of the offline training process of the adaptive compensation kernel in one embodiment;

[0024] Figure 4 FIG. 1 is a schematic diagram illustrating the principle improvement effect of the compensation mechanism in one embodiment; wherein, Figure 4 (a) a compensation relationship for prior art, Figure 4 (b) the compensation relationship of the present invention;

[0025] Figure 5 FIG. 1 is a diagram showing a significant reduction in calculation error in one embodiment;

[0026] Figure 6 FIG1 is a diagram showing a significant improvement in error distribution in one embodiment;

[0027] Figure 7 Schematic diagram of a flow chart of a random calculation method based on an adaptive compensation mechanism in one embodiment. DETAILED DESCRIPTION

[0028] To make the above-mentioned objects, features, and advantages of the present invention more readily apparent, specific embodiments of the present invention are described in detail below with reference to the accompanying drawings. The following description sets forth numerous specific details to facilitate a full understanding of the present invention. However, the present invention can be implemented in many other ways than those described herein, and those skilled in the art may make similar modifications without departing from the scope of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0029] The following describes the implementation of the present invention in detail with reference to the accompanying drawings in the embodiments of the present invention.

[0030] In one embodiment, see Figure 1 A stochastic computation processing element (PE) 200 based on an adaptive compensation mechanism is provided. The PE includes a parallel main data path 210 and a side data path 220. The main data path 210 is configured to perform a stochastic computation multiplication operation on two 7-bit unsigned values ​​of input operands. The side data path 220 is configured to generate a 4-bit compensation parameter based on the two input operands and to compensate and correct an initial result of the multiplication operation based on the compensation parameter.

[0031] The main data path 210 includes a random number generator 211, a multiplier array 212, an adder tree circuit 213, and an accumulator 233. The random number generator 211 is used to convert the 7-bit unsigned values ​​of the two operands into a 128-bit random bit stream. The multiplier array 212 is used to perform parallel multiplication operations on the 128-bit random bit stream and output a product bit stream. The adder tree circuit 213 is used to count the bits with a value of 1 in the product bit stream and output an initial result. The accumulator 233 is used to accumulate the compensated initial results and output the final multiplication result.

[0032] The side data path 220 includes a decoder, an adaptive compensation core 100, a compensation correction unit 230, a sign multiplication logic 214, and a complement converter 232. The decoder is used to decode the two input operands into a 1-bit sign bit and a 7-bit unsigned value, respectively. The adaptive compensation core 100 is used to calculate and output a 4-bit compensation parameter in real time based on the 7-bit unsigned values ​​of the two operands. The compensation correction unit 230 is used to perform a left shift operation on the initial result based on the compensation parameter, and the compensated initial result is then sent to the accumulator 233. The sign multiplication logic 214 is used to generate the product sign based on the 1-bit sign bit of the two operands and then send it to the accumulator 233 through the complement converter 232. The adaptive compensation core 100 is a hardware logic circuit solidified from a pre-trained feedforward fully connected neural network. It is pre-trained using a training data set consisting of operand pairs and ideal compensation parameters.

[0033] As can be understood, the stochastic computation processing unit 200 integrates an adaptive compensation core 100 (also referred to as the AS-Net Core). This adaptive compensation core 100 is composed of a lightweight, offline-trained neural network circuit. Its input is connected to the processing unit's operand input path, and its output is connected to the processing unit's compensation and correction logic. During operation, the adaptive compensation core 100 runs in parallel with the stochastic computation main data path 210, providing a compensation parameter precisely matched to each multiplication operation in real time and dynamically.

[0034] The random calculation workflow implemented by the random calculation processing unit based on the adaptive compensation mechanism can be roughly described as the following steps: obtaining the binary first operand and second operand involved in the multiplication operation; inputting the first operand and the second operand into a pre-trained compensation parameter prediction model (implemented by the adaptive compensation core 100), the compensation parameter prediction model being a neural network model; the compensation parameter prediction model generates a compensation parameter for compensating for the random calculation error based on the currently input first operand and second operand; obtaining the initial result obtained after the random calculation multiplication operation of the first operand and the second operand; and using the compensation parameter generated by the compensation parameter prediction model to compensate for the initial result to obtain the final multiplication result.

[0035] The above-mentioned random computing processing unit 200 based on the adaptive compensation mechanism can generate a continuous and smooth compensation response surface by adopting a data-driven neural network prediction model to replace the compensation mechanism based on fixed rules in the existing technology. This not only fundamentally avoids the boundary jump error introduced by segmented processing, but also provides a "tailor-made" refined compensation scheme for each specific pair of input operands; this refined compensation capability enables the random computing processing unit to correct the scaling error while also fitting and correcting the inherent precision loss caused by information compression to a certain extent, ultimately achieving high fidelity of the calculation results, and greatly improving the overall performance of random computing based on the adaptive compensation mechanism.

[0036] More specifically, this embodiment solves the problem of inherent precision loss in a data-driven manner: since the generation process of the ideal compensation parameter (shift_ideal) is based on the optimization target of the final ideal product (P_ideal), during the training process, the neural network (such as the adaptive compensation kernel 100) not only learns how to compensate for the systematic scaling error caused by division by 128, but its internal nonlinear fitting capability also enables it to learn the non-systematic deviations related to the input value caused by random fluctuations and information compression; therefore, the compensation parameters finally generated actually include comprehensive compensation for the two errors, thereby being able to correct the inherent precision loss while correcting the scaling error, thereby achieving higher computational fidelity than traditional methods.

[0037] like Figure 1 As shown, an embodiment of the present invention provides a stochastic computation processing unit 200 integrated with an adaptive compensation core 100. The stochastic computation processing unit 200 is used to complete a complete stochastic computation multiply-accumulate (SC-MAC) operation with compensation. The unit includes a main data path 210 and a side data path 220, both of which operate in parallel. After receiving two binary inputs in INT8 format (i.e., the first operand and the second operand), the overall workflow of the stochastic computation processing unit 200 can be as follows:

[0038] First, each input operand (denoted as X_value[7:0]) is decoded into a 1-bit sign bit and a 7-bit unsigned value.

[0039] The main data path 210 is responsible for performing the multiplication operation of the random computation. The 7-bit unsigned values of the first and second operands are sent into the respective random number generators (X_SNG and W_SNG) 211, which convert the 7-bit unsigned values into 128-bit random bit streams (SN_X and SN_W). Subsequently, the two random bit streams are subjected to parallel multiplication operation by a multiplier array 212 (which can be composed of 128 AND gates, for example), to obtain a product bit stream. Then, a counting of the bits with value 1 in the product bit stream is performed by an addition tree circuit 213, to output an 8-bit unsigned preliminary result (which can be denoted as count_y).

[0040] Meanwhile, the side data path 220 is responsible for generating the compensation parameter. The 7-bit unsigned values of the first and second operands are directly sent into the core of the present application, the adaptive compensation core 100, after being decoded by a decoder. The adaptive compensation core 100 computes and outputs a 4-bit compensation parameter (which can be denoted as shift_pred) in real time according to the two 7-bit unsigned values by internal neural network inference. The X_sign and W_sign are the outputs after being processed by zero value bypass logic.

[0041] Finally, in the compensation correction unit 230, the output results of the main data path 210 and the side data path 220 are combined. The compensation parameter shift_pred is used to control a barrel shifter 231 to perform left shift operation on the unsigned preliminary result count_y, to obtain a compensated value (i.e. the compensated preliminary result). Meanwhile, the sign bits of the first and second operands are subjected to a sign multiplication logic (such as an XOR gate) 214 to obtain a product sign. The product sign will determine whether the compensated value is positive or negative (which can be converted by a complement converter 232) to be sent into a final accumulator 233 for accumulation, and the accumulation result is stored in a register to output the final multiplication result.

[0042] In an embodiment, the adaptive compensation core 100 comprises an input layer, a hidden layer and an output layer. The input layer comprises a first input node 111 and a second input node 112 for receiving the 7-bit unsigned values of the two operands, respectively. The hidden layer comprises at least one hidden neuron node for performing neuron inference according to the 7-bit unsigned values of the two operands. The output layer comprises an output neuron node 131 for outputting the compensation parameter.

[0043] It can be understood that the design and training implementation of the adaptive compensation core 100 can be as follows: as Figure 2The internal logic of the adaptive compensation core 100 is solidified by a lightweight feedforward fully connected neural network. The feedforward fully connected neural network includes an input layer, a hidden layer, and an output layer. The input layer includes a first input node 111 and a second input node 112, which are respectively used to receive the 7-bit unsigned values ​​of the two operands input by the previous circuit (denoted as X_bc and W_bc). The hidden layer includes at least one hidden neuron node. In this embodiment, Figure 2 As shown, four hidden neuron nodes may be selected, such as a first hidden neuron node H1, a second hidden neuron node H2, a third hidden neuron node H3 and a fourth hidden neuron node H4. The output layer includes an output neuron node 131 for outputting a 4-bit compensation parameter shift_pred.

[0044] In one embodiment, the activation function of the hidden layer is a rectified linear unit (ReLU), which can be implemented in hardware by a simple comparator. The output layer uses a linear activation, and its output can be further processed by saturation logic to ensure that its value is within a preset range (e.g., 0 to 15).

[0045] like Figure 3 As shown, the weights and bias parameters within the adaptive compensation core 100 are obtained through an independent offline training process. The offline training process first generates the corresponding ideal compensation parameters (which can be denoted as shift_ideal) for a large number of operand pairs (each operand pair consists of two operands) through high-precision random calculation simulation, thereby constructing a training dataset. Subsequently, a loss function such as mean square error (MSE) is used to train the training dataset through optimization algorithms such as gradient descent. Figure 2 The adaptive compensation core 100 shown here accurately fits the nonlinear mapping relationship from input operand pairs to ideal compensation parameters. After training, the floating-point parameters in the adaptive compensation core 100 (such as weights and bias parameters) are symmetrically quantized to the hardware-friendly INT4 format.

[0046] In one embodiment, all multiplication operations of variables by constants in the adaptive compensation core are implemented using constant coefficient multipliers.

[0047] It can be understood that the hardware implementation and optimization of the adaptive compensation core 100 can be as follows: in the hardware implementation of this embodiment, the quantized INT4 format weight and bias parameters will be used as constants during design and directly hard-coded into the hardware description language code describing the adaptive compensation core 100.

[0048] A key optimization involves implementing all "variable by constant" multiplication operations (for example, in the hidden layer, the multiplication of the output values ​​of the first input node 111 and the second input node 112 by the corresponding weights) using constant coefficient multipliers (KCMs). This allows existing synthesis tools to automatically optimize these multiplication operations into a series of combinational logic circuits consisting of shifts and additions, without the use of general-purpose multiplier logic. This optimization enables the adaptive compensation core 100 to be implemented as a pure hardware logic circuit with extremely low area and power consumption.

[0049] In some embodiments, the principle improvement effect of the compensation mechanism of the random calculation processing unit 200 based on the adaptive compensation mechanism is described as follows: Figure 4 As shown, Figure 4 (a) is the compensation relationship of the existing technology, which presents a discrete step shape and introduces errors at the segment boundaries. In contrast, Figure 4 (b) is the compensation relationship of the present invention, which generates a continuous and smooth compensation surface through neural network fitting, which can fit the error more accurately.

[0050] The significant reduction in calculation errors is as follows Figure 5 As shown in the figure, in a series of tests on small value multiplication scenarios, the present invention reduces the mean relative error (MRE) from 99.4% of the existing PWSC solution to 18.2%. At the same time, it suppresses the maximum relative error (MaxRE) from 700% to within 77.8%. Figure 5 The scheme name "AEC (Ours)" is used to represent the scheme of the present invention. The small value interval is defined as the multipliers X and W are both in the range of [1, 15].

[0051] Significant improvement in error distribution such as Figure 6 As shown, the error distribution of existing technologies (such as Default and PWSC) is relatively wide, while the error distribution of the present invention is more concentrated. The data shows that in small value tests, 24.9% of the samples with the existing PWSC solution have an error exceeding 100%, while the present invention reduces this proportion to zero, effectively avoiding the occurrence of extreme errors.

[0052] In summary, the present invention significantly improves the accuracy and stability of random calculations while maintaining low hardware overhead by improving the compensation mechanism.

[0053] In one embodiment, Figure 7 As shown, a random calculation method based on an adaptive compensation mechanism is also provided. Based on the random calculation processing unit based on the adaptive compensation mechanism of any of the above embodiments, the random calculation method may include the following steps S12 to S18:

[0054] S12, obtaining a first binary operand and a second binary operand;

[0055] S14, inputting the first operand and the second operand into the main data path to perform a random multiplication operation to obtain an initial result;

[0056] S16, after calculating compensation parameters for the first operand and the second operand input side data path, performing compensation correction on the initial result according to the compensation parameters to obtain a compensated initial result;

[0057] S18, accumulating the compensated initial results through the main data path to obtain a final multiplication result.

[0058] The above-mentioned random calculation method based on the adaptive compensation mechanism can generate a continuous and smooth compensation response surface by adopting a data-driven neural network prediction model to replace the compensation mechanism based on fixed rules in the prior art. This not only fundamentally avoids the boundary jump error introduced by segmented processing, but also provides a "tailor-made" refined compensation scheme for each specific pair of input operands. This refined compensation capability enables the present invention to correct the scaling error while also fitting and correcting the inherent precision loss caused by information compression to a certain extent, ultimately achieving high fidelity of the calculation results, greatly improving the overall performance of the random calculation method based on the adaptive compensation mechanism.

[0059] It can be understood that the explanation of the above-mentioned components can be understood by referring to the corresponding features of the above-mentioned random calculation processing unit based on the adaptive compensation mechanism, and will not be repeated in this embodiment.

[0060] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0061] The above embodiments merely illustrate several implementations of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention. It should be noted that a person skilled in the art would be able to make numerous variations and improvements without departing from the spirit of the present invention, and all such variations and improvements fall within the scope of protection of the present invention.

[0062] Furthermore, the terms "one" and "plurality" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features being referred to. Thus, a feature specified as "one" or "plurality" may explicitly or implicitly include at least one such feature. In the description of the present invention, "plurality" means at least two, such as two, three, etc., unless otherwise specifically defined.

[0063] In the present invention, unless otherwise specified or limited, the terms "connected" and "connection" should be understood in a broad sense. For example, "connected" and "connected" can refer to communication connection or electrical connection; direct connection or indirect connection through an intermediate medium; internal communication between two components or interaction between two components, unless otherwise specified. Those skilled in the art will understand the specific meanings of the above terms in the present invention based on specific circumstances.

Claims

1. A random computing processing unit based on an adaptive compensation mechanism, characterized in that: The invention comprises a parallel main data path and a side data path, wherein the main data path includes a random number generator, a multiplier array, an adder tree circuit and an accumulator, wherein the random number generator is used to convert the 7-bit unsigned values ​​of the two input operands into a 128-bit random bit stream, the multiplier array is used to perform parallel multiplication operations on the random bit stream and output a product bit stream, the adder tree circuit is used to count the bits with a value of 1 in the product bit stream and output an initial result, and the accumulator is used to accumulate the compensated initial results and output a final multiplication result; The side data path includes a decoder, an adaptive compensation core, a compensation correction unit, a sign multiplication logic, and a complement converter. The decoder is used to decode the two input operands into a 1-bit sign bit and a 7-bit unsigned value, respectively. The adaptive compensation core is used to calculate and output a 4-bit compensation parameter in real time based on the 7-bit unsigned values ​​of the two operands. The compensation correction unit is used to perform a left shift operation on the initial result according to the compensation parameter to obtain the compensated initial result. The sign multiplication logic is used to generate the product sign based on the 1-bit sign bit of the two operands and then send it to the accumulator through the complement converter. The adaptive compensation core is a hardware logic circuit solidified from a pre-trained feedforward fully connected neural network, and is pre-trained using a training data set consisting of operand pairs and ideal compensation parameters.

2. The random calculation processing unit based on the adaptive compensation mechanism according to claim 1, characterized in that: The adaptive compensation kernel includes an input layer, a hidden layer and an output layer. The input layer includes a first input node and a second input node for respectively receiving 7-bit unsigned numerical values ​​of two operands. The hidden layer includes at least one hidden neuron node for performing neuron inference according to the 7-bit unsigned numerical values ​​of the two operands. The output layer includes an output neuron node for outputting compensation parameters.

3. The random calculation processing unit based on the adaptive compensation mechanism according to claim 2, characterized in that: The hidden layer includes four hidden neuron nodes, and the activation function of the hidden layer is the rectified linear unit.

4. The random calculation processing unit based on the adaptive compensation mechanism according to any one of claims 1 to 3, characterized in that: All multiplication operations of variables by constants in the adaptive compensation core are implemented using constant coefficient multipliers.

5. A random computing method based on an adaptive compensation mechanism, characterized in that: Based on the random calculation processing unit based on the adaptive compensation mechanism according to any one of claims 1 to 4, the random calculation method comprises the following steps: Get the first and second binary operands; Inputting the first operand and the second operand into the main data path to perform a random multiplication operation to obtain an initial result; After calculating compensation parameters for the first operand and the second operand input side data path, compensation correction is performed on the initial result according to the compensation parameters to obtain a compensated initial result; The compensated initial results are accumulated through the main data path to obtain the final multiplication result.

Images