A method and apparatus for fixed-point multiply-add for edge ai chips

Abstract

This invention relates to the field of artificial intelligence chip technology, and in particular to a fixed-point multiply-accumulate method and apparatus for edge artificial intelligence chips. The method includes acquiring two signed low-bit operands and a shared signed common operand; performing a signed addition on the first operand and a second operand shifted left by a preset interval bit width to generate a packed operand; feeding the packed operand and the common operand into a single DSP hard core for multiplication to obtain a packed product, which is then continuously accumulated via an internal feedback path; in parallel, detecting the crossing state of the low-bit calculation channel and updating the compensation count value based on the sign of the low-bit remaining value before and after accumulation and the current sign of the first product; and after all accumulations are completed, restoring the two independent signed accumulation results based on the final packed accumulation result and the compensation count value. This invention improves the utilization rate of the DSP's internal data path, reduces external resource overhead, and helps maintain a higher operating frequency.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence chip technology, and in particular to a fixed-point multiply-accumulate method and apparatus for edge artificial intelligence chips. Background Technology

[0002] Edge AI chips need to perform real-time neural network inference under limited power consumption, area, and storage bandwidth. The core computations of networks such as convolutional neural networks and Transformers can be reduced to a large number of multiply-accumulate operations. To reduce storage and power consumption, current inference generally uses low-ratio specific-point quantization and often utilizes the DSP hard core in the FPGA to handle multiply-accumulate operations. Since the result of a single signed low-bit multiplication only requires a small bit width to represent, when the wide-bit DSP hard core only calculates one low-bit multiplication, a large number of data bits in its multiplier and accumulator are idle. To address this, existing methods package two low-bit data paths sharing the same operand into the same wide-bit multiplication path at a predetermined bit width interval, and obtain the two products through a single wide-bit multiplication.

[0003] However, existing low-bit DSP packing methods have the following drawbacks when applied to edge AI chips: First, limited support for sign types. In symmetric quantization, both activation values ​​and weights can be negative, as can convolution and attention calculations. Some existing methods only support multiplication of unsigned and signed numbers, or require common operands to be non-negative, failing to cover these scenarios. Second, difficulty in correctly separating path results. When the accumulated result of the low-bit channel exceeds the reserved bit width, it will change the high-bit field with carry or borrow. The two's complement sign extension of negative low-bit numbers will also intrude into the high-bit field. Directly splitting the accumulated result into fixed bit segments will lead to errors in both paths. Third, external correction overhead is large and restricts timing. Existing solutions require setting up multiple correction counters such as negative product, carry, and borrow outside the DSP, and splitting, correcting, and feeding back the output cycle by cycle. This consumes additional resources such as lookup tables, flip-flops, wide-bit adders, and wiring. It also introduces long combinational logic in the accumulation feedback path, reducing the operating frequency and energy efficiency. Furthermore, it contradicts the low power consumption and high throughput requirements of edge AI chips and lacks versatility. At present, a fixed-point multiply-accumulate method and device for edge AI chips is needed. Summary of the Invention

[0004] To address the technical problems of limited symbol type support, difficulty in correctly separating two-way accumulation results, and resource consumption and timing constraints of external cycle-by-cycle correction when existing low-bit DSP packing methods are applied to edge AI chips, this invention provides a fixed-point multiply-accumulate method and apparatus for edge AI chips.

[0005] In a first aspect, the present invention provides a fixed-point multiply-accumulate method for edge artificial intelligence chips, which adopts the following technical solution: A fixed-point multiply-accumulate method for edge AI chips includes: Obtain the first signed low-bit operand, the second signed low-bit operand, and the signed common operand shared by the two operations from the input item, and perform sign extension on each of them; According to the preset calculation channel interval bit width n, the second operand after sign expansion is shifted left by n bits, and a signed addition operation is performed with the first operand after sign expansion to generate a packaged operand; The packaged operand and the signed common operand are fed into a single DSP hard core for multiplication to obtain the packaged product. The packaged product is then continuously accumulated through the internal feedback path of the DSP hard core to obtain the packaged accumulation result. In parallel with continuous accumulation, based on the sign of the low-order residual value before accumulation, the sign of the current first-way product, and the sign of the low-order residual value after accumulation, the crossing state of the first-way accumulation result within the n-bit signed representation interval is detected, and the signed compensation count value is updated. After all input items are accumulated, the independent signed accumulation results of the first and second paths are restored based on the final packaged accumulation result and the final signed compensation count value, and a result valid signal is generated.

[0006] Furthermore, the sign extension includes copying and filling the sign bit of the first signed low-bit operand to the higher bits, thereby extending it to the data bit width of the DSP hard core input combination unit; The sign bit of the second signed low-bit operand is copied and filled to the high-bit side, so that after shifting it left by n bits, it can be accommodated within the data bit width of the input combination unit; The sign bit of the signed common operand is copied and filled to the higher bits, thus expanding it to the data bit width of the corresponding input of the DSP hard-core multiplier. The signed common operand after sign extension is input into the multiplier of the DSP hard core, so that the first signed low-bit operand and the second signed low-bit operand share the same signed common operand.

[0007] Furthermore, the generation of the packaged operands includes shifting the sign-extended second-way operand left by n bits according to a preset lane interval bit width to obtain the high-order calculation channel operand, performing a signed addition operation on the high-order calculation channel operand and the sign-extended first-way operand, and using the operation result as the signed packaged operand. The signed addition operation is performed according to the signed integer operation relationship. Based on the calculation channel interval width n, the first operand is set in the low-order calculation channel of the packaged operand, and the second operand is set in the high-order calculation channel of the packaged operand. The expression of the signed packaged operand is: , in, Let i be the signed packing operand corresponding to the i-th input item. For the first signed low-bit operand after sign extension, For the second-way signed low-bit operand after sign extension, The preset calculation channel interval bit width.

[0008] Furthermore, the step of continuously accumulating the packaged product through the internal feedback path of the DSP hard core includes using the arithmetic logic unit inside the DSP hard core to add the packaged product of the current cycle to the packaged accumulation result of the previous cycle stored in the accumulation register inside the DSP hard core, so as to obtain the packaged accumulation result of the current cycle. The current cycle's packed accumulation result is written to the accumulation register through the feedback path inside the DSP hard core for accumulation of the next input item; Repeat the addition and write operations until all input items have been processed, and the resulting packaged summation satisfies: , in, This is the packaged and accumulated result after the i-th operation is completed. This is the packaged and accumulated result after the operation of the (i-1)th term. Pack the i-th term into a product, and initialize it with the following values. .

[0009] Furthermore, the detection of the crossing state of the first-path accumulation result within the n-bit signed representation interval includes interpreting the lower n bits of the packaged accumulation result as a signed residual value, and extracting the sign of the signed residual value as the sign of the lower bit residual value after accumulation. Determine the signed compensation count value, whereby the signed compensation count value is the difference between the first-path true accumulation result and the signed residual value relative to... An integer multiple of is used to record the cumulative offset of the first-way accumulation result relative to the signed residual value. The expression for the signed compensation count value is: ; in, This is the signed compensation count value after the i-th operation is completed. This represents the true cumulative result of the first path for the i-th term. This is the signed residual value obtained by interpreting the lower n bits of the summation result of the i-th item.

[0010] Furthermore, the detection of the crossing state of the first-way accumulation result within the n-bit signed representation interval also includes determining the current sign of the first-way product based on the sign state of the first-way signed low-bit operands and the signed common operands. When both the first-way signed low-bit operand and the signed common operand are not 0, the current first-way product sign is obtained by XORing their sign bits; When either the first signed low-bit operand or the signed common operand is 0, the sign of the current first-way product is determined to be non-negative.

[0011] Furthermore, updating the signed compensation count value includes comparing the sign of the low-order remaining value before accumulation, the sign of the current first-way product, and the sign of the low-order remaining value after accumulation. When the low-order residual value before accumulation is non-negative, the current first-way product is non-negative, and the low-order residual value after accumulation is negative, it is determined that the first-way accumulation result positively crosses the upper boundary of the n-bit signed representation interval, and the signed compensation count value is incremented by 1. When the low-order residual value before accumulation is negative, the current first-way product is negative, and the low-order residual value after accumulation is non-negative, it is determined that the first-way accumulation result crosses the lower boundary of the n-bit signed representation interval in a negative direction, and the signed compensation count value is decremented by 1. In other symbol combinations, if the first-way accumulation result does not cross the boundary of the n-bit signed representation interval, the signed compensation count value remains unchanged, and the update amount of the signed compensation count value satisfies: , in, The amount of time to update the signed compensated count value. This is the symbol for the first product path. To accumulate the sign of the lowest-order remaining value, This is the sign of the lower-order residual value after accumulation.

[0012] Furthermore, the recovery of the independent signed accumulation results of the first and second paths includes interpreting the lower n bits of the final packaged accumulation result as the signed residual value of the first path after all input items have been accumulated; The final signed compensated count value is sign-extended, shifted left by n bits, and added to the first-path signed residual value to obtain the first-path true accumulation result. The expression for the first-path true accumulation result is: , in, This is the result of the first path's true cumulative sum. For the final packaged and accumulated result, The preset calculation channel interval bit width, For the final signed compensated count value, To sum the final packaged results The lower n bits are interpreted as signed residuals.

[0013] Furthermore, the recovery of the independent signed accumulation results of the first and second paths also includes, after all input items have been accumulated, interpreting the high-order field of the final packaged accumulation result as a signed number and extracting the (n-1)th bit of the final packaged accumulation result; The second-way true accumulation result is calculated based on the signed number, the (n-1)th bit, and the final signed compensation count value. The (n-1)th bit is used to compensate for the bit weight difference generated when the lower n bits are converted from unsigned fields to signed residual values. The final signed compensation count value is used to eliminate the occupation of the higher bits caused by the cumulative offset of the first-way accumulation result. The expression for the second path's actual cumulative result is: , in, This is the actual cumulative result of the second path. To increase the bit width of the DSP accumulation result, This means that the high-order bits of the final packaged summation result are interpreted as signed numbers. This is the (n-1)th element of the final packaged and accumulated result.

[0014] Secondly, a fixed-point multiply-accumulate device for edge artificial intelligence chips includes: The input interface is used to receive the first signed low-bit operand, the second signed low-bit operand, the signed common operand shared by the two operations, the input valid signal, the reset signal, and the operation end signal; The signed input processing module is used to perform sign extension on operands and generate packed operands; The DSP multiply-accumulate module includes a single DSP hard core, which integrates a pre-adder, a multiplier, and an accumulator to generate the current cycle's packaged accumulation result. The signed-Q compensation module is used to update the signed compensation count value based on the sign of the low-order residual value before accumulation, the current first-way product sign, and the sign of the low-order residual value after accumulation. The result recovery module is used to recover the independent signed accumulation results of the first and second paths based on the final packaged accumulation result and the final signed compensation count value, and to generate a result valid signal. The control module is used to control the timing of the signed input processing module, the DSP multiply-accumulate module, the signed compensation module, and the result recovery module based on the input valid signal, the reset signal, and the operation end signal.

[0015] In summary, the present invention has the following beneficial technical effects: 1. This invention packages two signed low-bit operands into a single signed low-bit operand according to a preset calculation channel interval width, and performs multiplication with the same signed common operand within a single DSP hard core. This achieves dual-channel signed low-bit multiplication-accumulation on a single DSP, significantly improving the data path utilization of the multiplier and accumulator inside the DSP hard core, thereby enhancing the fixed-point computation density and matrix operation throughput of edge AI chips.

[0016] 2. This invention utilizes the accumulator and the accumulation result feedback path inside the DSP hard core to continuously accumulate the packaged product, so that the packaged accumulation result is continuously fed back inside the DSP without having to output to the outside for correction and then write back every cycle. This avoids the introduction of external combinational correction logic in the DSP accumulation feedback path, effectively reduces the combinational logic delay and routing pressure on the critical path, and is conducive to maintaining a high circuit operating frequency and system timing performance.

[0017] 3. This invention sets up a sign compensation module outside the DSP to determine the crossing state and update the sign compensation count value by accumulating the low-order residual value sign before and after addition and the current first-way product sign. This realizes the replacement of the traditional external wide-bit accumulator, carry counter and cycle-by-cycle correction feedback logic with a lightweight compensation structure, which significantly reduces the resource overhead of lookup table, flip-flops, wide-bit adder and accumulator register, and reduces chip area and dynamic power consumption.

[0018] 4. This invention completes operand packing according to the signed integer operation relationship through the signed input processing module, so that when the low-order operand is negative, its sign extension can correctly participate in the subsequent multiplication and addition operation. It realizes support for low-bit multiplication and addition accumulation operation when both inputs and the common operand are signed numbers. It overcomes the limitation of existing packing methods that are only applicable to unsigned common operands or specific sign types, and expands the applicable scope of the dual-channel multiplication and addition accumulation structure.

[0019] 5. This invention, by restoring two independent signed accumulation results at once based on the final packaged accumulation result and the final signed compensation count value after all input items have been accumulated, transforms the complex cycle-by-cycle result splitting and correction operation into the final single-cycle result restoration, reduces computational latency and control overhead during the operation process, and improves the overall computational efficiency and energy efficiency ratio of edge artificial intelligence chips in low-bit neural network inference tasks. Attached Figure Description

[0020] Figure 1 This is a schematic diagram of the overall process of a fixed-point multiply-accumulate method for edge artificial intelligence chips according to an embodiment of the present invention.

[0021] Figure 2This is a schematic diagram of the structure of the single-DSP dual-channel signed low-bit multiply-accumulate operation unit according to an embodiment of the present invention.

[0022] Figure 3 This is a schematic diagram of the sign extension and packing format of two-way signed INT8 operands according to an embodiment of the present invention.

[0023] Figure 4 This is a schematic diagram of the data path in which the packaged operands of this invention are pre-added, multiplied and accumulated inside the DSP48E2.

[0024] Figure 5 This is a schematic diagram illustrating the detection and updating of the signed-Q compensation value according to an embodiment of the present invention.

[0025] Figure 6 This is a schematic diagram of the processing flow for recovering the two-channel accumulation results based on the final accumulation result of the DSP and the signed-Q compensation value according to an embodiment of the present invention. Detailed Implementation

[0026] The present invention will be further described in detail below with reference to the accompanying drawings.

[0027] Example 1: Refer to Figure 1 This embodiment of a fixed-point multiply-accumulate method for edge AI chips includes: S1. Obtain the first signed low-bit operand, the second signed low-bit operand, and the signed common operand shared by the two operations from the input item, and perform sign extension on each of them; S2. According to the preset calculation channel interval bit width n, shift the sign-extended second operand to the left by n bits, and perform a signed addition operation with the sign-extended first operand to generate a packaged operand; S3. The packaged operand and the signed common operand are fed into a single DSP hard core for multiplication to obtain the packaged product. The packaged product is then continuously accumulated through the internal feedback path of the DSP hard core to obtain the packaged accumulation result. S4. In parallel with continuous accumulation, based on the sign of the low-order residual value before accumulation, the sign of the current first-way product, and the sign of the low-order residual value after accumulation, detect the crossing state of the first-way accumulation result within the n-bit signed representation interval, and update the signed compensation count value. S5. After all input items have been accumulated, based on the final packaged accumulation result and the final signed compensation count value, restore the independent signed accumulation results of the first and second paths, and generate a result valid signal.

[0028] Specifically, a fixed-point multiply-accumulate method for edge AI chips includes the following steps: S1. In a preferred embodiment, this embodiment is verified using a Xilinx ZCU102 development board. The target device is the Zynq UltraScale+ MPSoC series XCZU9EG-2FFVB1156E, and the DSP hard core used is DSP48E2. The DSP48E2 integrates a 27-bit pre-added data path, a 27×18-bit signed multiplier, a 48-bit arithmetic logic unit, and a 48-bit accumulator, and has a pre-adder register, a multiplication result register (M register), an accumulation result register (P register), and a P register feedback path. For the multiplication-addition-accumulation operation of two signed INT8 operands and one signed INT8 common operand, this embodiment sets the calculation channel (lane) spacing bit width n to 18 bits, uses the DSP48E2's internal pre-adder to pack the signed low-bit operands, and uses the internal multiplier, arithmetic logic unit, and P register feedback path to continuously accumulate the packed product.

[0029] like Figure 1 , Figure 2 As shown, based on the aforementioned hardware environment, the process of operand acquisition, sign extension, and port mapping in the signed input processing module is described in detail. First, the first path of the signed low-bit operand of the current input item is acquired. Second-way signed low-bit operands and the signed common operands shared by the two-way operations. Here, i represents the sequence number of the current input item. All three operands are 8-bit signed integers, represented in two's complement form. To enable subsequent packed multiplication and addition operations within a single DSP48E2, the sign extension of the three operands is required to match their bit widths with the corresponding input port widths of the DSP48E2, thus maintaining the correctness of the signed integer arithmetic relationships.

[0030] Specifically, the first path of signed low-bit operands The sign bit is copied and padded to the higher bits, expanding its bit width to 27 bits to match the data bit width of the DSP48E2 input combination unit; the second-path signed low-bit operand is padded... The sign bit is copied and padded to the higher bits, expanding its bit width to 27 bits so that it can be accommodated within the data bit width of the input combination unit after being shifted left by 18 bits; the signed common operand... The sign bit is copied and padded to the higher-order bits, extending its bit width to 18 bits to match the data bit width of the corresponding input of the DSP48E2 multiplier. This sign extension is achieved by copying and padding the most significant bit of the original operand to the extension bits, thus maintaining the numerical magnitude and sign characteristics of each operand.

[0031] like Figure 3As shown, after sign extension is completed, the extended signed common operands will be... The inputs to the multiplier port of the DSP48E2 allow the first and second signed low-bit operands to share the same signed common operand, thus achieving two low-bit multiplications equivalent to one wide-bit multiplication within a single DSP48E2. In a specific port connection configuration, the lower 27 bits of port A, the 27 bits of port D, and the 18 bits of port B of the DSP48E2 are configured as follows: , , , in, The lower 27 bits of port A of the DSP48E2, with a total bit width of 27 bits, correspond to one input of the preamplifier. The sign bit of the second signed low-bit operand is 1 bit, which is filled into the 26th bit (most significant bit) of port A. The complete 8-bit data of the second signed low-bit operand is filled into bits 25 through 18 of port A. Fill with 18 bits of all zeros, filling bits 17 through 0 of port A. This is the D port of the DSP48E2, with a total bit width of 27 bits, corresponding to another input of the preamplifier. To change the sign bit of the first signed low-bit operand Copy 19 times to form a 19-bit sign-extended field, and fill bits 26 to 8 of port D; The complete 8-bit data of the first signed low-bit operand is filled into bits 7 through 0 of port D. This is port B of the DSP48E2, with a total bit width of 18 bits, corresponding to another input of the multiplier. To set the sign bit of the signed common operand Copy 10 times to form a 10-bit sign-extended field, and fill it to bits 17 through 8 of port B; The complete 8-bit data of the signed common operand is filled into bits 7 through 0 of port B.

[0032] Through the above sign extension and port mapping, the first-way operand The second operand enters the preamplifier via port D. The common operand enters the pre-adder in high-order form via port A. The data enters the multiplier via port B, thereby completing the acquisition of operands, sign extension, and input configuration, providing the data foundation for subsequent generation of packaged operands and execution of packaged multiplication and addition.

[0033] S2. Based on the above sign extension and port configuration, the process of generating the packet operand is described in detail. According to the preset calculation channel interval width n, the second-path signed low-bit operand after sign extension is shifted left by n bits to obtain the high-bit calculation channel operand. This left shift operation is numerically equivalent to multiplying the second-path operand by 2n. A signed addition operation is performed between the high-bit calculation channel operand and the first-path signed low-bit operand after sign extension. The result is used as the signed packet operand. The signed addition operation is performed according to the signed integer arithmetic relationship, rather than directly concatenating the two operands as unsigned fields.

[0034] In the specific implementation of the DSP48E2, the pre-adder is configured to perform a signed addition of the lower 27 bits of port A and port D, where the input of port D is the first operand, sign-extended to 27 bits. The lower 27 bits of the input at port A are the second operand, which is then sign-extended and shifted left by 18 bits. The pre-adder output serves as the 27-bit input to the multiplier, thereby generating the packed operands internally within the DSP48E2. Based on the calculation channel spacing width n, the first operand is located in the lower-order calculation channel of the packed operand, and the second operand is located in the higher-order calculation channel. The two operands are packed together by the pre-adder according to the signed integer arithmetic relationship to ensure that when the lower-order calculation channel input is negative, its sign extension can correctly participate in subsequent pre-addition and multiplication operations.

[0035] Signed packed operands satisfy: , in, Let i be the signed packing operand corresponding to the i-th input item. For the first signed low-bit operand after sign extension, For the second-way signed low-bit operand after sign extension, The preset calculation channel interval bit width is denoted by 'i', where 'i' represents the input item number. In a preferred embodiment, for two signed INT8 operands, the calculation channel interval bit width 'n' is set to 18 bits, at which point the operands are packaged... For a 27-bit signed number, where the first operand... Occupying the lower 18 bits, second-way operand Occupying the 18th to 26th bit range, the two operands are isolated by an 18-bit interval, providing a structural basis for the subsequent generation and accumulation of the packaged product.

[0036] S3. Based on the generated packed operands, they are multiplied together with the signed common operands and fed into a single DSP hard core to obtain the packed product. In the specific implementation of the DSP48E2, the packed operands output by the pre-adder are used as one input to the multiplier, and the signed common operands after sign expansion are used as the other input. The multiplier performs a 27×18-bit signed multiplication operation and outputs the packed product. This packed product is mathematically equivalent to the sum of the first product and the second product after shifting the calculated channel spacing bit width, satisfying: , in, For the i-th item, the product is packed together. Let $n$ be the i-th signed common operand, and $n$ be the preset computation channel spacing bit width. In this embodiment, $n=18$. Let the lower-order bits calculate the channel product. High-order calculation of channel product Then the packaged product can be expressed as .

[0037] like Figure 4 As shown, the packed product is continuously accumulated via the internal feedback path of the DSP hard core to obtain the packed accumulation result. Specifically, using the arithmetic logic unit inside the DSP hard core, the packed product of the current cycle is added to the packed accumulation result of the previous cycle stored in the DSP hard core's internal accumulation register to obtain the packed accumulation result of the current cycle; the packed accumulation result of the current cycle is written to the accumulation register through the internal feedback path of the DSP hard core for accumulation of the next input item; the above addition and writing operations are repeated until all input items are calculated. In the DSP48E2, the operation mode is configured to add the multiplier output to the accumulated result of the previous cycle stored in the P register, the arithmetic logic unit performs ordinary addition, and the carry input is set to 0, resulting in: , in, This is the packaged and accumulated result after the i-th operation is completed. This is the packaged and accumulated result after the operation of the (i-1)th term. Let the i-th term be a packaged product, and the initial value be... After N multiplication and addition operations, the packaged and accumulated result is: , Define the actual cumulative results of the first path and the second path as follows: , , The final packaged summation result satisfies the following condition as the actual summation result of the two paths: , in, This is the final packaged and accumulated result after all calculations are completed. This is the result of the first path's true cumulative sum. The result is the actual accumulation result for the second channel, where N is the total number of terms in the multiplication and addition operation. Upon reset, the P register is cleared. When the input is valid, the DSP continues to execute the above multiplication and addition operations internally. The result of the P register is directly fed back to the next accumulation within the DSP, without needing to be re-inputted through an external port, nor is it necessary to correct the output and write it back to the DSP in each cycle, thus ensuring that the accumulation process is carried out continuously within the DSP.

[0038] S4. In parallel with the continuous accumulation within the DSP, the signed compensation module performs cross-state detection and signed compensation count value update for the low-order calculation channel. The lower n bits of the packed accumulation result are interpreted as a signed residual value, and the sign of this signed residual value is extracted as the sign of the low-order residual value after accumulation. Specifically, after the i-th operation is completed, the sign of the low-order residual value after accumulation... Depend on The The position is determined, that is The sign value 1 represents a negative number, and the sign value 0 represents a non-negative number; similarly, the sign value of the accumulated lowest-order residual value is... The result of the previous period's packaged cumulative results The Position determined.

[0039] like Figure 5 As shown, the signed compensation count value is determined. It is defined as the true cumulative result of the first path. With the above signed residual values The difference relative to An integer multiple of is used to record the cumulative offset of the first-way accumulation result relative to the signed residual value, satisfying: , in, This is the signed compensation count value after the i-th operation is completed. This represents the true cumulative result of the first path for the i-th term. To extract the signed residual value obtained by interpreting the lower n bits of the summation result of the i-th item, the expression is: .

[0040] Current first-way product symbol Based on the first signed low-bit operand With signed common operands The symbol state is determined, satisfying: , in, This is the symbol for the first product path. The sign bit of the first signed low-bit operand. For the sign bit of the signed common operand, For logical AND operation, For XOR operation, when and When neither is 0, Obtained by XORing the sign bits of the two, when and When either of these is 0, the current first-path product symbol is determined to be non-negative. This symbol determination logic is executed in parallel with the DSP multiply-accumulate operation, and the current first-path product symbol, after being delayed by a register of the same level as the DSP data path, maintains timing alignment with the low-order residual value symbols before and after accumulation in the corresponding period on the same operation term.

[0041] Based on the above three symbols, the signed compensation module compares the signs of the lowest-order residual values ​​before accumulation. The first product symbol at present and the sign of the low-order residual value after accumulation Determine whether positive or negative overflow occurs in the n-bit signed representation interval of the first-path accumulation result.

[0042] when (Non-negative) (Non-negative) and When (negative), the result of the first accumulation path is determined to cross the upper boundary of the n-bit signed representation interval in a positive direction, and the signed compensation count is incremented by 1; when (burden), (Negative) and When the sign is non-negative, the first-way accumulation result is determined to have crossed the lower boundary of the n-bit signed representation interval in a negative direction, and the signed compensation count is decremented by 1; in other sign combinations, the first-way accumulation result is determined not to have crossed the boundary of the n-bit signed representation interval, and the signed compensation count remains unchanged. The update amount of the signed compensation count satisfies: , and ,in, The amount of time to update the signed compensated count value. This is the signed compensation count value after the operation of the (i-1)th term is completed, i.e., the signed-Q compensation count value. This is the symbol for the first product path. To accumulate the sign of the lowest-order remaining value, As the sign of the low-order residual value after accumulation, this signed compensation module does not need to save the independent correction amount corresponding to each negative product, nor does it need to set a negative product counter, a positive wrapback counter, and a negative wrapback counter separately. All historical offset information of the first-way accumulation result relative to the low n-order signed residual value is uniformly represented by the signed compensation count value.

[0043] Signed compensation detection is performed in parallel with the next multiply-accumulate operation in the DSP, thus avoiding the addition of external combinational logic to the DSP's internal P-feedback path. The valid input signal, the low-order product sign, and the operation end signal are delayed by registers of the same number of stages as the DSP data path to ensure timing alignment. The signed compensation module updates the compensation value only when the corresponding valid signal is 1; when invalid cycles exist in the pipeline, the accumulation state, the previous sign state, and the compensation value remain unchanged. The end flag of the last input is delayed by the same number of pipeline stages and generates the final result valid signal after the last compensation update, ensuring that the DSP's final result and the signed compensation count value belong to the same accumulation state. Simultaneously, the absolute value of the current first-path product should be less than... To ensure that each time a first-way product is added, the low-order calculation channel crosses the boundary of the n-bit signed representation interval at most once, the bit width of the signed (Q) compensation count value is determined based on the maximum number of boundary crossings that the low-order calculation channel may generate under a predetermined number of accumulated terms.

[0044] S5. After all input items have been accumulated, the result recovery module reads the final packed accumulation result output from the DSP48E2 accumulation result register. And the final signed compensation count value output by the signed compensation module. Perform a single recovery of the two independent signed accumulation results. First, pack the final accumulation results into a single file. The lower n bits are interpreted as the first-way signed residual value, which is determined by... The (n-1)th to (0)th bits are decoded according to the rules for signed integers; subsequently, the final signed compensation count value is... After sign expansion, shift left by n bits and add to the signed residual value of the first path to obtain the true accumulation result of the first path. The true accumulation result of the first path satisfies: , in, This is the result of the first path's true cumulative sum. For the final packaged and accumulated result, The preset calculation channel interval bit width, For the final signed compensated count value, To sum the final packaged results The lower n bits are interpreted as signed residuals.

[0045] like Figure 6 As shown, in this embodiment, the calculation channel interval bit width n is set to 18 bits, and the DSP accumulation result bit width is... The result is 48 bits; first, start with the final packaged and accumulated result. Extract the least significant signed remainder, the most significant bit of the least significant bit, and the highest significant bit of the least significant bit: , , , in, To sum the final packaged results The signed residual value obtained by interpreting bits 17 through 0; To sum the final packaged results The signed number is obtained by interpreting bits 47 through 18. For the final packaged cumulative result The 17th bit, which is the highest bit of the low-order field.

[0046] Based on the extracted values, the result recovery module will ultimately obtain the signed compensated count value. After sign extension, shift left by 18 bits and combine with the signed remainder value in the lower bits. Add them together to obtain the actual accumulated result of the lower-order calculation channel: , In the formula, This is the result of the first path's true cumulative sum. This is the final signed compensation count value, which is used to compensate for the integer multiple difference caused by the cumulative offset of the first-path accumulation result within the 18-bit signed representation interval.

[0047] Next, restore the second path's true summation result. Then, package the final summation result. PN The high-order field is interpreted as a signed number, and this high-order field is... The The result is composed of bits up to the nth bit; simultaneously, the (n-1)th bit of the final packaged and accumulated result is extracted. This bit is used to compensate for the bit weight difference generated when the lower n bits are converted from unsigned fields to signed residual values, based on the aforementioned signed number, the (n-1)th bit, and the final signed compensation count value. Calculate the second true accumulation result, where the final signed compensation count value is used to eliminate the occupation of high-order fields caused by the cumulative offset of the first accumulation result. The second true accumulation result satisfies: , in, This is the actual cumulative result of the second path. To increase the bit width of the DSP accumulation result, This means that the high-order bits of the final packaged summation result are interpreted as signed numbers. This is the (n-1)th element of the final packaged and accumulated result.

[0048] Corresponding to this embodiment , The specific parameters, the actual accumulation result of the high-order calculation channel, are determined by the DSP high-order field. Low-order field, highest-order bit and the final signed compensation count value Joint recovery: , in, This is the actual cumulative result of the second path; Used to compensate for the bit weight difference generated when the lower 18 bits are converted from an unsigned field to a signed residual value; Used to eliminate the overhead of the high-order fields caused by the cumulative offset of the first-way accumulation result.

[0049] After the control module finishes the operation of all input items, the control result recovery module outputs the first and second true cumulative results and generates a result validity signal to indicate that the multiplication-accumulation operation is complete. The result recovery operation is only performed once after all multiplication-accumulation operations are completed and does not participate in the DSP's cycle-by-cycle accumulation feedback, thereby avoiding the introduction of additional combinational logic in the accumulation critical path.

[0050] The recovered first and second true cumulative results are directly used as the multiply-accumulate operation results of neural network inference in the edge AI chip. They are applied to the convolution operation of the convolutional neural network, the matrix multiplication and attention calculation of the Transformer, as well as the low-ratio eigenproduct calculation of other two inputs sharing the same operand. This enables real-time AI calculation to be completed under the conditions of limited power consumption, limited area and limited storage bandwidth, thereby improving the computing throughput, resource utilization and energy efficiency of the edge AI chip.

[0051] Example 2: This example differs from Example 1 in that it provides a fixed-point multiply-accumulate device for an edge artificial intelligence chip, comprising: The device is deployed in a Xilinx FPGA with an integrated DSP48E2 hard core and includes an input interface, a symbolic input processing module, a DSP multiply-accumulate module, a signed-Q compensation module, a result correction module, and a control module.

[0052] The input interface is used to receive the first signed low-bit operand. Second-way signed low-bit operands Shared signed common operands for two-way operations Input valid signal, reset signal and operation end signal.

[0053] The signed input processing module is used to perform sign expansion on the first operand, the second operand, and the common operand. The packaged operand generation module is used to generate packaged operands by left-shifting the second operand according to a preset lane interval bit width and then performing a signed addition with the first operand.

[0054] The DSP multiply-accumulate module includes an internal pre-adder, a multiplier, a 48-bit arithmetic logic unit (ALU), a P register, and an internal feedback path for the P register. The 48-bit ALU, the P register, and the internal feedback path together constitute the DSP hard-core internal accumulator. The pre-adder generates packed operands, the multiplier calculates the product of the packed operands and the common operand, and an M register (multiplication result register) is provided at the multiplier's output to store the packed product. The internal accumulator adds the current packed product output from the M register to the previous cycle's packed accumulation result fed back via the P register's internal feedback path using the 48-bit ALU, and writes the current cycle's packed accumulation result back to the P register for accumulation in the next cycle.

[0055] The signed-Q compensation module includes a signed-Q compensation count register, positive cross-boundary detection logic, and negative cross-boundary detection logic, which are used to update the signed-Q compensation count value based on the sign of the low-order residual value before accumulation, the current first-way product sign, and the sign of the low-order residual value after accumulation.

[0056] The result recovery module is used to read the final packed accumulation result and the final signed-Q compensation count value output by the DSP48E2 after all input items have been accumulated, and to recover the independent signed accumulation results of the first and second channels.

[0057] The control module is used to control the reset of the DSP48E2, enable the input register, enable the P register accumulation, update the signed-Q compensation counter, and generate the result valid signal. When the input is valid, the DSP48E2 is controlled to perform packing multiplication and internal accumulation. When all input items have finished operating, the result recovery module is controlled to output the two final accumulation results.

[0058] Through the above technical solution, this embodiment simultaneously completes two signed low-bit multiplication and accumulation operations within a single DSP. Only a small-width signed-Q compensation counter, a sign status register, and result recovery logic need to be configured externally to the DSP. Compared to a structure where the DSP only performs multiplication and requires two external full accumulators, this embodiment reduces the resource overhead of external wide-bit adders and accumulation registers. Compared to a structure that outputs, corrects, and feeds back to the DSP cycle by cycle, this embodiment does not add external combined correction logic to the DSP's accumulation feedback path, which helps maintain the operating frequency of the DSP's internal pipeline.

[0059] The above technical solutions can be applied to convolution operations in convolutional neural networks, matrix multiplication and attention operations in Transformers, as well as low-ratio t-product calculations where two inputs share the same operand, thereby increasing the number of signed low-bit operations per cycle performed by a single DSP without changing the FPGA hardware structure.

[0060] The above are all preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Therefore, all equivalent changes made in accordance with the structure, shape and principle of the present invention should be covered within the scope of protection of the present invention.

Claims

1. A fixed-point multiply-accumulate method for edge artificial intelligence chips, characterized in that, include: Obtain the first signed low-bit operand, the second signed low-bit operand, and the signed common operand shared by the two operations from the input item, and perform sign extension on each of them; According to the preset calculation channel interval bit width n, the second operand after sign expansion is shifted left by n bits, and a signed addition operation is performed with the first operand after sign expansion to generate a packaged operand; The packaged operand and the signed common operand are fed into a single DSP hard core for multiplication to obtain the packaged product. The packaged product is then continuously accumulated through the internal feedback path of the DSP hard core to obtain the packaged accumulation result. In parallel with continuous accumulation, based on the sign of the low-order residual value before accumulation, the sign of the current first-way product, and the sign of the low-order residual value after accumulation, the crossing state of the first-way accumulation result within the n-bit signed representation interval is detected, and the signed compensation count value is updated. After all input items are accumulated, the independent signed accumulation results of the first and second paths are restored based on the final packaged accumulation result and the final signed compensation count value, and a result valid signal is generated.

2. The fixed-point multiply-accumulate method for edge AI chips according to claim 1, characterized in that, The sign extension process includes copying and filling the sign bit of the first signed low-bit operand to the higher bits, thereby extending it to the data bit width of the DSP hard core input combination unit. The sign bit of the second signed low-bit operand is copied and filled to the high-bit side, so that after shifting it left by n bits, it can be accommodated within the data bit width of the input combination unit; The sign bit of the signed common operand is copied and filled to the higher bits, thus expanding it to the data bit width of the corresponding input of the DSP hard-core multiplier. The signed common operand after sign extension is input into the multiplier of the DSP hard core, so that the first signed low-bit operand and the second signed low-bit operand share the same signed common operand.

3. The fixed-point multiply-accumulate method for edge AI chips according to claim 1, characterized in that, The process of generating the packaged operands includes shifting the sign-extended second-way operands left by n bits according to a preset lane interval bit width to obtain the high-order calculation channel operands, performing a signed addition operation on the high-order calculation channel operands and the sign-extended first-way operands, and using the result of the operation as the signed packaged operands. The signed addition operation is performed according to the signed integer arithmetic relationship. Based on the calculation channel interval width n, the first operand is set in the low-order calculation channel of the packaged operand, and the second operand is set in the high-order calculation channel of the packaged operand. The expression of the signed packaged operand is: ， in, Let i be the signed packing operand corresponding to the i-th input item. For the first signed low-bit operand after sign extension, For the second-way signed low-bit operand after sign extension, The preset calculation channel interval bit width.

4. The fixed-point multiply-accumulate method for edge AI chips according to claim 1, characterized in that, The step of continuously accumulating the packaged product through the internal feedback path of the DSP hard core includes using the arithmetic logic unit inside the DSP hard core to add the packaged product of the current cycle to the packaged accumulation result of the previous cycle stored in the accumulation register inside the DSP hard core, so as to obtain the packaged accumulation result of the current cycle. The current cycle's packed accumulation result is written to the accumulation register through the feedback path inside the DSP hard core for accumulation of the next input item; Repeat the addition and write operations until all input items have been processed, and the resulting packaged summation satisfies: ， in, This is the packaged and accumulated result after the i-th operation is completed. This is the packaged and accumulated result after the operation of the (i-1)th term. Pack the i-th term into a product, and initialize it with the following values. .

5. A fixed-point multiply-accumulate method for edge AI chips according to claim 1, characterized in that, The detection of the crossing state of the first-path accumulation result within the n-bit signed representation interval includes interpreting the lower n bits of the packaged accumulation result as a signed residual value, and extracting the sign of the signed residual value as the sign of the lower bit residual value after accumulation. Determine the signed compensation count value, whereby the signed compensation count value is the difference between the first-path true accumulation result and the signed residual value relative to... An integer multiple of is used to record the cumulative offset of the first-way accumulation result relative to the signed residual value. The expression for the signed compensation count value is: ； in, This is the signed compensation count value after the i-th operation is completed. This represents the true cumulative result of the first path for the i-th term. This is the signed residual value obtained by interpreting the lower n bits of the summation result of the i-th item.

6. A fixed-point multiply-accumulate method for edge artificial intelligence chips according to claim 5, characterized in that, The detection of the crossing status of the first-way accumulation result within the n-bit signed representation interval also includes determining the current sign of the first-way product based on the sign status of the first-way signed low-bit operands and the signed common operands. When both the first-way signed low-bit operand and the signed common operand are not 0, the current first-way product sign is obtained by XORing their sign bits; When either the first signed low-bit operand or the signed common operand is 0, the sign of the current first-way product is determined to be non-negative.

7. A fixed-point multiply-accumulate method for edge AI chips according to claim 6, characterized in that, The updated signed compensation count value includes comparing the sign of the low-order remaining value before accumulation, the sign of the current first-way product, and the sign of the low-order remaining value after accumulation. When the low-order residual value before accumulation is non-negative, the current first-way product is non-negative, and the low-order residual value after accumulation is negative, it is determined that the first-way accumulation result positively crosses the upper boundary of the n-bit signed representation interval, and the signed compensation count value is incremented by 1. When the low-order residual value before accumulation is negative, the current first-way product is negative, and the low-order residual value after accumulation is non-negative, it is determined that the first-way accumulation result crosses the lower boundary of the n-bit signed representation interval in a negative direction, and the signed compensation count value is decremented by 1. In other symbol combinations, if the first-way accumulation result does not cross the boundary of the n-bit signed representation interval, the signed compensation count value remains unchanged, and the update amount of the signed compensation count value satisfies: ， in, The amount of time to update the signed compensated count value. This is the symbol for the first product path. To accumulate the sign of the lowest-order remaining value, This is the sign of the lower-order residual value after accumulation.

8. A fixed-point multiply-accumulate method for edge AI chips according to claim 1, characterized in that, The recovery of the independent signed accumulation results of the first and second paths includes interpreting the lower n bits of the final packaged accumulation result as the signed residual value of the first path after all input items have been accumulated; The final signed compensated count value is sign-extended, shifted left by n bits, and added to the first-path signed residual value to obtain the first-path true accumulation result. The expression for the first-path true accumulation result is: ， in, This is the result of the first path's true cumulative sum. For the final packaged and accumulated result, The preset calculation channel interval bit width, For the final signed compensated count value, To sum the final packaged results The lower n bits are interpreted as signed residuals.

9. A fixed-point multiply-accumulate method for edge artificial intelligence chips according to claim 8, characterized in that, The recovery of the independent signed accumulation results of the first and second paths also includes, after all input items have been accumulated, interpreting the high-order field of the final packaged accumulation result as a signed number and extracting the (n-1)th bit of the final packaged accumulation result; The second-way true accumulation result is calculated based on the signed number, the (n-1)th bit, and the final signed compensation count value. The (n-1)th bit is used to compensate for the bit weight difference generated when the lower n bits are converted from unsigned fields to signed residual values. The final signed compensation count value is used to eliminate the occupation of the higher bits caused by the cumulative offset of the first-way accumulation result. The expression for the second path's actual cumulative result is: ， in, This is the actual cumulative result of the second path. To increase the bit width of the DSP accumulation result, This means that the high-order bits of the final packaged summation result are interpreted as signed numbers. This is the (n-1)th element of the final packaged and accumulated result.

10. A fixed-point multiply-accumulate device for an edge artificial intelligence chip, performing the method of claim 1, characterized in that, include: The input interface is used to receive the first signed low-bit operand, the second signed low-bit operand, the signed common operand shared by the two operations, the input valid signal, the reset signal, and the operation end signal; The signed input processing module is used to perform sign extension on operands and generate packed operands; The DSP multiply-accumulate module includes a single DSP hard core, which integrates a pre-adder, a multiplier, and an accumulator to generate the current period's packed accumulation result. The signed-Q compensation module is used to update the signed compensation count value based on the sign of the low-order residual value before accumulation, the current first-way product sign, and the sign of the low-order residual value after accumulation. The result recovery module is used to recover the independent signed accumulation results of the first and second paths based on the final packaged accumulation result and the final signed compensation count value, and to generate a result valid signal. The control module is used to control the timing of the signed input processing module, the DSP multiply-accumulate module, the signed compensation module, and the result recovery module based on the input valid signal, the reset signal, and the operation end signal.

Images