Configurable mixing precision calculation array circuit based on outer product
By designing a configurable mixed-precision computing array circuit, the problem of single precision in traditional arrays is solved, improving resource utilization and computing efficiency, supporting multiple precision modes, and adapting to different neural network needs.
Patent Information
- Application Number
- CN202511856003.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-10
- Publication Date
- 2026-03-17
AI Technical Summary
Traditional computing arrays only support a single precision mode, which cannot meet the computational precision requirements of different neural network layers. They also have low resource utilization, incomplete symbol support, and limited application scenarios.
Design a configurable mixed-precision computing array circuit based on outer product. Using a basic int4 multiplier as a reference, it supports multiple precision calculations, expands by 1 bit sign bit, and incorporates a matrix transpose switch to reduce storage overhead and flexibly adapt to the computing needs of neural networks.
It improves the utilization rate of multiplier resources, supports multiple computing modes, increases resource utilization by 2-4 times, expands application scenarios, and realizes improved computing efficiency and dynamic reconfiguration.
Smart Images
Figure CN121682000A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of artificial intelligence accelerator, and particularly relates to a configurable mixed precision calculation array circuit based on outer product. BACKGROUND
[0002] With the diversified development of deep learning models, higher requirements are put forward for the precision support of computing hardware. Traditional matrix multiplication accelerators usually only support single precision calculation (such as int8), and it is difficult to adapt to the differentiated needs of different neural network layers for computing precision. Especially in the edge computing scene, it is necessary to achieve dynamic balance between computing precision and energy efficiency.
[0003] The prior art has the following main problems:
[0004] Lack of precision flexibility: traditional computing arrays usually only support 1-2 fixed precision modes, and cannot adapt to the demand of mixed precision calculation.
[0005] Low resource utilization: in single precision mode, the configuration of the computing unit cannot be dynamically adjusted according to the computing demand.
[0006] Inadequate symbol support: existing int4 / int8 computing units mostly adopt a single signed / unsigned design, which limits the application scenarios. SUMMARY
[0007] In view of the above problems or deficiencies, the present application provides a configurable mixed precision calculation array circuit based on outer product. The present application takes the basic int4 multiplier as a benchmark, fuses the results of the basic multiplier according to different configuration modes to support high-precision calculation, and improves the utilization rate of multiplier resources; by expanding the 1-bit sign bit, the array supports signed multiplication of signed numbers, signed multiplication of unsigned numbers, and unsigned multiplication of unsigned numbers, and flexibly adapts to various computing needs of neural networks; a matrix transposition switch is built-in to reduce the overhead of external transposition; the output data bit width is limited by functions such as saturation truncation to reduce storage overhead.
[0008] A configurable mixed precision calculation array circuit based on outer product, as shown in Figure 1 The circuit is used to accelerate matrix calculation, and the input calculation data is two matrices A and B. A road and B road respectively represent the data of matrix A and matrix B.
[0009] The control module is used to receive external instruction information and distribute configuration information, and read the data of the data buffer module and send it to the computing array for calculation. The configuration information includes calculation mode selection, whether to select signed input for A road and B road input, and whether to select saturation truncation for output result.
[0010] The data buffer module is used for buffering data required for input calculation, and is composed of n+1 sram banks, sequentially recorded as bank 0 to bank n. Among them, bank 0 and n computing arrays are directly connected, and store A path data; bank 1 - bank n are respectively connected with n computing arrays in point-to-point mode, and store B path data. The data bit width of sram is 4k bits.
[0011] The computing array has n, which is used for receiving data of the data buffer module for calculation. Each computing array is composed of k*k array MAC (Multiplier Accumulator) units, wherein k is an integer multiple of 4; considering the actual wiring influence, k≤64.
[0012] Each MAC unit receives 5bit data of A and B paths after symbol processing module expansion for multiplication and accumulation calculation, and one multiplication and accumulation calculation is divided into three periods: the first period is for input data registration, the second period is for multiplication operation, and the third period is for accumulation operation.
[0013] The control module provides data valid signal and the last data valid signal to the computing array, and when the last data valid signal is received, the final result is registered in the output interface after 3 periods to wait for the result fusion module to take away, at this time the computing array has completed a complete matrix operation, and the next matrix operation can be started.
[0014] The symbol processing module is responsible for selecting how to expand 1bit symbol bit for input data according to different configuration information provided by the control module before the input data is sent to the MAC unit.
[0015] Further, the expansion mode of the symbol processing module is divided into signed and unsigned, and 1bit signal is used to control whether to start signed calculation for A and B paths respectively;
[0016] For unsigned type, the expanded symbol bit is set to 0 for four computing precisions;
[0017] For signed type:
[0018] Int4 mode: the highest bit of each 4bit number is used as the symbol bit expansion;
[0019] Int8*int4 mode, if the input of B path is int8 and the input of A path is int4: e and f in B path are respectively the low bit data and high bit data of the first int8, g and h are respectively the low bit data and high bit data of the second int8, the symbol bits of e and g are filled with 0, and the highest bit of f and h is taken as the expanded symbol bit; the highest bit of the four data in A path is used as the expanded symbol bit;
[0020] Int8 mode: both A and B inputs are int8, a and b in A are low and high data of the first int8 respectively, c and d are low and high data of the second int8 respectively, the sign bits of a and c are padded with 0, and the highest bits of b and d are taken as extended sign bits; e and f in B are low and high data of the first int8 respectively, g and h are low and high data of the second int8 respectively, the sign bits of e and g are padded with 0, and the highest bits of f and h are taken as extended sign bits;
[0021] Int16 mode: for A, a, b, c and d represent four low-to-high parts of int16 input respectively, so only d has a sign bit, and the highest bit of d is taken as a sign bit for extension; e, f, g and h in B represent four low-to-high parts of int16 input respectively, so only h has a sign bit, and the highest bit of h is taken as a sign bit for extension.
[0022] The result fusion module is responsible for receiving the output results of the computing array and performing result fusion processing according to different configuration information of the control module. The result fusion is divided into four calculation precisions, namely int4 mode, int8 mode, int8*int4 mode and int16 mode. Then, the result fusion module writes the output data into the specified sram as the next input or directly outputs it to the off-chip according to the external configuration.
[0023] Further, the result fusion is specifically:
[0024] In the int4 mode, the result does not need to be fused, and the single computing array finally outputs 32*32 results.
[0025] In the int8*int4 mode, the result in MAC0 remains unchanged, the result in MAC1 is right-shifted by 4 bits, and then added to the result in MAC0 and output, and the single computing array finally outputs 32*16 results.
[0026] In the int8 mode, the result in MAC0 remains unchanged, the results in MAC1 and MAC2 are right-shifted by 4 bits, the result in MAC3 is right-shifted by 8 bits, then the four results are added to fuse into one int8 result, and the single computing array finally outputs 16*16 results.
[0027] In the int16 mode, first, follow the rules of the int8 mode to fuse four MACs into four int8 results, then keep the left-up int8 result unchanged, right-shift the right-up and right-down int8 results by 8 bits, right-shift the right-down int8 result by 16 bits, finally add the four results to fuse into one int16 result, and the single computing array finally outputs 8*8 results.
[0028] The configurable hybrid precision calculation array circuit based on outer product has the following specific working procedure:
[0029] Step 1: The control module receives an external request, configures the calculation array, the symbol processing module and the result fusion module, and waits to receive external input data.
[0030] Step 2: The external device writes the data to be calculated into the sram of the data buffer module through a bus or dma until all the data required for one calculation is written.
[0031] Step 3: After the control module detects that all the data is written, a data valid signal is generated and sent to the calculation array, and the sram data is read and sent into the calculation array; the sram data is first expanded according to the configuration information provided by the control module through the symbol processing module, and then input into the calculation array for multiplication and accumulation operation, and the calculation result is accumulated in the calculation array until the last valid data is read from the data buffer module through the control module and then input into the calculation array, and the result is stored in the output interface of the calculation array.
[0032] Step 4: The result fusion module obtains the output data from the output interface of the calculation array, fuses the results according to the configuration information, and then outputs in a pipeline manner. If it is an int4 mode, it is bypassed and directly output; if it is another mode, it is fused and then output.
[0033] Step 5: After all the results are output, a complete calculation is completed, and a completion signal is sent.
[0034] In summary, the present application supports high-precision calculation by fusing the results of the basic int4 multiplier according to different configuration modes, supports up to 12 different specifications, improves the utilization rate of multiplier resources, and the utilization rate of multiplier resources reaches 100%; by expanding 1-bit sign bit, the array supports signed multiplication of signed numbers, signed multiplication of unsigned numbers, and unsigned multiplication of unsigned numbers, and flexibly adapts to various computing needs of neural networks; the built-in matrix transposition switch reduces the overhead of external transposition; the output data bit width is limited by functions such as saturation truncation, reducing storage overhead. The present application improves the calculation efficiency, supports up to 4 computing precisions and 12 computing modes, and improves the resource utilization rate by 2-4 times; the application scenario is expanded, the signed / unsigned calculation is supported, and the dynamic reconfiguration at runtime is supported. BRIEF DESCRIPTION OF DRAWINGS
[0035] Fig. 1 is a schematic block diagram of the present application;
[0036] Figure 2 Schematic diagram of the calculation array;
[0037] Fig. 3 is a schematic diagram of the result fusion module;
[0038] Figure 4 Matrix partitioning scheme design for the embodiment;
[0039] Figure 5 A schematic diagram of the computational pipeline of this invention. Detailed Implementation
[0040] The present invention will now be described in detail with reference to the accompanying drawings and embodiments.
[0041] A configurable mixed-precision computing array circuit based on outer product, with the specific architecture as follows: Figure 1 As shown, it includes a control module, a data buffer module, a computation array, a symbol processing module, and a result fusion module. The input computation data consists of two matrices, A and B, where path A and path B represent the data in matrix A and matrix B, respectively.
[0042] Figure 2 This is a schematic diagram of the computation array. For inputs A and B, the input bit width is 128 bits, corresponding to 32 groups of 4-bit input values. Each 4-bit value of input A is broadcast to all MACs in the corresponding row. For example, data 'a' is broadcast to all MACs at the same level as 'a'. Each 4-bit value of input B is broadcast to all MACs in the corresponding column. For example, data 'e' is broadcast to all MACs at the same height as 'e'.
[0043] After data enters the MAC (Macro-MAC) register, it is stored internally in the first cycle, multiplied in the second cycle, and accumulated in the third cycle. Pipeline computation then begins. When the last valid data signal is received, all MAC accumulation results are saved to the output interface for the result fusion module to retrieve, then fused and output. If the valid data signal for the next calculation and the last valid data signal from the previous calculation are both pulled high simultaneously, the next calculation can proceed directly, achieving bubble-free computation.
[0044] The sign bit extension method is divided into signed and unsigned. Both paths A and B use a 1-bit signal to control whether signed computation is enabled.
[0045] For unsigned types, the extended sign bit can be set to 0 for all four computational precisions.
[0046] For signed types:
[0047] Int4 mode: Extends the most significant bit of each 4-bit number as a sign bit.
[0048] Int8*int4 mode: Here, input B is int8 and input A is int4. In input B, e and f are the low and high bits of the first int8, respectively, and g and h are the low and high bits of the second int8, respectively. The sign bits of e and g are padded with 0, and the most significant bit of f and h is used as the extended sign bit. In input A, the most significant bit of all four data points is used as the extended sign bit.
[0049] Int8 mode: Both inputs for paths A and B are int8. In path A, a and b are the low and high bits of the first int8, respectively, and c and d are the low and high bits of the second int8, respectively. The sign bits of a and c are padded with 0, and the most significant bit of b and d is used as the extended sign bit. In path B, e and f are the low and high bits of the first int8, respectively, and g and h are the low and high bits of the second int8, respectively. The sign bits of e and g are padded with 0, and the most significant bit of f and h is used as the extended sign bit.
[0050] Int16 mode: For path A, a, b, c, and d represent the four parts of the int16 input from low to high, so only d has a sign bit, and the highest bit of d is used as the sign bit for extension; In path B, e, f, g, and h represent the four parts of the int16 input from low to high, so only h has a sign bit, and the highest bit of h is used as the sign bit for extension.
[0051] After the computation array completes its calculations, the result fusion module is responsible for fusing the results and outputting them. Figure 3 This is a schematic diagram illustrating the principle of result fusion. In Int4 mode, results do not need to be fused, and a single computation array ultimately outputs 32*32 results. In Int8*int4 mode, the result in MAC0 remains unchanged, the result in MAC1 is shifted right by 4 bits and added to the result in MAC0 before output, resulting in a single computation array outputting 32*16 results. In Int8 mode, the result in MAC0 remains unchanged, the results in MAC1 and MAC2 are shifted right by 4 bits, and the result in MAC3 is shifted right by 8 bits. Then, the four results are added and fused into a single int8 result, resulting in a single computation array outputting 16*16 results. In Int16 mode, following the rules of int8 mode, four MACs are grouped together and fused into four int8 results. Then, the int8 result in the upper left remains unchanged, the int8 results in the upper right and lower right are shifted right by 8 bits, and the int8 result in the lower right is shifted right by 16 bits. Finally, the four results are added and fused into a single int16 result, resulting in a single computation array outputting 8*8 results.
[0052] Since large matrix multiplications usually require block-based computation in practical applications, this invention proposes a matrix splitting method for the computation array to maximize the reuse of input data and mask delays. Figure 4The specific matrix partitioning scheme and SRAM data mapping rules are implemented as follows:
[0053] The SRAM storing the input data is divided into 5 banks, denoted as banks0-4. For int4, bank0 stores the gray blocks of input matrix A, based on each column of the gray block. Each point has 4 bits of data, for a total of 32 points, so each address stores 128 bits of data. Banks1-4 store the 4 gray blocks of input matrix B, similar to input matrix A, but based on each row of the gray block. Due to the characteristics of outer product multiplication, the input data for each array consists of one column of gray blocks from matrix A and one row of gray blocks from matrix B. The input data only needs to be used once for a complete calculation, maximizing input reuse. After the SRAM data is preloaded, the matrix A data in bank0 is broadcast to the 4 arrays for data reuse. Banks1-4 correspond to the 4 calculation arrays for matrix B. After a complete calculation, the matrix B data is fixed, and only the data in bank0 needs to be replaced, completing the data reuse of banks1-4. Pipeline reading of data can mask this transmission delay until all matrix B data has been traversed before replacement.
[0054] Figure 5 This is a schematic diagram of the computational pipeline of the present invention. After reading the instruction, the external circuit begins to preload data. After the data loading is completed, the control module initiates the computation instruction, reads data from the SRAM and sends it to the outer product calculation array, and simultaneously begins loading the A matrix data to be used next. In this way, the delay in loading matrix data is masked, improving the overall performance. After the calculation is completed, the next calculation can begin immediately, and the results are fused and output simultaneously. Therefore, as long as the K dimension of the matrix calculation is greater than or equal to the result output, bubble-free pipelined calculation and result output can be achieved.
[0055] As can be seen from the above embodiments, this invention uses a basic int4 multiplier as a benchmark and fuses the results of the basic multiplier according to different configuration modes to support high-precision calculations. It can support up to 12 different calculation specifications, improving the multiplier resource utilization to 100%. By expanding the sign bit by 1 bit, the array can support signed multiplication, signed multiplication, and unsigned multiplication, flexibly adapting to various computational needs of neural networks. A built-in matrix transpose switch reduces the overhead of external transposition. Functions such as saturation truncation limit the output data bit width, reducing storage overhead. This invention not only improves computational efficiency, supports up to 4 calculation precisions and 12 calculation modes, increases resource utilization by 2-4 times, improves support for signed / unsigned calculations, and supports dynamic reconfiguration at runtime, greatly expanding application scenarios.
Claims
1. An outer product based configurable mixed-precision computing array circuit, comprising: The control module, the data buffer module, the calculation array, the symbol processing module and the result fusion module are included; the input calculation data are two matrices A and B, and A path and B path respectively represent the data of matrix A and matrix B; The control module is used for receiving external instruction information, distributing configuration information and reading data of the data buffer module to send the data to the calculation array for calculation; the configuration information includes calculation mode selection, whether to select signed input for A path and B path input and whether to select saturation truncation for output result; The data buffer module is used for buffering data required for input calculation, is composed of n+1 sram banks and is sequentially recorded as bank 0 to bank n; wherein bank 0 and n calculation arrays are directly connected and store A path data; bank 1 to bank n are respectively connected with n calculation arrays in point-to-point mode and store B path data; The data bit width of the sram is 4k bits; The calculation array has n, which is used for receiving data of the data buffer module for calculation; each calculation array is composed of k*k array MAC units, wherein k is an integer multiple of 4; Each MAC unit receives 5-bit data of A and B paths after expansion by the symbol processing module for multiplication and accumulation calculation, and one multiplication and accumulation calculation is divided into three periods: the first period is used for input data storage, the second period is used for multiplication operation and the third period is used for accumulation operation; The control module provides a data valid signal and a last data valid signal to the calculation array, and when the last data valid signal is received, the final result is stored in the output interface after three periods to wait for the result fusion module to take away, at this time, the calculation array has completed a complete matrix operation, and the next matrix operation can be started; The symbol processing module is responsible for selecting how to expand 1-bit symbol bit for input data according to different configuration information provided by the control module before the input data is sent to the MAC unit; The result fusion module is responsible for receiving the output result of the calculation array and performing result fusion processing according to different configuration information of the control module, and the result fusion is divided into four calculation precisions, which are int4 mode, int8 mode, int8*int4 mode and int16 mode; then the result fusion module writes the output data into the specified sram as the next input or directly outputs to the outside according to the external configuration.
2. The outer product based configurable mixed precision computation array circuit of claim 1, wherein: The expansion mode of the symbol processing module is divided into signed and unsigned, and 1-bit signal is used to control whether to start signed calculation for A and B paths; For the unsigned type, the extended symbol bit is set to 0 for the four calculation precisions; For the signed type: Int4 mode: the highest bit of each 4-bit number is expanded as a symbol bit; Int8*int4 mode, if the B input is int8 and the A input is int4: e and f in the B are respectively the low data and the high data of the first int8, g and h are respectively the low data and the high data of the second int8, 0 is supplemented to the sign bit of e and g, and the highest bit of f and h is taken as the extended sign bit; the four data in the A all use the highest bit as the extended sign bit; Int8 mode: the A and B inputs are both int8, a and b in the A are respectively the low data and the high data of the first int8, c and d are respectively the low data and the high data of the second int8, 0 is supplemented to the sign bit of a and c, and the highest bit of b and d is taken as the extended sign bit; e and f in the B are respectively the low data and the high data of the first int8, g and h are respectively the low data and the high data of the second int8, 0 is supplemented to the sign bit of e and g, and the highest bit of f and h is taken as the extended sign bit; Int16 mode: for the A, a, b, c and d represent respectively the four parts from low to high of the int16 input, so only d has the sign bit, and the highest bit of d is taken as the sign bit extension; e, f, g and h in the B represent respectively the four parts from low to high of the int16 input, so only h has the sign bit, and the highest bit of h is taken as the sign bit extension.
3. The configurable hybrid precision calculation array circuit based on outer product of claim 1, wherein: the result fusion is specifically: in the int4 mode, the result does not need to be fused, and the single calculation array finally outputs 32*32 results; in the int8*int4 mode, the result in MAC0 remains unchanged, the result in MAC1 is right shifted by 4 bits, and then added with the result in MAC0 to be output, and the single calculation array finally outputs 32*16 results; in the int8 mode, the result in MAC0 remains unchanged, the results in MAC1 and MAC2 are right shifted by 4 bits, the result in MAC3 is right shifted by 8 bits, then the four results are added to be fused into one int8 result, and the single calculation array finally outputs 16*16 results; in the int16 mode, first, the rule of the int8 mode is followed, that is, every four MACs are a group, and are fused into four int8 results, then the left-up int8 result remains unchanged, the right-up and right-down int8 results are right shifted by 8 bits, the right-down int8 result is right shifted by 16 bits, finally, the four results are added to be fused into one int16 result, and the single calculation array finally outputs 8*8 results.
4. The outer product-based configurable mixed-precision computation array circuit of claim 1, wherein: The k≤64.
5. The outer product-based configurable mixed-precision computation array circuit of claim 1, wherein, The specific working process is: Step 1: the control module receives an external request, configures the calculation array, the sign processing module and the result fusion module, and waits to receive external input data; Step 2: the external writes the data to be calculated into the sram of the data buffer module through the bus or dma until all the data required for one calculation is written in; Step 3: after the control module detects that all the data is written, a data valid signal is generated and sent to the calculation array, and the sram data is read and sent into the calculation array. The sram data is first sent into the symbol processing module to complete corresponding symbol bit extension according to configuration information provided by the control module, and then is sent into the calculation array to perform multiplication and accumulation operation, and the calculation result is accumulated in the calculation array until the last valid data is read from the data buffer module by the control module and then is input into the calculation array, and the result is registered in the output interface; Step 4, the result fusion module obtains the output data from the output interface of the calculation array, performs result fusion according to the configuration information, and then performs output in a pipeline manner; if it is an int4 mode, the output is bypassed; if it is another mode, the output is fused and then outputted; Step 5, after all the results are outputted, a complete calculation is completed, and a completion signal is sent.