A pipeline-based universal multiple-precision matrix multiplication-addition operation circuit

Abstract

This invention discloses a pipelined, general-purpose multi-precision matrix multiplication and addition circuit, belonging to the field of AI chip design. This invention is designed for a highly parallel multi-precision tensor computation unit based on a many-core architecture. It implements matrix multiplication and addition operations with different data precisions based on a pipelined structure, including four data precisions: int4, int8, fp16, and fp32, as well as four mixed precision operation modes: int4 mixed with int32, int8 mixed with int32, fp16 mixed with fp32, and int8 mixed with fp32. The proposed circuit and method design independent pipelines for different operation modes, allowing different types of data to reuse a single computation module, achieving a hardware resource reuse rate of 88.9%. This invention is widely applicable to tensor computation. By designing independent pipelines for different operation modes and reusing hardware circuits, it eliminates the need for separate circuits for integer and floating-point data operations, thus improving hardware reuse.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence chip design technology, and in particular to a pipelined general-purpose multi-precision matrix multiplication and addition circuit. Background Technology

[0002] With the rapid development of artificial intelligence technology, deep neural networks (DNNs) have become the cornerstone of core fields such as computer vision, natural language processing, and autonomous driving. The powerful capabilities of these models are built upon massive matrix multiplication and addition operations. From convolutional layers to fully connected layers, and then to self-attention mechanisms, their core computations can all be reduced to large-scale matrix or tensor multiplication and addition operations. Therefore, the performance and energy efficiency of the hardware units performing these multiplication and addition operations (such as the processing cores in GPUs, TPUs, and NPUs) directly determine the overall performance of the artificial intelligence computing system.

[0003] However, artificial intelligence models have drastically different requirements for computational precision at different stages (such as training and inference) and in different application scenarios (such as cloud and edge), which has created an urgent need for multi-precision computing capabilities.

[0004] As algorithm complexity increases and the number of parameters grows, matrix multiplication and addition, as a key fundamental operation in artificial intelligence algorithms, requires significant optimization. Traditional arithmetic circuits completely separate the design of operations with different precisions; for example, integer operations and floating-point operations use different operation modules. This greatly reduces the hardware reuse rate of arithmetic circuits and brings huge area overhead. Summary of the Invention

[0005] The purpose of this invention is to provide a pipelined general-purpose multi-precision matrix multiplication and addition circuit to solve the problems in the background art.

[0006] To solve the above technical problems, the present invention provides a pipeline-based general-purpose multi-precision matrix multiplication and addition circuit, which is a matrix multiplication and addition operation unit used to perform the product of row vectors in matrix A and column vectors in matrix B, and the accumulation of point data in matrix C; The arithmetic circuit includes 5 full adders and 4 multipliers, for a total of 9 arithmetic units; it also includes comparators, XOR gates, right shifters, original two's complement converters, first-1 detection shifters, and normalizers. When performing int8 mix fp32 type operations, an additional 32-bit full adder and two original complement converters are added, and a mode-specific shifter and a two-to-one comparator are designed, along with a first-to-one detection shifter and a two-input comparator differencer. The arithmetic circuit supports eight arithmetic modes: int4, int8, fp16, fp32, int4 mix int32, int8 mix int32, fp16 mix fp32, and int8 mix fp32. In the int4, int8, int4 mix int32, and int8 mix int32 modes, a four-stage pipelining (multiply-add-add-add) is used, completing 16 operand multiplications and one point operand accumulation within 8 clock cycles. In the fp16 and fp16 mix fp32 modes, a 2+5 stage pipelining is used, completing 16 fp16 floating-point operand multiplications and one floating-point operand accumulation within 20 clock cycles. In the fp32 mode, a 2+5 stage pipelining is used, completing 16 fp32 floating-point operand multiplications and one fp32 floating-point operand accumulation within 33 clock cycles. In the int8 mix... In fp32 operation mode, a 4-stage pipeline is used to complete 16 pairs of int4 floating-point operand multiplications and 1 fp32 floating-point operand accumulation within 17 clock cycles.

[0007] In one implementation, under the four operation modes of int4, int8, int4 mix int32, and int8 mix int32, a single multiplier always processes a pair of integer operands, and the four multipliers execute in parallel; a single full adder always processes a pair of operands, both of which are 32-bit two's complement, and the operation topology of the four full adders is a three-level addition tree; the operation process loads data into the internal data buffer in the first cycle, and since the 16 cycles need to be divided into four entries into the operation pipeline, four four-level pipeline calculations are performed in cycles 2-8, during which the multipliers and full adders are reused, and the data is output in the ninth cycle.

[0008] In one implementation, under the fp16, fp16 mix fp32 operation modes, it is divided into a PRE-state two-stage pipeline and a COM-state five-stage pipeline; In PRE state, a two-stage pipeline is formed using four full adders and one 5-to-1 comparator. The maximum exponent sum of the corresponding data in the rows of matrix A and the columns of matrix B is calculated in five cycles. Then, in the sixth cycle, it is compared with the exponent of the point data in matrix C through a 2-to-1 comparator to obtain the maximum exponent. In COM mode, the exponent offset is first calculated. Then, a five-stage pipeline consisting of four multipliers, five original complement converters, and four full adders is used to calculate the product of the mantissas of the rows of matrix A and the mantissas of the columns of matrix B within eight cycles. Based on the pre-shifted offset, the exponent is shifted and complemented. The complement data is then input into the addition tree for accumulation. The accumulated complement result is then normalized.

[0009] In one implementation, the fp32 operation mode is divided into a two-stage pipeline in PRE mode and a five-stage pipeline in COM mode. The COM mode of fp32 operation mode uses 4 multipliers, 1 original two's complement converter, and 4 full adders to form a 5-stage pipeline. Since the 4 16-bit multipliers complete the calculation of 1 pair of fp32 mantissas, a 2+1 two-stage adder tree is used here to integrate the outputs of the multipliers to obtain an unsigned 26-bit mantissa product of 1 pair of fp32s. In order to perform 16 signed operations of 1 pair of fp32 mantissa products, the sign bit is added and the complement is taken at the same time. The resulting 32-bit two's complement mantissa product enters the full adder for operation. The COM mode in fp32 operation mode needs to perform 16 pipeline tasks.

[0010] In one implementation, in the int8 mix fp32 operation mode, the integer part multiply-accumulate is completed in a four-stage pipeline, and then the int8 multiply-accumulate sum and the fp32 operand are added together in 9 cycles.

[0011] In one implementation, the full adder is a 32-bit full adder. Except for one full adder dedicated to int8 mix fp32 type operations, the other four 32-bit full adders form a reconfigurable full adder group that supports two operation topologies. This group is responsible for processing the accumulation of integer multiplication results, the multiplication results of floating-point mantissa after order alignment, and the addition of floating-point order. For integer multiplication and accumulation, and fp16 mantissa multiplication and accumulation, the four full adders are combined into a three-level addition tree topology and executed in a three-level pipeline. For the mantissa multiplication and accumulation of fp32, the four full adders are divided into two groups. Three of them are in one group and perform the same operation as the first two levels of the three-level addition tree. The remaining full adder is a separate level and needs to be input after the addition tree output is shifted and complemented.

[0012] In one implementation, the right shifter is implemented using a 5-stage shifter, with each stage performing right shifts of 1, 2, 4, 8, and 16 bits respectively, supporting a maximum of 31-bit right shifts. When performing floating-point operand multiplication and accumulation, the maximum value of the exponent in the product exponent of 16 pairs of multiply-accumulate operands and the exponent in the accumulation operand is taken, and the exponent is aligned and the mantissa is shifted by pre-shifting the multiply-accumulate sum and vector operands to speed up floating-point multiplication and accumulation operations.

[0013] In one implementation, the original two's complement converter performs original two's complement conversion on the input data; after accumulating the mantissa product after the order shift and accumulating it with the mantissa of the accumulated operand, the obtained operation result is normalized; since the full adder always performs unsigned or signed two's complement calculations, the original two's complement converter is required to be added before the data input of the normalizer.

[0014] In one implementation, the normalizer calculates the number of leading zeros in the mantissa sum of the original code, and shifts the sum to the left according to the number of leading zeros, while simultaneously calculating the offset of the final exponent; after the exponent is shifted, the normalized mantissa product is truncated and rounded to generate the final floating-point result.

[0015] In one implementation, the normalizer is a multi-precision universal type. The number of decimal places in the sum of fp16 and fp32 is different. The shifted sum is left-shifted to the high-order bits to get the number. Different numbers are subtracted from the leading zeros under different precision calculations, and the leading zero counting module is reused. When performing mantissa accumulation, the operands of fp16 are 22 bits with the decimal point at the 20th bit, while the operands of fp32 are 26 bits with the decimal point at the 24th bit. According to the IEEE 754 standard, there needs to be one 1 before the decimal point. The relationship between the final exponent offset and the number of leading zeros is: 'd10 - number of leading zeros' in FP16 format and 'd6 - number of leading zeros' in FP32 format. After the normalizer performs normalization shift on the mantissa product and outputs the exponent offset, it splits the mantissa and sign bit, rounds the mantissa, and finally concatenates the floating-point calculation result.

[0016] This invention provides a pipeline-based general-purpose multi-precision matrix multiplication and addition circuit, which aims to improve the hardware reuse rate of the operation circuit and reduce area overhead by designing independent pipelines for different operation modes and reusing the same operation modules in different pipeline stages. Attached Figure Description

[0017] Figure 1 This is a schematic diagram of the overall structure of the pipeline-based general multi-precision matrix multiply-accumulate circuit proposed in this invention.

[0018] Figure 2 This is a schematic diagram of the full adder group operation topology proposed in this invention.

[0019] Figure 3 This is a schematic diagram of the right shifter based on 5-level shifting used in this invention.

[0020] Figure 4 This is a schematic diagram illustrating a typical application scenario of the present invention.

[0021] Figure 5 This is a schematic diagram of the four-stage pipeline for integer operations proposed in this invention.

[0022] Figure 6 This is a schematic diagram illustrating the integer operations applied in this invention.

[0023] Figure 7 This is a schematic diagram of the PRE-state two-stage pipeline for floating-point arithmetic proposed in this invention.

[0024] Figure 8 This is a schematic diagram of the five-stage pipeline in COM state proposed in this invention, consisting of fp16, fp16 mix, and fp32.

[0025] Figure 9 This is a schematic diagram of the fp16 and fp16 mix fp32 operations used in this invention.

[0026] Figure 10 This is a schematic diagram of the five-stage pipeline in FP32 COM state proposed in this invention.

[0027] Figure 11 This is a schematic diagram of the fp32 operation applied in this invention. Detailed Implementation

[0028] The following detailed description, in conjunction with the accompanying drawings and specific embodiments, provides a pipelined general-purpose multi-precision matrix multiplication and addition circuit proposed in this invention. The advantages and features of this invention will become clearer from the following description. It should be noted that the drawings are all in a very simplified form and use non-precise scales, and are only used to facilitate and clarify the illustration of the embodiments of this invention.

[0029] This invention provides a general-purpose multi-precision matrix multiplication and addition circuit and method. It is a computational unit for matrix multiplication and addition, performing the multiplication and accumulation of a row and a column of numbers in a matrix, followed by addition to a single number. The circuit supports four common data precisions and eight operation modes, exhibiting strong versatility and wide applicability. Furthermore, the proposed circuit and method feature separately designed pipelines for different operation modes, precisely controlling data flow and eliminating the need for separate circuit designs for different data types and operation modes, thus improving hardware reusability.

[0030] This invention proposes a pipelined general-purpose multi-precision matrix multiplication and addition circuit and method, the basic principles of which are as follows: The overall architecture of the multi-precision tensor computation unit is as follows: Figure 1 As shown, the basic arithmetic circuit consists of the following basic modules: four 16-bit multipliers, four 32-bit full adders, one 5-to-1 comparator, one 2-to-1 comparator, four 1-bit multipliers (XOR gates), five 5-stage right shifters, six original two's complement converters, and one normalizer. Furthermore, when performing int8 mix fp32 type operations, one 32-bit full adder and two original two's complement converters are added. Mode-specific shifters and 2-to-1 comparators are also designed, along with one first-1 detection shifter and one two-input comparator differencer to alleviate wiring and timing constraints. In this case, the int8 multiplication-accumulation and fp32 addition operations no longer reuse the full adder group composed of four 32-bit full adders and the original two's complement converter.

[0031] As shown in Table 1, the multi-precision tensor computation unit supports eight operation modes. In the `int4`, `int8`, `int4 mixint32`, and `int8 mix int32` operation modes, a four-stage pipelining is used, enabling 16 operand multiplications and one point operand accumulation within 8 clock cycles. In the `fp16` and `fp16 mix fp32` operation modes, a 2+5 stage pipelining is used, enabling 16 fp16 floating-point operand multiplications and one floating-point operand accumulation within 20 clock cycles. In the `fp32` operation mode, a 2+5 stage pipelining is used, enabling 16 fp32 floating-point operand multiplications and one fp32 floating-point operand accumulation within 33 clock cycles. In the `int8 mix fp32` operation mode, a four-stage pipelining is used, enabling 16 int4 floating-point operand multiplications and one fp32 floating-point operand accumulation within 17 clock cycles. The design of each basic module is described in detail below.

[0032] Operation mode Stream levels Cycle required for single-point calculation int4 4 8 int8 4 8 int4 mix int32 4 8 int8 mix int32 4 8 fp16 2+5 20 fp32 2+5 33 fp16 mix fp32 2+5 20 int8 mix fp32 4 17

[0033] Table 1. Operational efficiency of multi-precision tensor computation unit The 16-bit multiplier is implemented using Booth encoding. By eliminating zeros in the encoded multipliers, the number of partial products is effectively reduced, thus lowering computational latency and hardware resource consumption, and improving the efficiency of the arithmetic circuit.

[0034] The full adder is a 32-bit full adder, except for the one dedicated to int8 mix fp32 operations, such as Figure 2 As shown, the remaining four 32-bit full adders form a reconfigurable full adder group supporting two operational topologies. They are responsible for processing the accumulation of integer multiplication results, the mantissa multiplication results of floating-point numbers after order alignment, and the addition of floating-point order. For integer multiplication accumulation and fp16 mantissa multiplication accumulation, the four full adders are combined into... Figure 2 The three-level addition tree topology in the FP32 uses a three-level pipelined execution. For the mantissa multiplication and accumulation of the FP32, the four full adders are divided into two groups. Three of them are in one group and perform the same operation as the first two levels of the three-level addition tree. The remaining full adder is a separate level and the output of the addition tree needs to be shifted and complemented before it can be input.

[0035] The right shifter is implemented using a 5-stage shifter, with each stage capable of right shifting 1, 2, 4, 8, and 16 bits respectively, supporting a maximum of 31-bit right shifts. Figure 3 As shown, when performing floating-point multiplication and accumulation, the maximum value of the exponent in the product of 16 pairs of multiply-accumulate operands and the exponent of the accumulation operand is taken. A pre-shift method is used to align the exponents and shift the mantissas of the multiply-accumulate sum and vector operands to speed up floating-point multiplication and accumulation operations. During this operation, the exponent only increases, and the corresponding mantissa only needs to be shifted right; therefore, the shifter only needs to have a right-shift function and does not need a left-shift function.

[0036] The original two's complement converter performs original two's complement conversion on the input data. After accumulating the mantissa product after the shifted order and adding it to the mantissa of the accumulated operand, the result needs to be normalized. Since the full adder always performs unsigned or signed two's complement calculations, an original two's complement converter is required before the data input of the normalizer. The principle of this general calculation is not elaborated here.

[0037] The normalizer calculates the number of leading zeros in the accumulated sum of the mantissas in the original code, and then shifts the accumulated sum left (excluding the sign bit) according to the number of leading zeros. Simultaneously, it calculates the offset of the final exponent. After the exponent shift, the normalized mantissa product is truncated and rounded to generate the final floating-point result. The normalizer is versatile across multiple precision levels. Although the number of decimal places in the accumulated sum differs between FP16 and FP32, the left shift of the shifted accumulated sum to the high-order bits only requires subtracting the number of leading zeros from different numbers for different precision calculations, thus reusing the leading zero counting module to simplify the circuit. When performing mantissa accumulation, the operands in fp16 are 22 bits, with the decimal point in the 20th bit; the operands in fp32 are 26 bits, with the decimal point in the 24th bit. According to the IEEE 754 standard, there needs to be one 1 before the decimal point. Therefore, the relationship between the final exponent offset and the number of leading zeros is: 'd10 - number of leading zeros' in FP16 format, and 'd6 - number of leading zeros' in FP32 format. After the normalizer performs normalization shift on the mantissa product and outputs the exponent offset, it separates the mantissa and sign bit, rounds the mantissa, and finally concatenates the results to produce the floating-point calculation.

[0038] Figure 4 It demonstrates typical application scenarios, using Figure 1 The pipelined general-purpose multi-precision matrix multiply-accumulate circuit and method shown perform multiplication and accumulation of row i of matrix A and column j of matrix B, and then sum the results with C. ij Accumulation operation. The data in matrix A and matrix B have the same precision, while the data precision in matrix C can be the same as or different from that in matrices A and B. A total of eight operation modes are supported: int4, int8, fp16, fp32, int4 mix int32, int8 mix int32, fp16 mix fp32, and int8 mix fp32.

[0039] Based on the basic principles and schematics described above, the pipelined computation of different operation modes is explained in detail. The operations referred to in this invention all involve 16 pairs of AB matrix multiplication and accumulation operands and one C matrix point operand.

[0040] The application method of the pipelined general-purpose multi-precision matrix multiply-accumulate circuit is as follows: (1) When performing operations on the four data types int4, int8, int4 mix int32, and int8 mix int32, a four-stage pipeline of multiplication-addition-addition is executed, such as Figure 5 As shown; in the four operation modes, a single multiplier always processes a pair of integer operands, and the four multipliers execute in parallel; a single full adder also always processes a pair of operands, both of which are 32-bit two's complement. The data type of the C matrix point data does not affect the working mode of the full adder. The operation topology of the four full adders is a three-level addition tree. In the first cycle, the data is loaded into the internal data buffer. Since the 16 cycles need to be divided into four entries into the operation pipeline, four four-level pipeline calculations are performed in cycles 2-8. During the process, the multipliers and full adders are reused. The data is output in the ninth cycle, as shown. Figure 6 As shown; (2) When performing operations on fp16 and fp16 mix fp32 types, the pipeline is divided into a two-stage pipeline in PRE state and a five-stage pipeline in COM state. The PRE state two-stage pipeline is as follows: Figure 7 As shown, the COM state five-stage pipeline is as follows: Figure 8 As shown. In PRE state, a two-stage pipeline consisting of four full adders and one 5-to-1 comparator is used to calculate the maximum exponent sum of the corresponding data in the rows of matrix A and the columns of matrix B within five cycles. Then, in the sixth cycle, it is compared with the exponent of the point data in matrix C through a 2-to-1 comparator to obtain the maximum exponent. In COM state, the exponent offset is first calculated. Then, a five-stage pipeline consisting of four multipliers, five original complement converters, and four full adders is used to calculate the product of the mantissas of the rows of matrix A and the mantissas of the columns of matrix B within eight cycles. Based on the pre-shifted offset, the exponent is shifted and complemented. The complement data is then input into the addition tree for accumulation. The accumulated complement result is further normalized and other operations. The complete calculation process is as follows. Figure 9 As shown; (3) When performing operations on fp32 type, it is still divided into a two-stage pipeline in PRE state and a five-stage pipeline in COM state, where the PRE state two-stage pipeline is as follows: Figure 7 As shown, the COM state five-stage pipeline is as follows: Figure 10 As shown, the COM state in fp32 operation mode uses a 5-stage pipeline consisting of 4 multipliers, 1 original two's complement converter, and 4 full adders. Since 4 16-bit multipliers are needed to complete one pair of fp32 mantissa calculations, a 2+1 two-level adder tree is used to integrate the multiplier outputs, resulting in an unsigned 26-bit mantissa product of one pair of fp32. To perform 16 signed operations on the mantissa products of fp32, the sign bit is added while the complement is taken, resulting in a 32-bit two's complement mantissa product, which is then processed by the full adder. Unlike fp16 and fp16 mix fp32 operations, the COM state in fp32 operation mode requires 16 pipelined tasks. The complete pipeline operation is as follows: Figure 11 As shown; (4) When performing operations on int8 mix fp32 type data, the integer part is multiplied and accumulated using the following method: Figure 5 The four-stage pipeline shown is completed, and then the int8 multiplication and the fp32 operand are added together in 9 cycles. The operations for each cycle are listed in Table 2.

[0041]

[0042] Table 2 shows the process of multiplying and summing int8 and adding fp32 operands. The invention proposes a pipelined general-purpose multi-precision matrix multiply-add circuit and method that performs a one-point operation on the result of AxB+C. The input and output consist of 16 pairs of multiply-add operands and one point operand. It supports eight operation modes: int4, int8, fp16, fp32, int4 mix int32, int8 mix int32, fp16 mix fp32, and int8 mix fp32. Excluding one dedicated full adder for int8 mix fp32, the remaining four 16-bit multipliers and four 32-bit full adders are controlled by different pipelines, performing operations in all eight operation modes. The proposed pipelined general-purpose multi-precision matrix multiply-add circuit achieves a hardware reuse rate of 88.9% across different operations.

[0043] The above details the design and function of each module in the circuit, and introduces pipelines for different operation modes, including a four-stage pipeline for integer operations and int8 mix fp32, and a 2+5 stage pipeline for floating-point operations. The advantage of this invention lies in its excellent hardware reuse rate; different pipeline operations are executed under different operation modes, but the same hardware modules are reused, achieving a hardware reuse rate of 88.9%.

[0044] The above description is merely a description of preferred embodiments of the present invention and is not intended to limit the scope of the present invention in any way. Any changes or modifications made by those skilled in the art based on the above disclosure shall fall within the protection scope of the claims.

Claims

1. A pipelined general-purpose multi-precision matrix multiplication and addition circuit, characterized in that, It is the unit of operation for matrix multiplication and addition, used to perform the product of the row vectors in matrix A and the column vectors in matrix B, and to accumulate the data of the points in matrix C; The arithmetic circuit includes 5 full adders and 4 multipliers, for a total of 9 arithmetic units; it also includes comparators, XOR gates, right shifters, original two's complement converters, first-1 detection shifters, and normalizers. When performing int8 mix fp32 type operations, an additional 32-bit full adder and two original complement converters are added, and a mode-specific shifter and a two-to-one comparator are designed, along with a first-to-one detection shifter and a two-input comparator differencer. The arithmetic circuit supports eight arithmetic modes: int4, int8, fp16, fp32, int4 mix int32, int8 mix int32, fp16 mix fp32, and int8 mix fp32. In the int4, int8, int4 mix int32, and int8 mix int32 modes, a four-stage pipelining (multiply-add-add-add) is used, completing 16 operand multiplications and one point operand accumulation within 8 clock cycles. In the fp16 and fp16 mix fp32 modes, a 2+5 stage pipelining is used, completing 16 fp16 floating-point operand multiplications and one floating-point operand accumulation within 20 clock cycles. In the fp32 mode, a 2+5 stage pipelining is used, completing 16 fp32 floating-point operand multiplications and one fp32 floating-point operand accumulation within 33 clock cycles. In the int8 mix... In fp32 operation mode, a 4-stage pipeline is used to complete 16 pairs of int4 floating-point operand multiplications and 1 fp32 floating-point operand accumulation within 17 clock cycles.

2. The pipelined general-purpose multi-precision matrix multiplication and addition circuit as described in claim 1, characterized in that, In the four operation modes of int4, int8, int4 mix int32, and int8 mix int32, a single multiplier always processes a pair of integer operands, and four multipliers execute in parallel; a single full adder always processes a pair of operands, both of which are 32-bit two's complement, and the operation topology of the four full adders is a three-level addition tree; the operation process loads the data into the internal data buffer in the first cycle, and since the 16 cycles need to be divided into four entries into the operation pipeline, four four-level pipeline calculations are performed in cycles 2-8, during which the multipliers and full adders are reused, and the data is output in the ninth cycle.

3. The pipelined general-purpose multi-precision matrix multiplication and addition circuit as described in claim 1, characterized in that, In fp16 and fp16 mix fp32 operation modes, it is divided into PRE-mode two-stage pipeline and COM-mode five-stage pipeline; In PRE state, a two-stage pipeline is formed using four full adders and one 5-to-1 comparator. The maximum exponent sum of the corresponding data in the rows of matrix A and the columns of matrix B is calculated in five cycles. Then, in the sixth cycle, it is compared with the exponent of the point data in matrix C through a 2-to-1 comparator to obtain the maximum exponent. In COM mode, the exponent offset is first calculated. Then, a five-stage pipeline consisting of four multipliers, five original complement converters, and four full adders is used to calculate the product of the mantissas of the rows of matrix A and the mantissas of the columns of matrix B within eight cycles. Based on the pre-shifted offset, the exponent is shifted and complemented. The complement data is then input into the addition tree for accumulation. The accumulated complement result is then normalized.

4. The pipelined general-purpose multi-precision matrix multiplication and addition circuit as described in claim 1, characterized in that, In fp32 operation mode, the pipeline is divided into a two-stage PRE-mode pipeline and a five-stage COM-mode pipeline. The COM-mode pipeline in fp32 operation mode uses four multipliers, one original two's complement converter, and four full adders to form a five-stage pipeline. Since the four 16-bit multipliers complete one pair of fp32 mantissa calculations, a 2+1 two-stage adder tree is used here to integrate the multiplier outputs, resulting in an unsigned 26-bit mantissa product of one pair of fp32. In order to perform 16 signed operations on the mantissa products of fp32, the sign bit is added and the complement is taken at the same time. The resulting 32-bit two's complement mantissa product enters the full adder for operation. In fp32 operation mode, the COM-mode pipeline needs to perform 16 pipeline tasks.

5. The pipelined general-purpose multi-precision matrix multiplication and addition circuit as described in claim 1, characterized in that, In the int8 mix fp32 operation mode, the integer part multiplication and accumulation is completed in a four-stage pipeline, and then the int8 multiplication and accumulation sum and the fp32 operand are added together in 9 cycles.

6. The pipelined general-purpose multi-precision matrix multiplication and addition circuit as described in claim 1, characterized in that, The full adder is a 32-bit full adder. In addition to a full adder dedicated to int8 mix fp32 type operations, the other four 32-bit full adders form a reconfigurable full adder group that supports two operation topologies. It is responsible for processing the accumulation of integer multiplication results, the result of floating-point mantissa multiplication after order alignment, and floating-point order addition. For integer multiplication and accumulation, and fp16 mantissa multiplication and accumulation, the four full adders are combined into a three-level addition tree topology and executed in a three-level pipeline. For the mantissa multiplication and accumulation of fp32, the four full adders are divided into two groups. Three of them are in one group and perform the same operation as the first two levels of the three-level addition tree. The remaining full adder is a separate level and needs to be input after the addition tree output is shifted and complemented.

7. The pipelined general-purpose multi-precision matrix multiplication and addition circuit as described in claim 6, characterized in that, The right shifter is implemented using a 5-stage shifter, with each stage performing right shifts of 1, 2, 4, 8, and 16 bits respectively, supporting a maximum of 31-bit right shifts. When performing floating-point operand multiplication and accumulation, the maximum value of the exponent in the product of 16 pairs of multiply-accumulate operands and the exponent in the accumulation operand is taken, and the exponent is aligned and the mantissa is shifted by pre-shifting the multiply-accumulate sum and vector operands to speed up floating-point multiplication and accumulation operations.

8. The pipelined general-purpose multi-precision matrix multiplication and addition circuit as described in claim 7, characterized in that, The original two's complement converter performs original two's complement conversion on the input data; after accumulating the mantissa product after the order shift and accumulating it with the mantissa of the accumulated operand, the obtained operation result is normalized; since the full adder always performs unsigned or signed two's complement calculations, the original two's complement converter is required to be added before the data input of the normalizer.

9. The pipelined general-purpose multi-precision matrix multiplication and addition circuit as described in claim 8, characterized in that, The normalizer calculates the number of leading zeros in the mantissa sum of the original code, shifts the sum to the left based on the number of leading zeros, and calculates the offset of the final exponent output. After the exponent is shifted, the normalized mantissa product is truncated and rounded to generate the final floating-point result.

10. The pipelined general-purpose multi-precision matrix multiplication and addition circuit as described in claim 9, characterized in that, The normalizer is a multi-precision universal type. The number of decimal places in the sum of fp16 and fp32 is different. The high-order bits of the shifted sum are left-shifted and the number is taken. Different numbers are subtracted from the leading zeros under different precision calculations. The leading zero counting module is reused. When performing mantissa accumulation, the operands of fp16 are 22 bits with the decimal point at the 20th bit, while the operands of fp32 are 26 bits with the decimal point at the 24th bit. According to the IEEE 754 standard, there needs to be one 1 before the decimal point. The relationship between the final exponent offset and the number of leading zeros is: 'd10 - number of leading zeros' in FP16 format and 'd6 - number of leading zeros' in FP32 format. After the normalizer performs normalization shift on the mantissa product and outputs the exponent offset, it splits the mantissa and sign bit, rounds the mantissa, and finally concatenates the floating-point calculation result.

Images