Multi-precision MAC tree type processing unit and systolic array structure

Abstract

The invention relates to the technical field of digital integrated circuits, and particularly provides a multi-precision MAC tree type processing unit and a systolic array structure.The processing unit comprises an MAC tree type structure, and the MAC tree type structure comprises a multi-precision multiplication module used for carrying out data multiplication operation on multiple sets of first precision data and second precision data, determining a corresponding first product result and a second product result; the adder tree module comprises a first hierarchical structure and a second hierarchical structure used for determining a product accumulation result; the first hierarchical structure comprises N node structures, each node structure comprises an adder and a first data selector, the adder is used for receiving the first product result, performing additive operation and generating a first addition result, and two paths of input signals of the first data selector are the first addition result and the second product result respectively. The problems of operation delay and storage bottleneck in the matrix operation process can be solved.

Description

Technical Field

[0001] This application belongs to the field of digital integrated circuit technology, and in particular relates to a multi-precision MAC tree processing unit and a pulsating array structure. Background Technology

[0002] With the rapid development of deep learning technology, large language models (LLMs) have demonstrated unprecedented semantic understanding and generation capabilities in fields such as natural language processing, dialogue systems, code generation, and assisted writing. From early architectures based on recurrent neural networks (RNNs) and long short-term memory networks (LSTMs), to the emergence of Transformers, and then to the ultra-large-scale pre-trained models represented by GPT, LlaMA, and DeepSeek in recent years, the number of parameters has rapidly expanded from billions to tens of billions, hundreds of billions, or even trillions. Simultaneously, model structures have expanded from single decoders to diverse topologies including encoder-decoder and sparse expert hybrid (MoE) models. However, the explosive growth in parameter size has brought about massive computational and storage demands: a single forward inference often requires hundreds of billions of matrix multiplications within milliseconds, while frequently accessing off-chip high-bandwidth memory (HBM) to move weights, activations, and key-value caches, causing traditional general-purpose CPU and GPU architectures to gradually expose bottlenecks in terms of energy efficiency, latency, and scalability. To alleviate these problems, the industry generally adopts a collaborative optimization approach combining algorithms and hardware. At the algorithm level, researchers have reduced computation and memory access through techniques such as sparsification, quantization, distillation, dynamic batch processing, and KV-Cache compression. At the hardware level, various architectural innovations have emerged, including tensor cores, systolic arrays, reconfigurable dataflow accelerators, near-memory computing, and optical interconnects.

[0003] Among them, systolic arrays have become one of the mainstream hardware paradigms for matrix multiplication and convolution operations due to their high parallelism, regular data flow, low control overhead, and good scalability. The core idea of ​​systolic arrays is to realize the pipelined transfer and on-site computation of data through two-dimensional or one-dimensional regularly arranged processing elements (PEs), thereby transforming memory access overhead into on-chip shifting and significantly reducing the dependence on high-bandwidth off-chip storage. However, existing systolic array designs are mainly aimed at convolutional neural networks (CNNs), whose workload characteristics are significantly different from those of large language models: the LLM inference stage is divided into two stages: prefill and decoding. Prefill mainly uses high-dimensional matrix-matrix multiplication (GEMM), while the decoding stage mainly uses mini-batch, high-concurrency matrix-vector multiplication (GEMV). The two operations have different characteristics. Secondly, the introduction of KV-Cache requires on-chip storage to simultaneously accommodate weights, activations, and ultra-long sequence caches, which greatly increases the capacity and bandwidth requirements of on-chip static random access memory.

[0004] Therefore, when traditional systolic arrays are applied to large language model inference, it is difficult to achieve a balance between utilization, energy efficiency and latency. Summary of the Invention

[0005] This application discloses a multi-precision MAC tree processing unit and a pulsating array structure, which can optimize the problems of computational latency and storage bottleneck in matrix operation, and achieve high throughput, low latency and high energy efficiency inference acceleration.

[0006] Other objectives and advantages of this application can be further understood from the technical features disclosed herein.

[0007] To achieve one, some, or all of the above objectives or other objectives, in a first aspect, this application provides a multi-precision MAC tree processing unit applied to a pulsating array structure. The processing unit includes a MAC tree structure for implementing multiplication-accumulation-addition operations, an accumulator for performing accumulation operations, and an output register for outputting the operation results; wherein the MAC tree structure includes: The multi-precision multiplication module is used to perform data multiplication operations on multiple sets of first-precision data and second-precision data, and determine the corresponding first product result and second product result; the number of bits in the first-precision data is less than the number of bits in the second-precision data. The addition tree module includes a first hierarchical structure for performing addition operations on data of a first precision and a second hierarchical structure for determining the result of multiplication accumulation. The first hierarchical structure includes N node structures, each node structure including an adder for receiving the first product result, performing addition operations, and generating a first addition result, and a first data selector. The two input signals of the first data selector are the first addition result and the second product result, respectively.

[0008] The multi-precision multiplication module includes a data preprocessing unit, 2N encoders, and 2N compressors. The data preprocessing unit is used to determine the precision of the input data and the mantissa of the input data, and to segment the mantissa into data and send it to the 2N encoders respectively. The 2N encoders are used to perform encoding operations, generate 2N partial products, and send them to the 2N compressors respectively. The compressors are used to perform partial product compression on the partial products, generate carry chain data and local chain data, and send them to the addition tree module for addition operations.

[0009] The number of N values ​​is related to the precision of the input data.

[0010] An encoding register is provided between the encoder and the compressor, and a compression register is provided between the compressor and the addition tree module.

[0011] The second-level structure includes N-1 adders arranged in a tree topology cascade.

[0012] Each adder is connected in series with a register.

[0013] One input of the first data selector is connected to a register in the same node structure, and the other input is connected to the multi-precision multiplication module.

[0014] The processing unit further includes a weight register for storing weight data, an activation register for storing activation data, a pipeline register for caching intermediate calculation results, a second data selector, and a third data selector. The input signal of the second data selector includes the calculation result of the previous processing unit and the calculation result of the current processing unit. The input signal of the third data selector includes the calculation result of the next processing unit and the calculation result of the current processing unit.

[0015] Secondly, this application provides a pulsating array structure, including a plurality of multi-precision MAC tree processing units arranged in a matrix as described in any one of the first aspects.

[0016] The pulsating array structure includes a weight buffer connected to the processing unit, a weight buffer for inputting weight data to the processing unit, and an input buffer for inputting activation data to the processing unit. The weight buffer and the input buffer are used to arrange and temporarily store multiple sets of weight data and activation data according to the data precision.

[0017] The aforementioned multi-precision MAC tree processing unit and pulsating array structure have at least the following technical advantages: 1) By replacing traditional serial accumulation with N-level parallel accumulation of the addition tree module, the pipelined startup and cascading transmission of traditional serial MAC are eliminated, and matrix-vector multiplication and accumulation can be completed in a single cycle without reducing computational latency. This optimizes the computational latency and storage bottleneck issues in the deployment and inference of large language models, achieving high throughput, low latency, and high energy efficiency inference acceleration. 2) It achieves parallelization and redundancy-free generation of partial products, reduces the limitation of bit width on multiplication latency, realizes parallel hierarchical compression, and reduces compression latency and waiting latency.

[0018] To make the above and other objects, features and advantages of this application more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the specific embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a schematic diagram of the MAC tree structure in a processing unit according to an embodiment of this application.

[0021] Figure 2 This is a schematic diagram of the structure of a multi-precision multiplication module in an embodiment of this application.

[0022] Figure 3 This is a circuit diagram of the encoder in the multi-precision multiplication module of this application embodiment.

[0023] Figure 4 This is a schematic diagram of the four precision multiplicative data segmentation methods described in the embodiments of this application.

[0024] Figure 5 This is a schematic diagram of the first four compressors of the multi-precision multiplication module in the embodiments of this application.

[0025] Figure 6 This is a schematic diagram of the four compressors following the multi-precision multiplication module in the embodiments of this application.

[0026] Figure 7 This is a schematic diagram of the internal structure of a processing unit in one embodiment of this application. Figure 8 This is a schematic diagram of the structure of a pulsating array in an embodiment of this application. Detailed Implementation

[0027] The foregoing and other technical contents, features, and effects of this application will be clearly presented in the following detailed description of a preferred embodiment with reference to the accompanying drawings. The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such terms can be used interchangeably where appropriate; this is merely a method of distinguishing objects with the same attributes in the embodiments of this application.

[0028] In their research, the inventors of this application discovered that the mainstream computing units of current neural network processors (NPUs) can be roughly classified into two categories: one is a tree structure represented by the multiply-accumulate-tree (MAC-tree), and the other is a traditional systolic array composed of regularly arranged processing units (PEs). The multiplication results of each level in the MAC-tree structure are cascaded and summarized through adder trees to form a tree-like reduction, exhibiting extremely low single-cycle latency; for example, for N-way multiplier inputs, the latency is only log2N. However, MAC-trees face significant area and wiring complexity during expansion. For instance, the number of multipliers and adders increases linearly with the input dimension, the tree-like addition path becomes difficult to maintain synchronous clock and load balance as the scale increases, and the data path is irregular with low reusability, thus limiting computational density and parallelism, making it difficult to meet the stringent high-throughput requirements of large models. Traditional pulse arrays perform multiply-accumulate and local transfer operations in a fixed timing sequence. Data propagates along the array without requiring complex control logic. As a result, they perform well in terms of area and energy efficiency thanks to their high-density PE arrays and local data reuse mechanisms. However, due to the sequential data flow, the time difference between the arrival of data in the first and last columns of the array increases linearly with the scale, resulting in high overall latency. This makes them unsuitable for token-level generation tasks. The throughput is high only when the array is full. For small-batch, high-concurrency processing scenarios in the LLM decoding stage, the utilization of processing units drops sharply.

[0029] To address at least one of the aforementioned technical problems, embodiments of this application provide a multi-precision MAC tree processing unit. This processing unit is applied to a pulsating array structure and includes a MAC tree structure for implementing multiplication-accumulation-addition operations, an accumulator for performing accumulation operations, and an output register for outputting the operation results. (Reference) Figure 1 The MAC tree structure includes a multi-precision multiplication module and an addition tree module. The multi-precision multiplication module performs data multiplication operations on multiple sets of first-precision data and second-precision data to determine the corresponding first and second product results; the number of bits in the first-precision data is less than the number of bits in the second-precision data. The addition tree module includes a first-level structure for implementing addition operations on the first-precision data and a second-level structure for determining the multiplication accumulation result; the first-level structure includes N node structures, each node structure including an adder for receiving the first product result, performing addition operations, and generating a first addition result, and a first data selector, where the two input signals of the first data selector are the first addition result and the second product result, respectively.

[0030] Specifically, this application uses the multi-precision multiplication module to perform data multiplication operations on different first-precision data (e.g., low-bit-width data) and second-precision data (e.g., high-bit-width data) to determine the corresponding first and second product results. Multiple nodes in the first hierarchical structure operate in parallel. The first adder performs multiplication and accumulation operations on the first-precision data, and the first data selector switches between multi-precision operations. The multiplication and accumulation results of the second-precision data skip the addition operations in the first hierarchical structure and are directly fed into the second hierarchical structure through the first data selector for single-step parallel accumulation. The second hierarchical structure determines the multiplication and accumulation result. The accumulator is used to temporarily store and accumulate intermediate results, and finally, the operation result is output through the output register. The multi-precision MAC tree processing unit in this embodiment replaces traditional serial accumulation with N-level parallel accumulation of the addition tree module. It eliminates the need for pipelined startup and cascading of traditional serial MACs, completing matrix-vector multiplication and accumulation in a single cycle without reducing computational latency. This optimizes the computational latency and storage bottleneck issues in large language model deployment and inference, achieving high throughput, low latency, and high energy efficiency inference acceleration.

[0031] It should be noted that, Figure 1 The example shown uses N=4. In practical applications, the number of N values ​​is related to the precision of the input data. For instance, assuming the input data is 64-bit X and 64-bit Y, if the data precision includes FP8, INT8, FP16, and BF16, FP8 and INT8 can be used as the first precision data, and FP16 and BF16 as the second precision data, with N set to 4. If the data precision includes FP8 and INT8, then FP8 and INT8 can be used as the first precision data, and only the first precision data operation data path is used, with N set to 4. If the data precision includes FP16 and BF16, then FP16 and BF16 can be used as the second precision data, and the second precision data operation data path is used, with N set to 2.

[0032] In some embodiments, N is the output parallelism of the first hierarchical structure, and the second hierarchical structure includes N-1 adders arranged in a tree topology cascade, such as... Figure 1 As shown. For example, if N=4 in the first level structure, then the second level structure is divided into 2 levels: 2 adders in the first level and 1 adder in the second level. For example, if N=16 in the first level structure, then the second level structure is divided into 4 levels: 8 adders in the first level, 4 adders in the second level, 2 adders in the third level, and 1 adder in the fourth level.

[0033] Further, refer to Figure 1Each adder is connected in series with a register to enable pipeline timing control, intermediate result localization buffering, and multi-precision data synchronization during the parallel accumulation process of the MAC tree structure. This allows the entire adder tree module to form an uninterrupted pipeline operation link with the preceding multi-precision multiplication module and the subsequent accumulator.

[0034] refer to Figure 1 One input of the first data selector is connected to a register in the same node structure to receive the first addition result from the register in the same node structure, and the other input is connected to the multi-precision multiplication module to directly receive the second product result obtained by the multi-precision multiplication module. The output of the first data selector is connected to the adder of the first stage in the second hierarchical structure to output the selected precision data. The first data selector enables dynamic switching between different data precisions, specifically including dynamic switching between multiply-add operations of multiple sets of low-precision data and multiply-add operations of multiple sets of high-precision data, achieving dynamic computation path and energy consumption balance in multi-precision operations. It should be noted that the other input of the first data selector can be directly connected to the register of the multi-precision multiplication module, or connected to a register in the same node structure, to indirectly obtain the second product result of the multi-precision multiplication module.

[0035] Further, refer to Figure 2 The multi-precision multiplication module includes a data preprocessing unit, 2N encoders, an encoding register, 2N compressors, and a compression register. The data preprocessing unit determines the precision and mantissa of the input data, and segments the mantissa data, sending each segment to one of the 2N encoders. The 2N encoders perform encoding operations, generating 2N partial products, which are then sent to the 2N compressors. The compressors compress the partial products, generating carry-chain data and local-chain data, which are then sent to the addition tree module for addition operations. By identifying the precision of the multiplier and multiplicand through the data preprocessing unit and segmenting the mantissa for different precision data to match the subsequent 2N encoders for encoding operations, parallelization and redundancy-free generation of partial products are achieved, reducing the limitation of bit width on multiplication latency. Furthermore, the 2N compressors enable parallel layered compression, reducing compression latency and waiting latency. This allows for the synchronization of operations on data of different precisions through multiple registers and compressors, ensuring that the partial product / compression results of high- and low-precision multiplications are aligned in time before being sent to the addition tree module. This avoids operational misalignment caused by different precisions and improves the operational accuracy of multi-precision multiplication.

[0036] refer to Figure 2An encoding register is set between the encoder and the compressor, and a compression register is set between the compressor and the addition tree module. Through register-level pipeline buffering, the pipelined execution of each stage of multiplication is realized. For example, after the preprocessing unit completes the preprocessing of the k-th group of input data, it is directly sent to the encoder; after the encoder completes the encoding of the k-th group of data, it stores the partial product in the encoding register and starts the encoding of the (k+1)-th group of data; while the compressor reads the partial product of the k-th group from the encoding register for compression, the encoder completes the encoding of the (k+2)-th group of data, realizing the pipelined parallel execution of preprocessing, encoding, compression, and addition, reducing computational latency.

[0037] To more clearly illustrate the multi-precision multiplication module described in this application, we will take the implementation of multiplication and addition operations with four different data precisions—FP8, BF16, FP16, and INT8—as an example. (Reference) Figure 2 The input data includes a multiplier X [63:0] and a multiplicand Y [63:0]. First, the input data is preprocessed, dividing the floating-point number into a sign bit, exponent bit, and mantissa bit. Then, the mantissa bit data of the multiplicand is segmented according to a data chain and sent to eight encoders to encode the corresponding multiplier data. The encoded partial product data then enters eight compressor modules for partial product compression. The resulting carry chain data and current chain data then enter the adder for addition operations, thus enabling multiplication operations of data with different precisions. Furthermore, in this embodiment, the multi-precision multiplier does not include a rounding module to avoid rounding affecting the data precision in subsequent addition operations.

[0038] The encoder described in this application is a radix-4 Booth encoder, such as... Figure 3As shown, the radix-4 Booth encoding encodes the multiplier by grouping the multiplicand into three bits. Multiple multiplicand data are linked to form a data chain, resulting in four multiplicand data chains for four different data precisions. The four precisions—FP8, BF16, FP16, and INT8—are implemented using eight encoders (B1-B7), each capable of encoding six multiplicand data bits (FP8, BF16, and FP16 require only three bits, while INT8 requires all six). For ease of description, each encoder's multiplicand data bit is referred to as a data packet. Each data packet contains 6 bits of multiplicand data. The next data packet uses the highest bit of the previous packet. When this bit is "0", it indicates encoding two different multiplicand data bits; when it is the original multiplicand data bit, it indicates encoding the same multiplicand data bit. This allows for encoding operations on multiplicand data of different precisions. To handle the sign bit of the partial product data, a sign bit preprocessing method is used. Specifically, the original multi-bit sign bit data preceding each partial product is transformed into pre-processed 2-bit data and placed before each partial product data. This reduces the computation on the multi-bit sign bit data. The specific encoding circuit is as follows: Figure 4 As shown, PP ji For partial product data, Unsign and sign are the sign bits of each partial product after encoding, and S is the sign bit of the multiplicand (used for INT8 data).

[0039] The Wallace tree structure contains 8 compressor units, divided into the first 4 compressor units (such as...). Figure 5 As shown), the last four compressor units (such as Figure 6 (As shown). Specifically, the first four modules are for compressing the partial product data after encoding all precision data. The first four compressor modules can perform partial product data compression operations on four INT8, BF16, FP16, and eight FP8 (two per compressor). The last four compressor units can individually perform compression operations on the remaining four partial product data of INT8. Internally, the compressor uses 3-2 compressor units for tree compression. This multi-precision multiplication module avoids the waste of hardware resources caused by using a single large multiplier to perform multiplication operations on data of different precisions. It can rationally allocate hardware resources according to the hardware resource requirements of different precision data, and achieve multiplication operations on data of different precisions through configuration, thus fully utilizing hardware resources and reducing computational latency. Therefore, the multi-precision multiplication module can achieve reasonable optimization of bandwidth and storage energy consumption when performing data operations on high and low precision data, reducing the coupling pressure between computing power and storage.

[0040] After compression through the Wallace tree, two data chains are obtained: a carry data chain and a current data chain. These two chains are input into the adder module for addition operations, ultimately achieving multiplication. Simultaneously, the multi-precision multiplication module described in this application implements multiple sets of multiplication operations with different precisions. In the design of this adder module, truncation operations are performed at some nodes to ensure that carry operations do not interfere with each other during different sets of operations, thus fulfilling the requirement that different sets of data be added separately.

[0041] refer to Figure 7 The processing unit further includes a weight register for storing weight data, an activation register for storing activation data, a pipeline register for caching intermediate calculation results, a second data selector MUX2, and a third data selector MUX3. The output registers include a first output register and a second output register. The input signal of the second data selector includes the calculation result of the previous processing unit and the calculation result of the current processing unit. The two input terminals of the second data selector are respectively connected to the first output register of the current stage and the first output register of the previous processing unit to implement a fixed-weight data stream mode. The input signal of the third data selector includes the calculation result of the next processing unit and the calculation result of the current processing unit. The two input terminals of the third data selector are respectively connected to the first output register of the current stage and the second output register of the next stage to implement a fixed-output data stream mode.

[0042] In summary, the multi-precision MAC tree processing unit described in this application replaces traditional serial accumulation with N-level parallel accumulation of the addition tree module. This eliminates the need for pipelined startup and cascading transfer required by traditional serial MAC, allowing matrix-vector multiplication and accumulation to be completed in a single cycle without reducing computational latency. This optimizes the computational latency and storage bottleneck issues in large language model deployment and inference, achieving high throughput, low latency, and high energy efficiency inference acceleration. Simultaneously, it achieves parallelization and redundancy-free generation of partial products, reducing the bit width limitation on multiplication latency and enabling parallel layered compression, thus reducing compression latency and waiting latency.

[0043] Based on the same inventive concept, and referring to the figures, embodiments of this application also provide a pulsating array structure, including a plurality of multi-precision MAC tree processing units as described in any of the above claims arranged in a matrix.

[0044] The pulsating array structure also includes a weight buffer connected to the processing unit, a weight buffer for inputting weight data to the processing unit, an input buffer for inputting activation data to the processing unit, a weight storage unit for storing the weight data, an activation storage unit for storing the activation data, and a control unit responsible for controlling the coordination between different modules of the entire pulsating array according to a clock. The weight buffer and input buffer are used to extract data from the weight storage unit and activation storage unit, respectively, and temporarily store multiple sets of weight data and activation data according to a certain arrangement rule based on data precision, providing input data to the processing unit array. Each processing unit has the same structure, and data flows between each processing unit. The accumulator module is responsible for accumulating the partial sums calculated by multiple processing units to complete the accumulation operation of the block matrix. The output of the processing unit array is connected to a two-to-one selector. If the output data is selected to flow from the bottom of the array, a fixed weight data flow mode is implemented; if the output data is selected to flow from the top of the array, a fixed output data flow mode is implemented.

[0045] In summary, the systolic array structure enables variable precision matrix multiplication and supports two data flow modes, thereby improving the computational parallelism and flexibility of the systolic array structure.

[0046] It should be noted that those skilled in the art can make various improvements and modifications to this application without departing from the principles thereof, and such improvements and modifications also fall within the scope of protection of the claims of this application. It should be understood that certain features of this disclosure described in the context of individual embodiments for clarity may also be provided in combination in a single embodiment. Conversely, various features of this disclosure described in the context of individual embodiments for clarity may also be provided individually or in any suitable combination or as part of any other described embodiment of this disclosure.

Claims

1. A multi-precision MAC tree-structured processing unit, characterized in that, Applied to a pulsating array structure, the processing unit includes a MAC tree structure for implementing multiplication-accumulation-addition operations, an accumulator for performing accumulation operations, and an output register for outputting the operation results; wherein, the MAC tree structure includes: The multi-precision multiplication module is used to perform data multiplication operations on multiple sets of first-precision data and second-precision data, and determine the corresponding first product result and second product result; the number of bits in the first-precision data is less than the number of bits in the second-precision data. The addition tree module includes a first-level structure and a second-level structure for determining the product addition result. The first-level structure includes N node structures, each node structure including an adder for receiving the first product result, performing addition operations, and generating a first addition result, and a first data selector. The two input signals of the first data selector are the first addition result and the second product result, respectively.

2. The multi-precision MAC tree processing unit according to claim 1, characterized in that, The multi-precision multiplication module includes a data preprocessing unit, 2N encoders, and 2N compressors. The data preprocessing unit is used to determine the precision of the input data and the mantissa of the input data, and to segment the mantissa into data and send it to the 2N encoders respectively. The 2N encoders are used to perform encoding operations, generate 2N partial products, and send them to the 2N compressors respectively. The compressors are used to perform partial product compression on the partial products, generate carry chain data and local chain data, and send them to the addition tree module for addition operations.

3. The multi-precision MAC tree processing unit according to claim 2, characterized in that, The number of N values ​​is related to the precision of the input data.

4. The multi-precision MAC tree processing unit according to claim 2, characterized in that, An encoding register is provided between the encoder and the compressor, and a compression register is provided between the compressor and the addition tree module.

5. The multi-precision MAC tree processing unit according to claim 1, characterized in that, The second-level structure includes N-1 adders arranged in a tree topology cascade.

6. The multi-precision MAC tree processing unit according to claim 5, characterized in that, Each adder is connected in series with a register.

7. The multi-precision MAC tree processing unit according to claim 6, characterized in that, One input of the first data selector is connected to a register in the same node structure, and the other input is connected to the multi-precision multiplication module.

8. The multi-precision MAC tree processing unit according to claim 1, characterized in that, The processing unit further includes a weight register for storing weight data, an activation register for storing activation data, a pipeline register for caching intermediate calculation results, a second data selector, and a third data selector. The input signal of the second data selector includes the calculation result of the previous processing unit and the calculation result of the current processing unit. The input signal of the third data selector includes the calculation result of the next processing unit and the calculation result of the current processing unit.

9. A pulsating array structure, characterized in that, It includes a plurality of multi-precision MAC tree processing units arranged in a matrix as described in any one of claims 1 to 7.

10. A pulsating array structure according to claim 9, characterized in that, The pulsating array structure includes a weight buffer connected to the processing unit, a weight buffer for inputting weight data to the processing unit, and an input buffer for inputting activation data to the processing unit. The weight buffer and the input buffer are used to arrange and temporarily store multiple sets of weight data and activation data according to the data precision.

Images