Matrix and vector operation oriented systolic array structure
By optimizing the connection and data flow of the systolic array, the computational parallelism and hardware resource utilization of the systolic array are improved, solving the problem of low hardware utilization in existing technologies and realizing efficient matrix-vector operations.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- BEIJING ZHONGKE YIHAI MICROELECTRONICS TECHNOLOGY RESEARCH INSTITUTE CO LTD
- Filing Date
- 2026-01-23
- Publication Date
- 2026-05-05
AI Technical Summary
Existing systolic arrays have low hardware utilization when performing matrix-vector operations, and cannot meet the needs of multiple sets of data operations in the decoding stage of large language model inference, resulting in low computational efficiency.
Design a systolic array structure for matrix and vector operations. By setting a first multiplexer between horizontally adjacent processors and a second multiplexer between vertically adjacent groups, optimize the data flow path and computation path, and improve parallelism and hardware resource utilization.
It improves computational density and parallelism, reduces computational latency, enhances hardware resource utilization and computational efficiency for multiple sets of data, and meets the matrix operation requirements of different properties and scales in the reasoning process of large language models.
Smart Images
Figure CN121979488A_ABST
Abstract
Description
Technical Field
[0001] This application belongs to the field of digital integrated circuit technology, and in particular relates to a pulsating array structure for matrix and vector operations. Background Technology
[0002] In recent years, the field of artificial intelligence has made revolutionary progress, one of the core driving forces being the rise of Large Language Models (LLMs). These models have demonstrated unprecedented capabilities in tasks such as natural language understanding, generation, translation, and reasoning. The most advanced LLMs currently available are based on the Transformer architecture. Proposed in 2017, the Transformer architecture features a unique self-attention mechanism and high parallelization capabilities. Most computational tasks in Transformer algorithms involve matrix-matrix and matrix-vector operations, characterized by very large matrix sizes and data volumes. The efficiency of these operations directly determines the throughput and energy efficiency of the system. Traditional von Neumann architecture processors, when performing these operations, frequently need to read operands from memory and write back intermediate results, creating a "memory wall" bottleneck. To alleviate this problem, the industry commonly employs on-chip multi-level caches, high-bandwidth HBMs, and dedicated acceleration units, but these are still limited by off-chip memory access bandwidth and power consumption.
[0003] In the large language model inference process based on Transformer, there are two stages: prefill and decoding. The prefill stage inputs the prompts into the model all at once to prepare for subsequent decoding. This stage is computationally intensive and highly parallel. In the decoding stage, new tokens are generated autoregressively based on the KV cache generated in the prefill stage and the first token. The problem with this stage is that it is memory and bandwidth intensive. The operation is a "small batch, high concurrency" matrix-vector multiplication (GEMV). Only the attention of one token needs to be calculated at a time, but the KV values need to be loaded frequently. Moreover, this process is a serial process and cannot be computed in large-scale parallelism by GPU. The current industry solution is to compute multiple values in batch processing to alleviate the disadvantage of low hardware unit utilization.
[0004] A systolic array is an array composed of a large number of repetitive processing units. Its core idea is to make data "flow," thereby transforming memory access overhead into on-chip data multiplexing, reducing the number of memory accesses, and making the structure more regular, the wiring more uniform, increasing the frequency, and achieving overlap between computation and communication. However, during the decoding stage, because it needs to perform matrix-vector operations, and multiple sets of data are processed simultaneously, systolic arrays suffer from low hardware utilization when performing matrix-vector operations. Furthermore, when using batch processing techniques, it cannot perform operations on multiple sets of data simultaneously, resulting in reduced computational efficiency. Summary of the Invention
[0005] This application discloses a systolic array structure for matrix and vector operations, which can improve computational density and parallelism, enhance hardware resource utilization during matrix and vector operations, and improve the efficiency of operations on multiple sets of data.
[0006] Other objectives and advantages of this application can be further understood from the technical features disclosed herein.
[0007] To achieve one or more of the above objectives or other objectives, in a first aspect, this application provides a pulsating array structure for matrix and vector operations, the pulsating array structure comprising: a plurality of processing units arranged in a matrix, each processing unit being connected horizontally via a first multiplexer, each processing unit being connected serially in the vertical direction, and each column consisting of a preset number of processing units as a group, with each group of processing units connected to each other via a second multiplexer.
[0008] Furthermore, the two input signals of the first multiplexer are broadcast activation data and activation data output by the previous processing unit, respectively; the two input signals of the second multiplexer are the output of the previous group of processing units and another input data path, respectively.
[0009] Furthermore, the first multiplexer is used to implement a broadcast connection of the activation data in the horizontal direction.
[0010] Furthermore, the processing unit includes: a weight register for loading weight data, an activation register for storing activation data, and an output register for outputting the registered calculation result data.
[0011] Furthermore, the weight registers of each group of processing units are connected serially in the vertical direction, and the weight data is input serially; the weight data between different groups of processing units uses different input data paths.
[0012] Furthermore, the activation data flows from left to right in the array, and performs data operations with the weight data in the processing unit to obtain partial sum data; the partial sum data flows vertically to the accumulator of the next-level processing unit; the accumulator performs an accumulation operation on the partial sum data.
[0013] Furthermore, activation data and weight data from the same group of processing units flow into the corresponding processing unit simultaneously; the time interval between the activation data and weight data from adjacent groups of processing units flowing into the corresponding processing unit is one clock cycle.
[0014] Furthermore, the output registers are connected in series vertically.
[0015] Furthermore, the weight register includes a first weight register and a second weight register, and the processing unit includes a third multiplexer for determining the weight data to be performed, wherein the two input signals of the third multiplexer are the first weight data from the first weight register and the second weight data from the second weight register, respectively.
[0016] Furthermore, the processing unit includes a fourth multiplexer for configuring the data stream mode and a fifth multiplexer for determining the output result. The two input signals of the fourth multiplexer are the operation result data of the previous processing unit and the operation result data of the current level, respectively. The two input signals of the fifth multiplexer are the operation result data of the current level and the operation result data output by the next level processing unit, respectively.
[0017] The aforementioned systolic array structure for matrix and vector operations has at least the following technical effects: 1) By setting a first multiplexer between horizontally adjacent processors and a second multiplexer between vertically adjacent groups, the data flow path and computation path are optimized, improving computational density and parallelism, and enhancing hardware resource utilization and computation efficiency for matrix and vector operations; 2) Without disrupting the original structure of the systolic array, the connection method of the systolic array, the data flow method of the processing unit, and the internal structure of the processing unit are optimized, improving computational parallelism and reducing computational latency; 3) By configuring the data flow method, the efficiency of matrix operations and computation of multiple groups of data is improved, while simultaneously satisfying matrix operations of different natures and scales in the pre-filling and decoding stages during the inference process of large language models, thereby improving hardware resource utilization.
[0018] To make the above and other objects, features and advantages of this application more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description
[0019] To more clearly illustrate the technical solutions in the specific embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0020] Figure 1 This is a schematic diagram of a pulsating array structure under a fixed-weight data flow method in the existing technology.
[0021] Figure 2 This is a schematic diagram of a pulsating array structure in the existing technology with a fixed output data stream.
[0022] Figure 3 This is a schematic diagram of the connection path of a processing unit in a pulsating array structure according to an embodiment of this application.
[0023] Figure 4 This is a schematic diagram of the overall structure of a pulsating array structure in an embodiment of this application.
[0024] Figure 5 This is a schematic diagram of the internal structure and connection path of a set of processing units in an embodiment of this application.
[0025] Figure 6 This is a schematic diagram of the structure of the multi-precision multiplier of the DOT calculation unit in the embodiments of this application.
[0026] Figure 7 This is a schematic diagram of the internal structure of the multi-precision multiplier in the embodiments of this application.
[0027] Figure 8 This is a schematic diagram of four precision multiplicative data segmentation methods in the embodiments of this application.
[0028] Figure 9 This is a circuit diagram of the encoder in the multi-precision multiplier described in the embodiments of this application.
[0029] Figure 10 This is a schematic diagram of the first four compressors of the multi-precision multiplier in the embodiments of this application.
[0030] Figure 11 This is a schematic diagram of the four compressors following the multi-precision multiplier in the embodiments of this application. Detailed Implementation
[0031] The foregoing and other technical contents, features, and effects of this application will be clearly presented in the following detailed description of a preferred embodiment with reference to the accompanying drawings. The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such terms can be used interchangeably where appropriate; this is merely a method of distinguishing objects with the same attributes in the embodiments of this application.
[0032] In their research, the inventors of this application discovered that there are currently two data flow methods for pulsating arrays: a fixed-weight data flow method and a fixed-output data flow method. The fixed-weight data flow method is described in reference [reference needed]. Figure 1 Weighted data B is pre-loaded into the weight register of each processing unit in the systolic array. The weight registers of each processing unit are serially connected. During registration, each clock cycle is executed. A set of data flows through the weight register of the processing unit, and after several clock cycles, the loading process of the array processing unit's weighted data is completed. The weight register includes a first weight register (Register1) and a second weight register (Register2), which are respectively connected to the two input ports of a data selector. The data selector enables simultaneous data storage and data computation during the operation. Activation data A flows from left to right in the systolic array, performing multiplication with the weighted data within each processing unit. The resulting data, along with data C, flows vertically to the input port of the accumulator module of the next layer's processing unit. The entire systolic array moves data according to the clock cycle, ultimately achieving matrix-matrix multiplication. The fixed output data stream method is described in reference [reference needed]. Figure 2 Activation data A and weight data B flow into the systolic array simultaneously, following a rule that the data flowing into the systolic array are delayed by one clock cycle. Within each processing unit, a multiply-add operation is performed on activation data A and weight data B. After the operation of a processing unit is completed, the resulting data O is output to an external storage unit using a column-wise concatenated output method.
[0033] The pulsating array structure under the above-mentioned fixed weight data stream and fixed output data stream methods can only perform one set of data operations in one clock cycle. When performing multiple sets of data operations in the decoding stage of the large language model inference process, it cannot adapt to the requirements of dynamic batch processing and streaming output, and wastes hardware resources, with a high proportion of memory access overhead, resulting in low hardware utilization and low computing efficiency.
[0034] To improve hardware resource utilization and computing efficiency, such as Figure 3As shown in the illustration, this application provides a systolic array structure for matrix and vector operations. The systolic array structure includes multiple processing units arranged in a matrix. Each processing unit is connected horizontally via a first multiplexer (MUX1) and vertically in series. Each column consists of a preset number of processing units as a group, and each group of processing units is connected via a second multiplexer (MUX2). By placing the first multiplexer between horizontally adjacent processors and the second multiplexer between vertically adjacent groups, the data flow and computation path are optimized, improving computational density and parallelism, and enhancing hardware resource utilization and computational efficiency for multiple data sets during matrix and vector operations.
[0035] It should be noted that, Figure 3 The example pulsation array is 8*8 in size, but it can be expanded to 16*16 or other sizes by copying, without any specific limitation here.
[0036] Further, refer to Figure 4 The overall functional architecture of the pulse array processing unit described in this application embodiment includes: a controller for controlling array operations, a weight input buffer and an activation input buffer for buffering input data streams, a processing unit array, an activation storage unit, a weight storage unit, an output storage unit, and an accumulator module.
[0037] refer to Figure 5 This diagram illustrates the internal structure and connection pathways of a group of four processing units. Each processing unit includes: a weight register for loading weight data; an activation register for storing activation data; a Dot-Operation Array (DOT) unit for performing multi-precision multiplication and multiply-add operations; a pipeline register for storing the result data of the DOT unit; a chain register for storing the result data of the previous processing unit; an accumulator for performing summation operations on partial sums within the current processing unit or cascaded processing units; and an output register for outputting the stored calculation result data. Specifically, the computation process of one processing unit is part of the overall pipeline of the systolic array. Each processing unit performs independent but coordinated calculations sequentially each clock cycle. Taking matrix multiplication as an example, the execution flow of the processing unit can be: the processing unit receives input activation data from the left (adjacent processing unit or array boundary), stores it in the activation register, the weight register loads weight data, and the DOT unit performs a vector dot product operation on the weight data and the activation data. Meanwhile, the processing unit receives a portion of the data from above (adjacent processing units or array boundaries) and stores it in a chained input register so that it can perform an accumulation operation with the portion of the data from the processing unit at the same level through an accumulator.
[0038] In some embodiments, the two input signals of the first multiplexer are broadcast activation data and activation data output by the previous processing unit, respectively; the two input signals of the second multiplexer are the output of the previous group of processing units and another input data path, respectively. The first multiplexer is used to implement a horizontal broadcast connection or a serial shift connection of the activation data, that is, to select the output of broadcast activation data or the activation data after horizontal concatenation through the first multiplexer. The option of implementing horizontal broadcast connection of activation data through the first multiplexer enables the reuse of the same activation data in each column of processing units, improving computational parallelism. The second multiplexer enables the switching of different data streams, thereby realizing a broadcast output of a fixed data stream, improving computational parallelism and enhancing the computational flexibility of the system architecture.
[0039] For example, when implementing matrix-vector operations, horizontally, the activation data is broadcast-connected through the first multiplexer, enabling the reuse of the same activation data in each column of the matrix, and each row processing unit can perform one matrix-vector operation; vertically, the second multiplexer enables the switching of different data streams, and the first processing unit of each group of processing units selects the weight data input through the second multiplexer. The weight data is loaded according to each clock cycle, and after loading, it performs data operations with the activation data, thus enabling multiple sets of matrix-vector operations.
[0040] In some embodiments, the weight registers of each group of processing units are connected serially in the vertical direction, and the weight data is input serially; different groups of processing units use different input data paths for their weight data. Specifically, multiple sets of weight data are pre-loaded into the weight register of each processing unit, and the weight registers of each processing unit are connected serially. During registration, the weight data flows through the weight register of the processing unit according to the clock cycle, and the loading process of the array processing unit weight data is completed after multiple clock cycles. The input activation data is sent into the processing unit in a trapezoidal queue. The activation data flows from the left to the right of the array and performs data operations with the weight data in the processing unit to obtain partial sum data. The partial sum data flows vertically to the accumulator of the next-level processing unit; the accumulator performs the accumulation operation on the partial sum data. The entire pulsating array moves the data according to the clock cycle, flowing once per cycle, and finally realizes a data flow mode with fixed weights to achieve matrix multiplication operations. For example, for 128 sets of data, this application can simultaneously send 8 sets of data into a processing unit, and only needs 16 cycles to complete the input of the entire activation data. Compared with the 128 cycles required by the prior art, this application greatly improves the parallelism of the operation.
[0041] In some embodiments, activation data and weight data from the same group of processing units flow into the corresponding processing unit simultaneously; the time interval between the flow of activation data and weight data from adjacent groups of processing units into their corresponding processing units is one clock cycle, achieving a fixed output data stream. Specifically, the activation data and weight data flow into the systolic array simultaneously, and according to the rule that the data flowing into the array are delayed by one clock cycle, the activation data and weight data are multiplied and added in a multiply-accumulate manner within each processing unit to obtain the multiply-accumulate result; after the processing unit finishes its operation, the result data can be obtained. Compared with traditional systolic arrays, the systolic array described in this embodiment is a composite systolic array, which reduces weight loading time and computation time, reduces latency, and improves computational parallelism without changing the bandwidth and scale.
[0042] In some embodiments, reference Figure 5 The weight registers include a first weight register (Register1) and a second weight register (Register2). The processing unit includes a third multiplexer (MUX3) for determining the weight data to be used in the operation. The two input signals of the third multiplexer are the first weight data from the first weight register and the second weight data from the second weight register, respectively. This ensures that data storage and data operation are performed simultaneously during the operation, and that the operation and data loading processes do not interfere with each other.
[0043] In some embodiments, reference Figure 5 The output registers include a first output register and a second output register. The first output register is used in fixed-weight data stream mode to output the calculation result of the current processing unit, and the second output register is used in fixed-data stream mode to output the calculation result of the current processing unit. The processing unit includes a fourth multiplexer (MUX4) for configuring the data stream mode and a fifth multiplexer (MUX5) for determining the output result. The input of the fourth multiplexer is connected to the chain register and the first output register. The two input signals of the fourth multiplexer are the calculation result data of the previous processing unit and the calculation result data of the current level, respectively, to select the output result of the cascaded previous processing unit to achieve fixed-weight data stream mode, or to select the accumulation operation of the current level to achieve fixed-output data stream mode. The input of the fifth multiplexer is connected to the first output register of the current level processing unit and the second output register of the next level processing unit. The two input signals of the fifth multiplexer are the calculation result data of the current level and the calculation result data output by the next level processing unit, respectively, to achieve output selection in fixed-data stream mode, that is, to select the output of the current level processing unit or the next level processing unit.
[0044] The pulsating array structure described in this embodiment implements a fixed-weight data stream when the output signal of the fourth multiplexer is the computation result data of the previous processing unit. When the output signal of the fourth multiplexer is the computation result data of the current level, it implements a fixed-output data stream. When the output signal of the fourth multiplexer is the computation result data of the current level, and the output signal of the first multiplexer is broadcast activation data, it implements a broadcast-style fixed-output data stream. Compared to the traditional fixed-output data stream, the broadcast-style fixed-output data stream described in this application enables multiple sets of weight data and activation data to enter the pulsating array in parallel, without requiring a one-clock delay, greatly reducing computational latency and improving computational efficiency.
[0045] In some embodiments, the DOT calculation unit includes a multi-precision multiplier, which is used to perform multiplication operations on multiple sets of data with different precisions. For example, taking the implementation of 8 sets of INT8 and FP8 multiplication operations and 4 sets of FP16 and BF16 multiplication operations by the multi-precision multiplier as an example, refer to... Figure 6 The multi-precision multiplier adopts a MAC tree structure as the multiply-add unit for multiple sets of data in the systolic array processing unit. Specifically, it includes multiple adders and a register (Reg) for storing intermediate results. This structure enables multiply-add operations on multiple sets of data, reducing waiting latency between the first and last columns and overall computational latency. It also avoids the problems of discrete structures performing poorly in multi-precision data operations and having low hardware resource utilization, while preventing the inability to finely utilize hardware resources for multiplication operations of different precision data, thus improving the efficiency of multi-precision data operations. The multi-precision multiplier described in this application achieves fine-grained utilization of hardware resources for operations on different precision data by modifying the structure of the radix-4 Booth encoder and the Wallace adder tree, reducing the hardware circuit area of the multiplier and minimizing computational latency.
[0046] The internal structure reference of the multiprecision multiplier Figure 7Based on the encoding characteristics of radix-4-Booth encoding, the multiplicand data is encoded in groups of 3 bits, and the multiplier data is encoded accordingly. This characteristic allows for the implementation of multiplicand data of different precisions to be combined into a unified data chain of equal width according to certain rules. This data chain is then divided into multiple data packets and sent to multiple encoders (B1-B8) for multiplication. The resulting partial sums require addition operations, necessitating compression using a Wallace tree (W1-W8). The results are then fed into the addition tree to perform addition operations on the carry chain and the current chain, ultimately achieving multiplication of multiple data with different precisions. Based on this, this application designs a data chain and a corresponding encoder, and after encoding, a compressor structure to perform partial sum data compression operations under different conditions. Finally, an adder completes the addition operation to obtain the final multiplication result.
[0047] refer to Figure 8 This application designs four different data chains and segmentation methods for different precision data. The four precisions (FP8, BF16, FP16, and INT8) are implemented using eight encoders. Each encoder encodes three multiplier data points, resulting in three partial product data lines. For ease of description later, the multiplicand data of each encoder is referred to as a data packet. Each data packet contains 6 bits of multiplicand data. The next data packet uses the highest bit of the previous data packet. When this bit is "0", it indicates encoding two different multiplier data points; when it is the original multiplicand data, it indicates encoding the same multiplier data point. This achieves the encoding operation of multiplier data with different precisions. To handle the sign bit of the partial product data, a sign bit preprocessing method is used. Specifically, the original multi-bit sign bit data at the beginning of each partial product is converted into preprocessed 2-bit data and placed before each partial product data line. This reduces the computation on the multi-bit sign bit data. For the specific encoding circuit, refer to [reference needed]. Figure 9 As shown, PPji is the partial product data, Unsign and sign are the sign bits of each partial product after encoding, and S is the sign bit of the multiplicand (used for INT8 data).
[0048] The Wallace tree structure contains 8 compressor units, divided into the first 4 compressor units (such as...). Figure 10 As shown), the last four compressor units (such as Figure 11As shown in the diagram, the first four compressors are for compressing the six partial product data points after encoding all precision data. Each encoder outputs three partial product data points, and the compression operation differs depending on the data type. Specifically, for FP8 data precision, each compressor can independently perform compression operations on two FP8 data points (each FP8 data point has three partial products), and the corresponding partial product data points of the two data points do not affect each other during compression and are operated independently. For BF16, FP16, and INT8 data precision, each compressor can complete the compression operation on one data point (each data point has three partial products, and the corresponding partial product data points for each data point need to be obtained through two encoders, so a total of six partial product data points are required). The last four compressor units are set up to implement the remaining four INT8 data points. The compressor internally uses a 3-2 compressor unit to perform the compression tree compression operation. Such a multi-precision multiplier avoids the waste of hardware resources caused by using a single large multiplier to perform multiplication operations on data of different precisions. The multiplier of this application can reasonably allocate hardware resources according to the hardware resource requirements of different precision data, and realize multiplication operations on data of different precisions through configuration, thereby making full use of hardware resources and reducing computational latency.
[0049] After compression through the Wallace tree, two data chains are obtained: a carry data chain and a current data chain. These two chains are input into an adder for addition operations, ultimately achieving multiplication. Since the multi-precision multiplier described in this application performs multiple sets of multiplication operations with different precisions, the adder design requires truncation operations at some nodes to ensure that carry operations do not interfere with each other during different sets of operations, thus fulfilling the requirement that different sets of data be added separately.
[0050] In summary, the systolic array structure for matrix and vector operations described in this application can implement three data flow modes: fixed-weight data flow, fixed-output data flow, and broadcast-type fixed-output data flow. This application optimizes the data flow and computation paths by setting a first multiplexer between horizontally adjacent processors and a second multiplexer between vertically adjacent groups, thereby improving computational density and parallelism, and enhancing hardware resource utilization and the efficiency of multi-group data operations during matrix and vector operations. Without disrupting the original structure of the systolic array, this application optimizes the connection method of the systolic array, the data flow mode of the processing units, and the internal structure of the processing units, improving computational parallelism and reducing computational latency. This application improves the efficiency of matrix operations and multi-group data operations by configuring the data flow mode, while simultaneously satisfying matrix operations of different natures and scales during the pre-filling and decoding stages of large language model inference, thus improving hardware resource utilization. The multi-precision multiplier described in this application rationally allocates hardware resources according to the hardware resource requirements of different precision data, achieving multiplication operations of different precision data through configuration, fully utilizing hardware resources, and reducing computational latency.
[0051] It should also be noted that the device embodiments described above are merely illustrative, and some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.
[0052] It should be noted that those skilled in the art can make various improvements and modifications to this application without departing from the principles thereof, and such improvements and modifications also fall within the scope of protection of the claims of this application. It should be understood that certain features of this disclosure described in the context of individual embodiments for clarity may also be provided in combination in a single embodiment. Conversely, various features of this disclosure described in the context of individual embodiments for clarity may also be provided individually or in any suitable combination or as part of any other described embodiment of this disclosure.
Claims
1. A pulsating array structure for matrix and vector operations, characterized in that, The pulse array structure includes: multiple processing units arranged in a matrix, each processing unit is connected horizontally through a first multiplexer, each processing unit is connected serially in the vertical direction, and each column consists of a preset number of processing units as a group, with each group of processing units connected to each other through a second multiplexer.
2. The pulsating array structure for matrix and vector operations according to claim 1, characterized in that, The two input signals of the first multiplexer are broadcast activation data and activation data output by the previous processing unit, respectively; the two input signals of the second multiplexer are the output of the previous group of processing units and another input data path, respectively.
3. The pulsating array structure for matrix and vector operations according to claim 2, characterized in that, The first multiplexer is used to implement a horizontal broadcast connection of the activation data.
4. The pulsating array structure for matrix and vector operations according to claim 1, characterized in that, The processing unit includes: a weight register for loading weight data, an activation register for storing activation data, and an output register for outputting the stored calculation result data.
5. A pulsating array structure for matrix and vector operations according to claim 4, characterized in that, The weight registers of each processing unit are connected serially in the vertical direction, and the weight data is input serially; the weight data of different processing units use different input data paths.
6. A pulsating array structure for matrix and vector operations according to claim 5, characterized in that, The activation data flows from left to right in the array, and performs data operations with the weight data in the processing unit to obtain partial sum data; the partial sum data flows vertically to the accumulator of the next-level processing unit. The accumulator performs an accumulation operation on the portion and the data.
7. A pulsating array structure for matrix and vector operations according to claim 4, characterized in that, Activation data and weight data from the same group of processing units flow into the corresponding processing unit simultaneously; the time interval between the flow of activation data and weight data from adjacent groups of processing units into the corresponding processing unit is one clock cycle.
8. A pulsating array structure for matrix and vector operations according to claim 7, characterized in that, The output registers are connected serially in the vertical direction.
9. A pulsating array structure for matrix and vector operations according to claim 4, characterized in that, The weight register includes a first weight register and a second weight register. The processing unit includes a third multiplexer for determining the weight data to be performed. The two input signals of the third multiplexer are the first weight data from the first weight register and the second weight data from the second weight register, respectively.
10. A pulsating array structure for matrix and vector operations according to claim 9, characterized in that, The processing unit includes a fourth multiplexer for configuring the data stream mode and a fifth multiplexer for determining the output result. The two input signals of the fourth multiplexer are the operation result data of the previous processing unit and the operation result data of the current level, respectively. The two input signals of the fifth multiplexer are the operation result data of the current level and the operation result data output by the next level processing unit, respectively.