Multi-stage time division nested pipeline systolic array and systolic array accelerator
Through the multi-level time-division nested flow pulsation array design, the frequency restriction problem of traditional pulsation arrays on the FPGA platform is solved, and efficient computing throughput is achieved, and suitable for scenarios such as neural network training.
Patent Information
- Application Number
- CN202510556306.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-04-29
- Publication Date
- 2025-08-12
AI Technical Summary
When traditional pulsating arrays perform high-precision floating-point operations on the FPGA platform, the internal calculation delay of PE processing units becomes a system bottleneck, and conventional pipeline cutting methods destroy data dependencies and lead to calculation errors.
The multi-level time division nested flow array design is adopted, and the multi-level flow parallel processing matrix multiplication task is realized by introducing k-level pipeline structure and time division multiplexing technology inside the PE processing unit to decouple the data dependency relationship.
It significantly improves the operating frequency and computing throughput, while maintaining the efficient data flow characteristics of the pulsating array, and is suitable for computing-intensive applications such as neural network training.
Smart Images
Figure CN120471124A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of systolic arrays, and in particular relates to a multi-stage time-division nested pipeline systolic array and a systolic array accelerator. Background Art
[0002] Matrix multiplication is the core operation in neural network computing. Systolic arrays, as a highly efficient hardware acceleration architecture, are widely used for matrix multiplication. Compared to traditional single instruction, multiple data (SIMD) architectures, systolic arrays significantly reduce on-chip memory bandwidth requirements by enabling direct data exchange between PEs.
[0003] The three mainstream computing architectures currently exist include single instruction multiple data (SIMD) architecture, data flow architecture, and systolic array architecture. Among them: SIMD architecture is simple to implement, but storage bandwidth becomes a performance bottleneck; data flow architecture can reduce storage access, but the control complexity is high; systolic array architecture combines high efficiency and regularity, but faces special challenges in FPGA (field programmable gate array) implementation.
[0004] When systolic arrays perform high-precision floating-point operations on FPGAs, computational latency within their PEs (Processing Elements) becomes a system bottleneck. While conventional approaches can achieve pipeline design by splitting logic links, this disrupts the data dependencies within the systolic arrays, leading to computational errors. Therefore, maintaining the advantages of systolic array architecture while effectively improving their computing performance and resource utilization on FPGA platforms has become a key technical challenge that needs to be addressed. Summary of the Invention
[0005] The purpose of the present invention is to provide a multi-stage time-division nested pipeline systolic array and a systolic array accelerator to solve the above-mentioned problems existing in the prior art.
[0006] In order to achieve the above object, the present invention adopts the following technical solutions:
[0007] In a first aspect, the present invention provides a multi-level time-division nested pipeline systolic array, comprising N×N homogeneous processing units PE, where N is a positive integer, and the N×N homogeneous processing units PE are arranged and connected in a grid form to form an N×N two-dimensional array structure, wherein the processing unit PE is internally integrated with a floating-point fused multiplication-addition operation unit FMA and an accumulation register reg, wherein the floating-point fused multiplication-addition operation unit FMA has a k-level pipeline structure and is used to complete multiplication and addition operations within k clock cycles, where k is a positive integer, and the floating-point fused multiplication-addition operation unit FMA stores the operation result of the current clock cycle in the accumulation register reg, and the accumulation register reg feeds back the operation result of the current clock cycle to the floating-point fused multiplication-addition operation unit FMA through a feedback loop as the accumulation input for its next clock cycle operation.
[0008] In one possible design, the processing unit PE includes a left input interface Left_in and an upper input interface Top_in, which are respectively used to receive matrix A elements from the left and matrix B elements from the upper side, and respectively pass the matrix A elements from the left and the matrix B elements from the upper side to the floating-point fused multiplication and addition unit FMA for multiplication and addition operations. The processing unit PE includes a unit output interface PE_out, which is used to output the unit final result accumulated by the processing unit PE after completing k matrix multiplications.
[0009] In a second aspect, the present invention provides a multi-stage time-division nested pipeline systolic array accelerator, comprising a control logic module, a matrix synchronization module, an input scheduling module A, an input scheduling module B, and the systolic array of the first aspect, wherein the control logic module is used to receive input data from an external host and parse it to obtain a matrix data stream and transmit it to the matrix synchronization module; the matrix synchronization module is used to receive asynchronously arriving matrix data streams and perform bit width conversion, and distribute the matrix data streams after bit width conversion to two independent output channels according to a split point dynamically set based on the matrix dimension, and output matrix A data to the input scheduling module A and output matrix B data to the input scheduling module B through the two independent output channels. Block B, the input scheduling module A is used to delay the matrix A data according to the data delay mechanism dynamically set based on the matrix dimension, obtain the matrix A elements arranged in a diagonal time sequence, and input the matrix A elements into the systolic array from the left side in sequence. The input scheduling module B is used to delay the matrix B data according to the data delay mechanism dynamically set based on the matrix dimension, obtain the matrix B elements arranged in a diagonal time sequence, and input the matrix B elements into the systolic array from the top side in sequence. The final results of each unit of the systolic array are aggregated into output data for output buffering, and finally fed back to the control logic module, which transmits the output data to the external host.
[0010] In one possible design, the matrix synchronization module includes a data width converter and a data distribution controller. The data width converter is used to convert the bit width of the asynchronously arriving matrix data stream, and the data distribution controller is used to distribute and process the matrix data stream after the bit width conversion according to a split point dynamically set based on the matrix dimension.
[0011] In a possible design, the matrix synchronization module further includes two FIFO buffers, which are respectively used to buffer and guide the matrix A data and matrix B data distributed by the data distribution controller to two independent output channels.
[0012] In one possible design, the control logic module includes an instruction decoding unit, which is used to perform operation code analysis on input data from an external host to obtain a decoding result.
[0013] In one possible design, the control logic module includes an address generator, which is used to generate cache addresses of matrix A data, matrix B data, and output data according to the decoding results.
[0014] In one possible design, the control logic module includes a state controller, which is used to control the operating states of the matrix synchronization module, the input scheduling module A, the input scheduling module B, and the systolic array.
[0015] In one possible design, the control logic module includes an enable signal generator, which is used to generate various activation signals and activate the matrix synchronization module, input scheduling module A, input scheduling module B and systolic array respectively through the various activation signals.
[0016] In one possible design, the control logic module includes a counter, which is used to provide a timing reference for controlling the matrix synchronization module, the input scheduling module A, the input scheduling module B, and the systolic array.
[0017] Beneficial effects: The multi-level time-division nested pipeline systolic array designed by the present invention solves the key technical difficulties in realizing a high-performance systolic array architecture on an FPGA platform. When a traditional systolic array realizes high-precision floating-point operations, the complex calculations inside the PE processing unit lead to severe limitations on the clock frequency, and the conventional pipeline cutting method will destroy the data flow order of the systolic array. The present invention realizes multi-level pipelines inside the PE processing unit by innovatively introducing time-division multiplexing technology, which can simultaneously process different matrix multiplication tasks in each time slice, effectively decoupling the data dependencies between PE processing units. The design of the present invention can significantly improve the operating frequency and computing throughput, while maintaining the original efficient data flow characteristics of the systolic array, and is suitable for computationally intensive application scenarios such as neural network training. BRIEF DESCRIPTION OF THE DRAWINGS
[0018] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
[0019] Figure 1 Schematic diagram of the architecture of the systolic array in Example 1 of the present invention;
[0020] Figure 2 Schematic diagram of the internal structure of the processing unit PE in Example 1 of the present invention;
[0021] Figure 3 Schematic diagram showing the comparison of the effects of the multi-stage time-division nested pipeline solution in Example 1 of the present invention;
[0022] Figure 4 Schematic diagram of the architecture of the systolic array accelerator in Example 2 of the present invention;
[0023] Figure 5 Schematic diagram of the control logic module in Example 2 of the present invention;
[0024] Figure 6 Schematic diagram of the architecture of the matrix synchronization module in Example 2 of the present invention;
[0025] Figure 7 Schematic diagram of data rearrangement of the input scheduling module in embodiment 2 of the present invention. DETAILED DESCRIPTION
[0026] It should be noted that the description of these embodiments is intended to help understand the present invention, but does not constitute a limitation of the present invention. The specific structural and functional details disclosed herein are merely intended to describe exemplary embodiments of the present invention. However, the present invention may be embodied in a variety of alternative forms, and should not be construed as being limited to the embodiments set forth herein.
[0027] It should be understood that, unless otherwise expressly specified or limited, the corresponding terms should be understood in a broad sense. For example, "connection" can refer to a fixed connection, a detachable connection, or an integral connection; it can refer to an electrical connection, a direct connection, an indirect connection through an intermediate medium, or internal communication between two components. Those skilled in the art will be able to understand the specific meanings of the above terms in the embodiments based on specific circumstances.
[0028] In the following description, certain details are provided to facilitate a complete understanding of the example embodiments. However, one of ordinary skill in the art will appreciate that the example embodiments can be practiced without these specific details. For example, a system may be shown in a block diagram to avoid obscuring the example with unnecessary detail. In other embodiments, well-known processes, structures, and techniques may be shown without unnecessary detail to avoid obscuring the embodiment.
[0029] Example 1:
[0030] This embodiment provides a multi-level time-division nested pipeline systolic array, such as Figures 1 to 2As shown, it includes N×N homogeneous processing units PE, N is a positive integer, and the N×N homogeneous processing units PE are arranged and connected in a grid form to form an N×N two-dimensional array structure. The processing unit PE is internally integrated with a floating-point fused multiplication-addition operation unit FMA and an accumulation register reg. The floating-point fused multiplication-addition operation unit FMA has a k-level pipeline structure and is used to complete multiplication and addition operations within k clock cycles. K is a positive integer. The floating-point fused multiplication-addition operation unit FMA stores the operation result of the current clock cycle in the accumulation register reg. The accumulation register reg feeds back the operation result of the current clock cycle to the floating-point fused multiplication-addition operation unit FMA through a feedback loop as the accumulation input for its next clock cycle operation. The processing unit PE includes a left input interface Left_in and an upper input interface Top_in, which are respectively used to receive matrix A elements from the left and matrix B elements from the upper side, and respectively pass the matrix A elements from the left and the matrix B elements from the upper side to the floating-point fused multiplication and addition unit FMA for multiplication and addition operations. The processing unit PE includes a unit output interface PE_out, which is used to output the unit final result accumulated by the processing unit PE after completing k matrix multiplications.
[0031] The core of the systolic array is an N×N array of homogeneous processing elements (PEs), capable of efficiently handling matrix multiplication operations. Each PE integrates a floating-point fused multiply-add (FMA) unit with a k-stage pipeline structure, capable of processing data from k different tasks in parallel. It performs a multiplication-add operation of a×b+c. Matrix A and matrix B elements enter through the left input interface Left_in and the top input interface Top_in, respectively. After the FMA operation, the result is stored in an accumulation register, forming a feedback loop that uses the current result as the accumulation input c for the next calculation. Simultaneously, the operation data is passed to adjacent PEs through a k-stage pipeline, namely, through the right output interface Right_out (corresponding to the left input interface Left_in) and the bottom output interface Bottom_out (corresponding to the top input interface Top_in), to the adjacent PE to the right and the adjacent PE below, respectively. A k-stage delay register chain controls the timing of data propagation. Due to the k-stage time-sharing pipeline structure, data from different tasks is interleaved within a single PE, each passing through the system at a fixed time interval. When k complete matrix multiplications are completed, the accumulated final results are output from PE_out. This time-division multiplexing mechanism significantly improves processor throughput and hardware resource utilization.
[0032] The basic working principle of the systolic array is to construct a regular array of processing units and adopt a data flow architecture so that data can be propagated in the array in an orderly manner like a pulse under the clock drive. For the two-dimensional matrix multiplication operation X·W=Y, the two-dimensional systolic array can be divided into two implementation methods: accumulation type processing unit and multiplication transfer type processing unit according to the functional characteristics of PE. Since neural network training requires frequent updates of weight parameters, this solution adopts multiplication transfer type processing unit. In this structure, X is input by row, W is input by column, and the intermediate calculation result Y is stored in each PE. The basic operation performed by each PE is a multiplication-accumulation operation, which can be expressed as:
[0033] O ij (t+1)=0 ij (t)+X i (t)·W j (t)
[0034] Among them, O ij (t) represents PE ij The cumulative result at time step t, Xi(t) represents the data input from the left, W j (t) represents the weight input from the upper side.
[0035] like Figure 3 As shown, the effect comparison of the multi-level time-division nested pipeline solution of this embodiment is shown, wherein, Figure 3 (a) shows the computational process and results as data flows when the pipeline is not split. Each PE takes three pieces of data, A, B, and C, as input (A and B are the matrix elements of the left and top inputs, and C is the intermediate result of the computation stored in the PE accumulator register). It performs a multiplication-accumulation operation, R = A × B + C, and passes the result to the next PE. Figure 3 (b) shows the data asynchrony problem after cutting the pipeline in the conventional way (compared to Figure 3 (a) When the water flow is not cut). Figure 3 (c) shows the multi-level time-division nested pipeline solution proposed in this embodiment, which cuts the PE logic path into two pipelines and introduces time-division multiplexing technology to process the data of two different matrix calculation tasks in every two clock cycles.
[0036] For a traditional systolic array, the iteration of its calculation process can be expressed as:
[0037]
[0038] The multi-stage time-division nested pipeline solution of this embodiment can process k independent matrix multiplication tasks simultaneously for a k-stage pipeline. The iteration of the calculation process is expressed as:
[0039]
[0040] In actual hardware implementation, k tasks are executed in an interleaved manner in time, and the tth clock cycle processes the relevant data of the task (tmodk). This interleaved execution mode can be expressed as Task active (t) = tmodk.
[0041] Theoretically, a k-level time-shared nested pipeline can increase the operating frequency to approximately k times and the throughput to approximately k times that of the basic design. The additional hardware overhead introduced is 2(k-1)×S registers, where S is the total number of processing units (PEs) in the systolic array.
[0042] Example 2:
[0043] This embodiment provides a multi-stage time-division nested pipeline systolic array accelerator, such as Figure 4 As shown, it includes a control logic module, a matrix synchronization module, an input scheduling module A, an input scheduling module B and the systolic array in Example 1, the control logic module is used to receive input data from an external host and parse it to obtain a matrix data stream and transmit it to the matrix synchronization module, the matrix synchronization module is used to receive the asynchronously arrived matrix data stream and perform bit width conversion, and distribute the matrix data stream after bit width conversion to two independent output channels according to the split point dynamically set based on the matrix dimension, and output the matrix A data to the input scheduling module A and the matrix B data to the input scheduling module B through the two independent output channels, the input scheduling module A is used to dynamically divide the matrix data stream according to the matrix dimension. The data delay mechanism dynamically set based on the matrix dimension is used to delay the matrix A data to obtain the matrix A elements arranged in a diagonal time sequence, and the matrix A elements are sequentially input into the systolic array from the left side. The input scheduling module B is used to delay the matrix B data according to the data delay mechanism dynamically set based on the matrix dimension to obtain the matrix B elements arranged in a diagonal time sequence, and the matrix B elements are sequentially input into the systolic array from the top side. The final results of each unit of the systolic array are aggregated into output data for output buffering and finally fed back to the control logic module. The control logic module transmits the output data to the external host, and the external host saves the output data to the DRAM memory.
[0044] The control logic module is mainly responsible for coordinating the working sequence and data flow of each functional unit, such as Figure 5As shown, the control logic module includes an instruction decoding unit, an address generator, a state controller, an enable signal generator, and a counter. The instruction decoding unit is used to parse the input data of the external host according to the operation code (inst[31:29]), obtain the decoding result, and determine the matrix dimension and operation type. The address generator is used to generate the cache address of the matrix A data, the matrix B data, and the output data according to the decoding result. The control logic module includes a state controller, which is used to control the operating state of the matrix synchronization module, input scheduling module A, input scheduling module B, and the systolic array (through control signals such as hold and gemm_sig). The enable signal generator is used to generate various activation signals (ena_a, ena_b, etc.) and activate the matrix synchronization module, input scheduling module A, input scheduling module B, and the systolic array respectively through each activation signal. The counter is used to provide a timing reference for the control of the matrix synchronization module, input scheduling module A, input scheduling module B, and the systolic array, triggering state transitions and address updates.
[0045] The main function of the matrix synchronization module is to receive the asynchronously arriving matrix data stream, perform appropriate bit width conversion and buffering, and then accurately distribute the data to two independent output channels according to the split point dynamically set based on the matrix dimensions. Figure 6 As shown, the matrix synchronization module includes a data width converter, a data distribution controller and two FIFO buffers. The data width converter is used to convert the bit width of the asynchronously arriving matrix data stream. The data distribution controller includes a distribution control logic part (counting the counter and calculating the segmentation threshold based on the control signal) and a signal selector, which is used to distribute and process the matrix data stream after the bit width conversion according to the segmentation point dynamically set based on the matrix dimension. The two FIFO buffers are used to buffer and guide the matrix A data and matrix B data distributed by the data distribution controller to two independent output channels.
[0046] Systolic array processing requires a specific data input pattern. The input data must be arranged diagonally and padded with zeros at appropriate positions, such as Figure 7 As shown, input scheduling modules A and B employ a data rearrangement architecture based on delay cells, known as a data delay mechanism. To accommodate k-level time-division multiplexing, a pyramidal array of delay cells applies varying degrees of delay to the input data to generate the correct timing pattern. Each square represents a delay cell with a delay of k clock cycles. For an n×n systolic array with k pipeline stages, when matrix data is input row by row, the i-th row undergoes a delay of (i-1)k clock cycles (i = 1, 2, ..., n). This delay mechanism converts conventionally arranged matrix data into a diagonal data stream that meets the processing requirements of the systolic array.
[0047] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention shall be included in the scope of protection of the present invention.
Claims
1. A multi-stage time-division nested pipeline systolic array, characterized in that: The invention comprises N×N homogeneous processing units PE, where N is a positive integer, and the N×N homogeneous processing units PE are arranged and connected in a grid form to form an N×N two-dimensional array structure. A floating-point fused multiplication-addition operation unit FMA and an accumulation register reg are integrated inside the processing unit PE. The floating-point fused multiplication-addition operation unit FMA has a k-level pipeline structure and is used to complete multiplication and addition operations within k clock cycles, where k is a positive integer. The floating-point fused multiplication-addition operation unit FMA stores the operation result of the current clock cycle in the accumulation register reg. The accumulation register reg feeds back the operation result of the current clock cycle to the floating-point fused multiplication-addition operation unit FMA through a feedback loop as the accumulation input for its next clock cycle operation.
2. A multi-stage time-division nested flow systolic array according to claim 1, characterized in that: The processing unit PE includes a left input interface Left_in and an upper input interface Top_in, which are respectively used to receive matrix A elements from the left and matrix B elements from the upper side, and respectively pass the matrix A elements from the left and the matrix B elements from the upper side to the floating-point fused multiplication and addition unit FMA for multiplication and addition operations. The processing unit PE includes a unit output interface PE_out, which is used to output the unit final result accumulated by the processing unit PE after completing k matrix multiplications.
3. A multi-stage time-division nested pipeline systolic array accelerator, characterized in that: The invention comprises a control logic module, a matrix synchronization module, an input scheduling module A, an input scheduling module B and the systolic array according to claim 1 or 2, wherein the control logic module is used to receive input data from an external host and parse it to obtain a matrix data stream and transmit it to the matrix synchronization module; the matrix synchronization module is used to receive the asynchronously arriving matrix data stream and perform bit width conversion, and distribute the matrix data stream after bit width conversion to two independent output channels according to a split point dynamically set based on the matrix dimension, and output matrix A data to the input scheduling module A and output matrix B data to the input scheduling module B through the two independent output channels; the input scheduling module A is used to The data of matrix A is delayed according to a data delay mechanism dynamically set based on the matrix dimensions to obtain the elements of matrix A arranged in a diagonal time sequence, and the elements of matrix A are sequentially input into the systolic array from the left side. The input scheduling module B is used to delay the data of matrix B according to the data delay mechanism dynamically set based on the matrix dimensions to obtain the elements of matrix B arranged in a diagonal time sequence, and the elements of matrix B are sequentially input into the systolic array from the top side. The final results of each unit of the systolic array are aggregated into output data for output buffering, and finally fed back to the control logic module, which transmits the output data to the external host.
4. The multi-stage time-division nested pipeline systolic array accelerator according to claim 3, characterized in that: The matrix synchronization module includes a data width converter and a data distribution controller. The data width converter is used to convert the bit width of the asynchronously arriving matrix data stream. The data distribution controller is used to distribute and process the matrix data stream after the bit width conversion according to the split points dynamically set based on the matrix dimensions.
5. The multi-stage time-division nested pipeline systolic array accelerator according to claim 4, characterized in that: The matrix synchronization module further includes two FIFO buffers, which are respectively used to buffer and guide the matrix A data and the matrix B data distributed by the data distribution controller to two independent output channels.
6. The multi-stage time-division nested pipeline systolic array accelerator according to claim 3, characterized in that: The control logic module includes an instruction decoding unit, which is used to perform operation code analysis on input data from an external host to obtain a decoding result.
7. The multi-stage time-division nested pipeline systolic array accelerator according to claim 3, characterized in that: The control logic module includes an address generator, which is used to generate cache addresses of matrix A data, matrix B data and output data according to the decoding result.
8. The multi-stage time-division nested pipeline systolic array accelerator according to claim 3, characterized in that: The control logic module includes a state controller, which is used to control the operating states of the matrix synchronization module, the input scheduling module A, the input scheduling module B and the systolic array.
9. The multi-stage time-division nested pipeline systolic array accelerator according to claim 3, characterized in that: The control logic module includes an enable signal generator, which is used to generate activation signals and activate the matrix synchronization module, input scheduling module A, input scheduling module B and the systolic array respectively through the activation signals.
10. The multi-stage time-division nested pipeline systolic array accelerator according to claim 3, characterized in that: The control logic module includes a counter, which is used to provide a timing reference for controlling the matrix synchronization module, the input scheduling module A, the input scheduling module B, and the systolic array.