Dynamic folding and unfolding method based on Systolic array

Abstract

The invention provides a dynamic folding and unfolding method based on a Systolic array, belongs to the technical field of integrated circuit design and parallel computing, and aims at solving the problems of resource idleness, bandwidth bottleneck and other pain points of a traditional Systolic array matrix multiplication, and provides a 4 * 4 basic array dynamic folding / unfolding framework from four aspects of an array structure, an operation mode and the like. A storage / pulse / half-flow mode is intelligently switched, a three-dimensional resource collaborative scheduling mechanism is established, a QC write-back channel is reserved, resources are utilized to the maximum extent, and the core problem of a traditional scheme is solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of integrated circuit design and parallel computing, and particularly relates to a dynamic folding and unfolding method based on a Systolic array. BACKGROUND

[0002] Matrix multiplication is one of the core operations of intensive computing. With the expansion of array signal size, the traditional single-core system serial operation (time complexity O(n 3 )) cannot meet the performance requirements, and multi-core parallel computing has become the mainstream direction. The Systolic array is widely used in hardware implementation of matrix multiplication due to its high parallel pipeline characteristics, but the existing technology has the following pain points:

[0003] Fixed array structure: The traditional Systolic array mostly adopts a fixed size (such as 4x4, 8x8), and the idle rate of operation resources is high (up to 30%-50%) when operating small-dimension matrices, and the array size cannot be expanded when operating large-dimension matrices, so data needs to be frequently returned, which leads to I / O bandwidth becoming a bottleneck and the operation period being prolonged by 20%-35%.

[0004] Poor mode adaptability: The existing solutions mostly adopt a single operation mode (such as pure flow operation or pure storage operation), and cannot dynamically adjust the data processing mode according to the matrix dimension, for example, using the storage mode for small-dimension matrices will increase the data cache delay, and using the flow mode for large-dimension matrices will lead to low data reuse rate.

[0005] Insufficient resource coordination: In the NoC multi-core system, the matching of the Systolic array with the bandwidth and storage resources lacks a dynamic scheduling mechanism, and when the matrix dimension crosses the critical value (such as 128x128), the contradictions of "bandwidth is idle but operation resources are insufficient" or "operation resources are idle but bandwidth is exhausted" easily occur, and the overall efficiency of the system is reduced. SUMMARY

[0006] To solve the above technical problems, the present application provides a dynamic folding and unfolding method based on a Systolic array.

[0007] The technical solution of the present application is:

[0008] The dynamic folding and unfolding method based on the Systolic array comprises

[0009] 1) Array dimension adaptive adjustment: according to the matrix dimension size, folding or unfolding the basic Systolic array (such as a 4x4 array) to generate a target array (such as 2x8, 3x5, linear array, etc.) that adapts to the current operation requirements, avoiding resource waste and performance bottlenecks.

[0010] 2) Intelligent switching of operation mode: based on the target array structure and I / O bandwidth state, automatically select stream operation, pulsatile operation or storage operation mode, ensure that data transmission and operation rhythm match, improve data multiplexing rate and resource utilization.

[0011] 3) Dynamic reservation of result return channel: for large dimension matrix operation, reserve independent result return channel when the array is folded, alleviate the problem of data return occupying operation bandwidth, and guarantee the continuity of operation.

[0012] Further,

[0013] Definition of basic array and parameterized modeling

[0014] Basic array module: define 4x4 systolic array as the core operation unit, each array node contains reconfigurable computing unit RCU, local RAM cache and NoC network interface, supporting IEEE754 standard 32-bit single-precision complex floating point operation;

[0015] Parameterized model construction: establish the mapping relationship model of matrix dimension n, array size rowxcol and bandwidth channel number Q, as shown in formula (1), where S is the total number of RCU nodes, r is the matrix block size, r must be an integer multiple of the array column number col:

[0016]

[0017] Through this model, the number of rows and columns of the target array can be automatically calculated according to the input matrix dimension n.

[0018] Further,

[0019] Dynamic folding and unfolding logic design

[0020] Unfolding operation: when the matrix dimension is not greater than 128, the 4x4 basic array is unfolded into 2x8 or 3x5 array;

[0021] Folding operation: when the matrix dimension exceeds 128, the basic array is folded into linear array.

[0022] Among them,

[0023] 2x8 array: split the basic array by row to form two groups of 8-column parallel subarrays, matrix A is divided by row and passed through Q A channel, that is, 8-way concurrent transmission, matrix B is divided by column and passed through Q B channel, that is, broadcast mode transmission, using storage operation mode, first cache B matrix block and then operate with A matrix row block;

[0024] 3x5 array: reduce Q A channel number, that is, from 4 rows to 3 rows, increase QB The number of channel columns is increased from 4 to 5, and the number of B matrix data transfer is reduced, which is suitable for 64-128 order matrix.

[0025] Array structure: 16 RCU nodes are connected in a linear chain, and 1 Q A The channel transmits A matrix data in time by row, and 1 Q B The channel transmits B matrix data in time by column, and 1 Q is reserved C The channel is used for result backwriting, and the total bandwidth channel number is reduced to 3;

[0026] Operation mode: semi-flow operation mode is adopted, the first element of A matrix and the first element of B matrix enter RCU0 buffer at t0, the second element of A matrix and the second element of B matrix enter RCU1 at t1, and RCU0 performs multiplication and accumulation operation at the same time, realizing the pipeline of data transmission-operation.

[0027] Further,

[0028] Mode switching and resource coordination control

[0029] Mode decision mechanism: design decision module, select operation mode according to the following conditions:

[0030] If n≤128 and Q≥8, that is, the bandwidth is sufficient: select storage / pulsation operation mode;

[0031] If n>128 or Q<8, that is, the bandwidth is limited: select semi-flow operation mode;

[0032] If the local RAM cache capacity is insufficient: start Q C Backwrite channel, temporarily store part of the results to off-chip DDR.

[0033] NoC cooperative scheduling: realize control through NoC three-layer network, that is, configuration layer, state layer and data layer:

[0034] Configuration layer: issue array folding / unfolding instructions and block parameters r;

[0035] State layer: real-time feedback RCU cache occupancy rate and channel busy / idle state;

[0036] Data layer: adopt "package-circuit connection (PCC)" protocol, use 8-way concurrent transmission when unfolding, and use 3-way time-sharing transmission when folding.

[0037] Further,

[0038] The specific work flow is as follows:

[0039] Input parameter analysis: the system receives the to-be-operated matrix A (m×p) and B (p×n), analyzes the matrix dimension m=n, reads the current NoC bandwidth channel number Q and RCU total number S;

[0040] Array structure calculation: Calculate the row and col of the target array according to formula (1), and determine whether the unfolded n≤128 or the folded n>128:

[0041] If n≤128: choose 2×8 or 3×5 to expand the array;

[0042] If n>128: Choose a linear folded array;

[0043] Mode and Channel Configuration: The decision module selects the operation mode and configures Q. A Q B Q C Channel parameters;

[0044] Matrix partitioning and transmission: Matrix A and B are partitioned into r×r blocks and transmitted to the target array RCU node through the NoC data layer, with the partitioned data cached in the local RAM;

[0045] Parallel computation and result integration: RCU nodes perform multiplication and accumulation operations according to the selected mode, intermediate results are temporarily stored in local RAM, and large-dimensional matrices are integrated through Q... C After the channel is written back to DDR and all block operations are completed, the integrated result yields the final matrix C(m×n).

[0046] The beneficial effects of this invention are

[0047] This invention improves the computational efficiency of matrix multiplication accelerators, adapts to multi-dimensional scenarios, optimizes the utilization of NoC / RCU resources, builds patent barriers for the company, reduces costs, expands scenarios, and increases market share. Attached Figure Description

[0048] Figure 1 This is a schematic diagram of the workflow of the present invention;

[0049] Figure 2 This is a block diagram for 2×8 array computation. Detailed Implementation

[0050] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are some embodiments of the present invention, but not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0051] This invention proposes a dynamic folding and unfolding method based on Systolic arrays, which, combined with the resource scheduling capabilities of NoC multi-core systems, achieves triple dynamic adaptation of "array structure - matrix dimension - operation mode":

[0052] Adaptive array dimension adjustment: Based on the matrix dimension size, the basic Systolic array (such as a 4×4 array) is folded or expanded to generate a target array (such as a 2×8, 3×5, or linear array) that adapts to the current computational requirements, thus avoiding resource waste and performance bottlenecks.

[0053] Intelligent switching of operation mode: Based on the target array structure and I / O bandwidth status, it automatically selects the streaming operation, pulse operation or storage operation mode to ensure that the data transmission and operation rhythm are matched, thereby improving the data reuse rate and resource utilization rate.

[0054] Dynamic reservation of result return channels: For large-dimensional matrix operations, an independent result write-back channel is reserved when the array is folded, which alleviates the problem of data return occupying computing bandwidth and ensures the continuity of operation.

[0055] 1. Basic Array Definition and Parametric Modeling

[0056] Basic array module: Defines a 4×4 Systolic array as the core computing unit. Each array node includes a reconfigurable computing unit (RCU), local RAM cache (2K points / node) and NoC network interface, supporting IEEE754 standard 32-bit single-precision complex floating-point operations (including multiplication, addition, multiplication and accumulation operations).

[0057] Parametric model construction: Establish a mapping relationship model between matrix dimension (n), array size (row×col), and number of bandwidth channels (Q), as shown in formula (1), where S is the total number of RCU nodes (16 in this scheme), and r is the matrix block size (which must be an integer multiple of the array column number col):

[0058]

[0059] This model can automatically calculate the number of rows (row) and columns (col) of the target array based on the dimension n of the input matrix.

[0060] 2. Dynamic folding and unfolding logic design

[0061] Unfolding operation (small-dimensional matrix, n≤128): When the matrix dimension is small (e.g., 32×32, 64×64, 128×128), the 4×4 basic array is unfolded into a 2×8 or 3×5 array.

[0062] 2×8 array: The basic array is split row-wise to form two sets of 8-column parallel subarrays. Matrix A is divided into rows and passed through Q. A Channel (8-way concurrent) transmission, matrix B is divided into columns and transmitted through Q. BChannel (broadcast mode) transmission uses a storage operation mode, first caching the column blocks of matrix B and then performing operations with the row blocks of matrix A, achieving 100% bandwidth utilization and 100% RCU occupancy.

[0063] 3×5 array: Reduce Q A Channel row count (reduced from 4 rows to 3 rows), increase Q B The number of channel columns has been increased from 4 to 5, reducing the number of data transfers in the B matrix. It is suitable for matrices of order 64-128, and the operation cycle is shortened by 5%-8% compared to a 2×8 array.

[0064] Folding operation (large-dimensional matrix, n>128): When the matrix dimension exceeds 128 (e.g., 256×256, 512×512, 1024×1024), the basic array is folded into a linear array.

[0065] Array structure: 16 RCU nodes are connected in series in a linear chain, Q A Channel 1 transmits matrix A data in a row-division time-division manner, Q B Channel 1 transmits B matrix data in a column-based time-division format, reserving Q... C One channel is used for result write-back, reducing the total bandwidth to three channels.

[0066] Operation mode: A semi-stream operation mode is adopted. At time t0, the first element of matrix A and the first element of matrix B enter the RCU0 buffer. At time t1, the second element of matrix A and the second element of matrix B enter the RCU1. At the same time, RCU0 performs multiplication and accumulation operations, realizing the "data transmission-operation" pipeline. The RCU occupancy rate is kept at 100%, and the data reuse rate is increased by 40%-50%.

[0067] 3. Mode switching and resource coordination control

[0068] Pattern Decision Mechanism: Design a decision module to select the operation mode based on the following conditions:

[0069] If n≤128 and Q≥8 (sufficient bandwidth): Select storage / pulse operation mode;

[0070] If n>128 or Q<8 (bandwidth limited): Select half-stream operation mode;

[0071] If the local RAM cache capacity (16 × 2K = 32K points) is insufficient (e.g., when n > 128, a single RCU needs to cache more than 2K of data): Start Q C The write-back channel temporarily stores some results in off-chip DDR to avoid cache overflow.

[0072] NoC Cooperative Scheduling: Control is achieved through a three-layer NoC network (configuration layer, state layer, and data layer).

[0073] Configuration layer: issues array folding / expanding commands and block parameters r;

[0074] State layer: Real-time feedback on RCU buffer utilization and channel busy / idle status;

[0075] Data layer: It adopts the "Packet-Circuit Connection (PCC)" protocol, which uses 8-way concurrent transmission when unfolded and 3-way time-division transmission when folded to ensure data transmission efficiency.

[0076] Taking a NoC multi-core system with 16 RCU nodes as an example, the workflow of this method is as follows:

[0077] Input parameter parsing: The system receives matrices A (m×p) and B (p×n) to be operated on, parses the matrix dimension m=n (this scheme is for square matrices, non-square matrices can be expanded with block logic), and reads the current NoC bandwidth channel number Q and the total number of RCUs S.

[0078] Array structure calculation: Calculate the row and col of the target array according to formula (1), and determine whether to expand (n≤128) or fold (n>128):

[0079] If n = 64: choose a 2×8 array, and the block size r = 8 (an integer multiple of the number of columns in the 2×8 array);

[0080] If n = 256: Select a linear folded array, with a block size r = 16 (to match the data transmission rhythm of the linear array).

[0081] Mode and Channel Configuration: The decision module selects the operation mode (e.g., storage mode for a 64-order matrix, half-stream mode for a 256-order matrix), and configures Q. A Q B Q C Channel parameters (e.g., 8 concurrent or 3 time-sharing).

[0082] Matrix partitioning and transmission: Matrix A and B are partitioned into r×r blocks and transmitted to the target array RCU node through the NoC data layer. The partitioned data is cached in the local RAM.

[0083] Parallel computation and result integration: RCU nodes perform multiplication and accumulation operations according to the selected mode, intermediate results are temporarily stored in local RAM, and large-dimensional matrices are integrated through Q... C After the channel is written back to DDR and all block operations are completed, the integrated result yields the final matrix C(m×n).

[0084] This invention uses the Xilinx Virtex7 series XC7V2000T FPGA chip, with one external DDR3 chip (supporting 8-way 64-bit concurrent channels), and the on-chip RAM is a 32-bit wide, 2K deep block RAM. The RCU node contains two PEs (each PE contains 2 FMUL + 2 FADD).

[0085] NoC adopts a 2D-Mesh topology (6×6 network scale), with three layers of network to realize configuration (34-bit data packets), status (32-bit data packets), and data (64-bit data packets) transmission respectively.

[0086] The system was synthesized and placed using Xilinx Vivado 2018.3; simulated and verified using Mentor Graphics Modelsim SE-64 10.2c; the system automatically generated the array structure and pattern configuration file by sending the matrix dimension n to the host computer, loaded it onto the FPGA via the Ethernet interface, and the calculation results were sent back to the PC for comparison and verification.

[0087] The above description is merely a preferred embodiment of the present invention and is used only to illustrate the technical solution of the present invention, and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention are included within the scope of protection of the present invention.

Claims

1. A dynamic folding and unfolding method based on a Systolic array, characterized in that, include 1) Adaptive adjustment of array dimensions: Based on the size of the matrix dimensions, the basic Systolic array is folded or expanded to generate a target array that adapts to the current computational requirements; 2) Intelligent switching of operation mode: Based on the target array structure and I / O bandwidth status, it automatically selects the streaming operation, pulse operation or storage operation mode to ensure that the data transmission and operation rhythm are matched; 3) Dynamic reservation of result return channels: For large-dimensional matrix operations, an independent result return channel is reserved when the array is folded to alleviate the problem of data return occupying computing bandwidth.

2. The method according to claim 1, characterized in that, Defining the basic array and parametric modeling Basic array module: Defines a 4×4 Systolic array as the core computing unit. Each array node includes a reconfigurable computing unit (RCU), local RAM cache, and NoC network interface, supporting IEEE 754 standard 32-bit single-precision complex floating-point operations. Parametric model construction: Establish a mapping relationship model between matrix dimension n, array size row×col, and bandwidth channel number Q, as shown in formula (1), where S is the total number of RCU nodes and r is the matrix block size: This model can automatically calculate the number of rows (row) and columns (col) of the target array based on the dimension n of the input matrix.

3. The method according to claim 2, characterized in that, r must be an integer multiple of the number of columns col in the array.

4. The method according to claim 3, characterized in that, Dynamic folding and unfolding logic design Unfolding operation: When the matrix dimension is no greater than 128, the 4×4 basic array is unfolded into a 2×8 or 3×5 array; Folding operation: When the matrix dimension exceeds 128, the basic array is folded into a linear array.

5. The method according to claim 4, characterized in that, 2×8 array: The basic array is split row-wise to form two sets of 8-column parallel subarrays. Matrix A is divided into rows and passed through Q. A The channel is an 8-way concurrent transmission, and matrix B is divided into blocks by column and transmitted through Q. B The channel transmits in broadcast mode and uses a storage operation mode, first caching the column blocks of matrix B and then performing operations with the row blocks of matrix A. 3×5 array: Reduce Q A The number of channel rows is reduced from 4 to 3, and Q is increased. B The number of channels is increased from 4 to 5, reducing the number of data transfers in the B matrix and making it suitable for matrices of order 64-128.

6. The method according to claim 5, characterized in that, Array structure: 16 RCU nodes are connected in series in a linear chain, with 1 Q channel. A The channel transmits A matrix data in a row-division time-division manner, with 1 Q channel. B The channel transmits B matrix data column-wise in a time-division manner, with one channel reserved for Q. C The channel is used for result write-back, reducing the total bandwidth and number of channels to 3. Operation mode: A semi-stream operation mode is adopted. At time t0, the first element of matrix A and the first element of matrix B enter the RCU0 buffer. At time t1, the second element of matrix A and the second element of matrix B enter the RCU1. At the same time, RCU0 performs multiplication and accumulation operations to realize the pipeline of data transmission and operation.

7. The method according to claim 6, characterized in that, Mode switching and resource coordination control Pattern Decision Mechanism: Design a decision module to select the operation mode based on the following conditions: If n≤128 and Q≥8, i.e., bandwidth is sufficient: select storage / pulse operation mode; If n>128 or Q<8, i.e., bandwidth is limited: select half-stream operation mode; If the local RAM cache is insufficient: Start Q C Write-back channel temporarily stores some results to off-chip DDR; NoC Cooperative Scheduling: Control is achieved through a three-layer NoC network, namely the configuration layer, state layer, and data layer. Configuration layer: issues array folding / expanding commands and block parameters r; State layer: Real-time feedback on RCU buffer utilization and channel busy / idle status; Data layer: It adopts the "Packet-Circuit Connection (PCC)" protocol, which uses 8-channel concurrent transmission when unfolded and 3-channel time-division transmission when folded.

8. The method according to claim 7, characterized in that, The specific workflow is as follows: Input parameter parsing: The system receives matrices A (m×p) and B (p×n) to be processed, parses the matrix dimension m=n, and reads the current NoC bandwidth channel number Q and the total number of RCUs S; Array structure calculation: Calculate the row and col of the target array according to formula (1), and determine whether the unfolded n≤128 or the folded n>128: If n≤128: choose 2×8 or 3×5 to expand the array; If n>128: Choose a linear folded array; Mode and Channel Configuration: The decision module selects the operation mode and configures Q. A Q B Q C Channel parameters; Matrix partitioning and transmission: Matrix A and B are partitioned into r×r blocks and transmitted to the target array RCU node through the NoC data layer, with the partitioned data cached in the local RAM; Parallel computation and result integration: RCU nodes perform multiplication and accumulation operations according to the selected mode, intermediate results are temporarily stored in local RAM, and large-dimensional matrices are integrated through Q... C After the channel is written back to DDR and all block operations are completed, the integrated result yields the final matrix C(m×n).

Images