Implementation method, device and medium of a general fixed-point matrix multiplier based on FPGA high-performance computing architecture

Abstract

This invention discloses a method, apparatus, and medium for implementing a general-purpose fixed-point matrix multiplier based on a high-performance computing architecture in an FPGA. The method includes: designing matrix partitioning strategies at different levels based on the parallelism of the AI ​​engine array resources on the Versal ACAP platform; data scheduling and multiplexing based on the data packet stream and data packet exchange of the AI ​​engine and AXI stream, accelerating the kernel through matrix partitioned multiplication, and designing data scheduling and multiplexing strategies; and implementing a high-throughput vectorized matrix multiplication pipeline on the AI ​​engine vector processor. This invention achieves multi-level partitioning of matrix multiplication based on the AXI stream transport protocol and AI engine array on the Versal ACAP platform, enabling efficient utilization of hardware resources, effectively improving data reuse rate, and achieving high parallelism, while achieving high computational speed under the high-speed clock of the AI ​​engine. This invention can be widely applied in the field of high-performance computing.

Description

Technical Field

[0001] This invention relates to the field of high-performance computing, and in particular to a method, apparatus and medium for implementing a general fixed-point matrix multiplier based on a high-performance computing architecture in FPGA. Background Technology

[0002] The Transformer model relies on a unique architecture that utilizes self-attention, enabling it to capture long-range dependencies in text. This allows the model to focus on information at different locations when processing input sequences, inspired by human visual attention mechanisms. By focusing on salient objects in a scene, it makes efficient use of limited visual information. Self-attention also allows for parallel training, fully leveraging the global information in the training data without being limited by sequential structure.

[0003] However, the computational complexity of large matrix multiplication introduced by the self-attention mechanism is very high. If the matrix dimension is large, a large number of multiplication and addition operations will lead to long computation times. Furthermore, large matrix multiplication requires storing a large number of intermediate results. During computation, sufficient memory needs to be allocated to store intermediate matrices, which can easily lead to memory shortages. Secondly, data transmission is also challenging due to the limited number of high-speed transmission channels in the hardware. The propagation delay of data between computing units can become a performance bottleneck. Moreover, the parallelism of large matrix multiplication is also limited by hardware computing resources; effectively partitioning matrices and managing parallel computing tasks is a challenge in practical applications.

[0004] Although there has been considerable research on accelerating matrix multiplication in hardware, it has largely remained at the academic research stage. Most of these efforts are hampered by low performance or impracticality, making them difficult to implement and deploy on high-performance hardware. Furthermore, the solutions employed in these studies are mostly based on a single hardware component (FPGA), or a combination of SoC and FPGA. While SoC is suitable only for sending commands, performing state transitions, and scheduling data, FPGA, despite its wide parallel processing capabilities and ability to process large amounts of data simultaneously, suffers from lower clock frequencies and consumes significant hardware resources for matrix multiplication, making it less than optimal for accelerating fixed-point matrix multiplication. Summary of the Invention

[0005] In order to at least partially solve one of the technical problems existing in the prior art, the present invention aims to provide a method, apparatus and medium for implementing a general fixed-point matrix multiplier based on a high-performance computing architecture in FPGA.

[0006] The first technical solution adopted in this invention is:

[0007] An implementation method for a general fixed-point matrix multiplier based on a high-performance computing architecture in FPGA includes the following steps:

[0008] Based on the parallelism of the AI ​​engine array resources of the Versal ACAP platform, matrix partitioning strategies at different levels are designed.

[0009] Data scheduling and multiplexing of data packets and data packet exchange based on AI engine and AXI stream, accelerating the kernel through matrix block multiplication, and designing data scheduling and multiplexing strategies;

[0010] A pipeline for high-throughput vectorized matrix multiplication operations implemented on the AI ​​engine vector processor.

[0011] Furthermore, the Versal ACAP platform comprises five scales and four levels of blocks, including off-chip (memory), on-chip (programmable logic), AI engine array, AI engine kernel, and vector processor, wherein the off-chip cached data is the complete input matrix that is not yet segmented.

[0012] Furthermore, the multi-level matrix partitioning strategy based on the Versal architecture includes:

[0013] Based on the hardware resource design, matrix operation algorithms and corresponding data scheduling strategies are designed for all levels of block partitioning except for the highest level of block partitioning outside of chips.

[0014] Data from the PS end is synchronized and stored in an off-chip buffer. The data scheduling unit performs the first outermost block division on this data.

[0015] Different blocks are transmitted from off-chip to on-chip, further divided into lower-level blocks according to the local buffer capacity of the AI ​​engine array, and then transmitted to the AI ​​engine array in the form of an AXI stream according to the blocks.

[0016] The AI ​​engine array organizes data into windows using packet streams and packet exchange, and the underlying AI engine controls the movement of these windows and performs calculations.

[0017] The final layer uses a special vector processor from an AI engine to perform high-throughput multiplication operations, and aggregates the calculation results from the lowest level in a preset format and sends them back to the PS.

[0018] Furthermore, to leverage the parallelism of the AI ​​engine array resources based on the Versal ACAP platform, different levels of matrix partitioning strategies are designed, including:

[0019] The large matrix is ​​divided into smaller sub-blocks by matrix partitioning, and the product of these sub-blocks is calculated separately. By performing different levels of block processing and data reuse on the matrix, dedicated hardware resources are allocated to each sub-block. By utilizing the parallelism of FPGA and AI engine, multiple processing units are designed to calculate different sub-blocks simultaneously, so that the product of each sub-block can be calculated independently. Finally, the results are accumulated to form the final product matrix.

[0020] Off-chip buffer timing multiplexing: the address generation unit generates addresses corresponding to the buffer according to the time order of different blocks and the time order of iteration within the blocks, and transmits them to the data reading unit.

[0021] On-chip buffer timing multiplexing: The PL terminal also serves to divide the entire AI engine array into blocks. Different blocks are passed into the AI ​​engine array in different time slots to achieve on-chip multiplexing.

[0022] AI engine array spatial reuse caches data from the on-chip buffer into different local cache units according to the needs of a single AI engine kernel. Data transmission to the AI ​​engine array is processed in a windowed manner. Data from the AXI stream is cached in local caches around a single AI engine kernel according to the window size. The AI ​​engine kernel obtains new data needed for computation by moving the window.

[0023] The AI ​​engine kernel uses time-series multiplexing, introducing a final layer of matrix blocks into the matrix multiplication operation at the lowest level of the kernel. The vector processor reads these blocks sequentially, performs multiplication operations, and accumulates the calculation results to pass them to higher levels through the AXI stream interface.

[0024] Furthermore, the data packet stream is designed to segment data into continuous data packets, each consisting of a header and multiple data words. These packets are routed and exchanged between the AI ​​engine and the programmable logic via an AXI stream. The data packet exchange is responsible for routing and exchanging data packets between the AI ​​engine and the programmable logic. The AI ​​engine kernel retrieves the corresponding input data according to the block it is responsible for computing, thus satisfying the needs of multiple AI engine kernels for the same block of data, as well as the needs of a single AI engine kernel for multiple blocks of data, thereby realizing the transmission and multiplexing of multiple data streams.

[0025] Furthermore, the data scheduling and multiplexing of the data packet stream and data packet exchange based on the AI ​​engine and AXI stream, accelerated by matrix block multiplication, and the data scheduling and multiplexing strategy are designed, including:

[0026] The matrix is ​​divided into smaller blocks by using a matrix partitioning strategy, and these sub-blocks are stored in the corresponding storage units of the current layer.

[0027] Based on the computational needs of the AI ​​engine, the AXI stream begins to work, reading data from the storage unit and transmitting the data to the AI ​​engine through the AXI interface;

[0028] During transmission, AXI streams distribute data to different AI engine kernels according to their needs; at the same time, AXI streams are responsible for coordinating data transmission between multiple kernels, which improves both data reuse and real-time performance.

[0029] After the calculation is completed, the AXI stream then transmits the result data from the AI ​​engine to the next processing unit or outputs it to the buffer.

[0030] Furthermore, the pipeline for implementing high-throughput vectorized matrix multiplication operations on the AI ​​engine vector processor includes:

[0031] The input matrix data stream is preprocessed by using the local storage buffer of the AI ​​engine and decomposing the matrix into smaller data blocks through a window function to adapt to the vector size and parallel capabilities of the processing unit.

[0032] Perform core multiplication operations, utilize the VLIW architecture to issue multiplication instructions in parallel, and execute compound instructions to enhance instruction pipeline execution capabilities;

[0033] By leveraging the parallelism of the core array, multi-core synchronous multiplication operations are achieved, optimizing data flow and control logic, and reducing transmission latency;

[0034] The processing results are post-processed and output, aggregated in local storage, and then transformed and output using window functions.

[0035] Furthermore, the matrix multiplication pipeline includes preprocessing of input data, core multiplication operations, computational acceleration processing, and post-processing and output of the final result, as detailed below:

[0036] Input data preprocessing: The AI ​​engine has a large amount of local storage as a buffer to store the input matrix data stream. Window functions are used to read the data stream and decompose the matrix into smaller data blocks. Sliding window parameters are set to update the data in the cache window. Data flows from the AI ​​engine array to the AI ​​engine kernel and is re-aligned and formatted to match the processing needs of the AI ​​engine.

[0037] Core multiplication operations: Instruction scheduling is performed, and multiple multiplication instructions are issued in parallel within the VLIW architecture of the AI ​​engine to make full use of each processing cycle; parallel execution is implemented to execute compound instructions containing multiple multiplication operations to enhance the instruction pipeline processing capability of each core and improve computational density.

[0038] Accelerated processing: In the AI ​​engine array, the execution of multiplication operations is synchronized across multiple cores, enabling parallel processing through the core array; data flow and control logic are optimized through different jumps and incremental window management, reducing data transmission latency; in addition, by iteratively processing the upper-level blocks, data is efficiently transmitted between AI engines by effectively utilizing the AXI stream interface and data packet management.

[0039] Final result post-processing and output: The final results are aggregated in local storage, and the results in the accumulator are split and output through window functions, and converted into a format suitable for output requirements; by utilizing the DMA controller and AXI stream of the AI ​​engine, the results are transferred to the next processing stage or memory.

[0040] The second technical solution adopted in this invention is:

[0041] An implementation device for a general-purpose fixed-point matrix multiplier based on a high-performance computing architecture in FPGA, comprising:

[0042] At least one processor;

[0043] At least one memory for storing at least one program;

[0044] When the at least one program is executed by the at least one processor, the at least one processor implements the method described above.

[0045] The third technical solution adopted in this invention is:

[0046] A computer-readable storage medium storing a processor-executable program, which, when executed by a processor, performs the method described above.

[0047] The beneficial effects of this invention are: Based on the AXI streaming protocol and AI engine array of the Versal ACAP platform, this invention realizes multi-level block division of matrix multiplication, which enables matrix multiplication to efficiently utilize hardware resources, effectively improve data reuse rate and achieve high parallelism, and achieve high computing speed under the high-speed clock of the AI ​​engine. Attached Figure Description

[0048] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following description is provided with accompanying drawings of the relevant technical solutions in the embodiments of the present invention or the prior art. It should be understood that the accompanying drawings described below are only for the purpose of clearly illustrating some embodiments of the technical solutions of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0049] Figure 1This is a schematic diagram of a multi-level matrix partitioning strategy based on the Versal architecture in an embodiment of the present invention;

[0050] Figure 2 This is a schematic diagram of fixed-point block matrix multiplication within the AI ​​engine kernel in an embodiment of the present invention;

[0051] Figure 3 This is a flowchart illustrating the steps of an implementation method for a general fixed-point matrix multiplier based on a high-performance computing architecture in an FPGA, as described in this invention. Detailed Implementation

[0052] The embodiments of the present invention are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention. The step numbers in the following embodiments are set only for ease of explanation, and there is no limitation on the order between the steps. The execution order of each step in the embodiments can be adaptively adjusted according to the understanding of those skilled in the art.

[0053] In the description of this invention, it should be understood that the orientation descriptions, such as up, down, front, back, left, right, etc., are based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limiting this invention.

[0054] In the description of this invention, "several" means one or more, "multiple" means two or more, "greater than," "less than," and "exceeding" are understood to exclude the stated number, while "above," "below," and "within" are understood to include the stated number. If "first" or "second" is used, it is only for distinguishing technical features and should not be construed as indicating or implying relative importance, or implicitly indicating the number of indicated technical features, or implicitly indicating the order of the indicated technical features. Furthermore, "and / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0055] In the description of this invention, unless otherwise explicitly defined, terms such as "set up," "install," and "connect" should be interpreted broadly, and those skilled in the art can reasonably determine the specific meaning of the above terms in this invention in conjunction with the specific content of the technical solution.

[0056] Terminology Explanation:

[0057] PS: Processing System, usually refers to the processing system side, especially in systems using the ARM core.

[0058] PL: Programmable Logic, usually related to the FPGA (Field Programmable Gate Array) core.

[0059] To address the existing technical problems, this invention proposes a method for linear fixed-point matrix operations based on an AI engine architecture processor. Xilinx's Versal ACAP platform computing accelerator card introduces this new AI engine architecture. The VCK190, belonging to this platform, includes an AI engine array consisting of 400 AI engine processors specialized for vector operations. This allows matrix operations to be divided into several vector multiplication and accumulation operations to achieve a high-performance fixed-point matrix multiplier.

[0060] like Figure 3 As shown, this embodiment provides a method for implementing a general-purpose fixed-point matrix multiplier based on a high-performance computing architecture in an FPGA, including the following steps:

[0061] A1: Based on the parallelism of AI engine array resources on the Versal ACAP platform, design matrix partitioning strategies at different levels.

[0062] First, there is the multi-level matrix partitioning strategy, such as... Figure 1 As shown, the system comprises five levels of blocks: off-chip (memory), on-chip (programmable logic), AI engine array, AI engine kernel, and vector processor. The off-chip cache contains the complete, unblocked input matrix. Specifically, step 1 includes the following steps:

[0063] A11: Based on hardware resources, design the block matrix operation algorithms and corresponding data scheduling strategies for all levels except the highest-level block segmentation (outside of the chip). During the top-level design, determine the total block size and the number of iterations for each level, then input them to the accelerator via memory mapping on the PS side. Utilize the characteristics of hardware platform buffer objects to abstract the PL-side memory, performing memory allocation and access without directly handling hardware details. Create buffers through buffer objects to synchronize data between host memory and PL-side memory. The memory mapping feature maps PL-side memory regions to the host address space, ensuring a one-to-one correspondence between host memory and PL-side memory addresses, allowing the PL-side data transmission control unit to process buffer data according to the same data structure.

[0064] A12: Data from the PS end is synchronized and stored in an off-chip buffer. The data scheduling unit will perform the first outermost block division on this data. The block size depends on the size that the PL end hardware resources can buffer at one time.

[0065] A13: Different blocks are transmitted from off-chip to on-chip in a time-division manner, and further divided into lower-level blocks according to the local buffer capacity of the AI ​​engine array, and then passed into the AI ​​engine array in the form of AXI streams according to the blocks.

[0066] A14: The AI ​​engine array organizes data into windows using packet streams and packet exchange. The underlying AI engine controls the movement of these windows and performs calculations.

[0067] A15: At the final layer, a special vector processor using the AI ​​engine performs high-throughput multiplication operations, including data input preprocessing, core multiplication operations, computational acceleration processing, and final result output. Finally, the calculation results at the lowest level are aggregated in the same format and ultimately returned to the PS (Power Supply) end.

[0068] A2. Data scheduling and multiplexing of data packets and data packet exchange based on AI engine and AXI stream, accelerating the kernel through matrix block multiplication, and designing data scheduling and multiplexing strategies.

[0069] Secondly, a data scheduling and multiplexing strategy based on the AI ​​engine and AXI streams is employed. Matrix block multiplication accelerates the kernel, achieving efficient data processing. The data packet stream design divides data into continuous data packet streams, each containing a header and multiple data words. These are routed and exchanged between the AI ​​engine and programmable logic via AXI streams. The AI ​​engine retrieves corresponding data blocks on demand, satisfying the needs of multiple engines for the same data or a single engine for multiple data blocks, thus achieving multi-data stream transmission and multiplexing. AXI streams coordinate data transmission between kernels, ensuring data multiplexing and real-time performance, and outputting the completed calculations to the next unit or buffer.

[0070] A3. A pipeline for high-throughput vectorized matrix multiplication operations implemented on the AI ​​engine vector processor.

[0071] Finally, a high-throughput vectorized matrix multiplication pipeline is implemented on the AI ​​engine's vector processor. First, the input matrix data stream is preprocessed. Utilizing the AI ​​engine's local storage buffer, a window function decomposes the matrix into smaller data blocks to accommodate the vector size and parallelism of the processing unit. Second, core multiplication operations are performed, using the VLIW architecture to issue multiplication instructions in parallel and execute compound instructions to enhance the instruction pipeline's execution capability. Then, the parallelism of the core array is leveraged to enable synchronous multiplication operations across multiple cores, optimizing data flow and control logic and reducing transmission latency. Finally, the processing results are post-processed and output. The results are aggregated in local storage, transformed using a window function (including format conversion, shifting, rounding, and saturation operations), and efficiently transferred to the next stage or memory using a DMA controller and AXI streams. This process ensures high throughput and data reuse within the kernel.

[0072] The above method will be explained in detail below with reference to the accompanying drawings and specific embodiments.

[0073] This embodiment provides a method for implementing a general fixed-point matrix multiplier based on a high-performance computing architecture in FPGA, including the following steps:

[0074] S1: Based on the parallelism of the AI ​​engine array resources on the Versal ACAP platform, a matrix partitioning strategy with different levels is designed. Data scheduling and data reuse technologies are introduced to balance data access latency, computation latency, and the rational consumption of hardware resources such as memory and programmable logic. Data scheduling refers to the process of controlling the transmission order and timing of matrix blocks, while data reuse refers to the simultaneous transmission of multiple data streams on the same physical channel. By designing AXI streams, coordinated transmission between different data streams required by different computing units is ensured. This includes allocating the same input data stream to different kernels, aggregating multiple data streams into a single kernel, and converging multiple output data streams into a single data stream. This improves the resource utilization efficiency and data reuse rate of AXI streams, and reduces the latency caused by multiple data transmissions.

[0075] In some embodiments, step S1 includes the following steps:

[0076] S1-1: First, the large matrix is ​​divided into smaller sub-blocks through matrix partitioning, and then the product of these sub-blocks is calculated separately. By performing different levels of block processing and data multiplexing on the matrix, dedicated hardware resources are allocated to each sub-block. Utilizing the parallelism of FPGA and AI engine, multiple processing units are designed to calculate different sub-blocks simultaneously. This allows the product of each sub-block to be calculated independently, and the results are finally accumulated to form the final product matrix. This method is particularly suitable for large-scale matrix multiplication because it can significantly reduce computation time.

[0077] S1-2: Off-chip buffer timing multiplexing. The address generation unit generates 32-bit addresses corresponding to the buffers according to the time order of different blocks and the time order of iterations within each block, and transmits them to the data read unit. Each address corresponds to 512 bits of data on the off-chip buffer, which is also the maximum number of bits that the AXI bus can transmit at one time. Each time the data read unit receives an address, it reads 512 bits from the off-chip buffer and transmits them to the on-chip matrix block unit.

[0078] S1-3: On-chip buffer timing multiplexing. The PL (Programmer) also serves to divide the entire AI engine array into blocks. Different blocks are transmitted to the AI ​​engine array in different time slots, achieving on-chip multiplexing. The on-chip matrix block unit further divides the data read from off-chip into blocks and stores them in the on-chip buffer. At the same time, different dimensions of the on-chip buffer are expanded or packaged into blocks to improve parallel read and write efficiency. Finally, according to the time sequence and the size that the AI ​​engine array can process at one time, the block data is transmitted to the AI ​​engine array via AXI stream.

[0079] S1-4: AI Engine Array Spatial Multiplexing. Through programming, a unique computation graph for the AI ​​engine array is designed. Data transmitted from the on-chip buffer is cached in portions required by individual AI engine cores onto different local cache units. The designed computation graph contains several AI engine cores, with a local group of AI engine cores processing a block within its layer. Furthermore, data transmission to the AI ​​engine array employs windowed processing. Data transmitted from the AXI stream is cached according to the window size in local caches surrounding individual AI engine cores. The AI ​​engine cores acquire new data needed for computation by moving the window.

[0080] S1-5: The AI ​​engine kernel uses time-series multiplexing. Even after multi-level block partitioning, the matrix size is still very large. To address this, a final level of matrix partitioning is introduced at the kernel's lowest-level matrix multiplication operation. The size of each block is determined by a single vector multiplication that the AI ​​engine kernel can execute in a single operation. The vector processor sequentially reads these blocks, performs multiplication operations, and accumulates the results before passing them to higher levels via the AXI stream interface.

[0081] S2: Data scheduling and multiplexing based on AI engine and AXI stream packet stream and packet switching. Considering the need for data multiplexing in matrix block multiplication to accelerate kernel processing, an efficient data scheduling and multiplexing strategy was designed and implemented. This protocol segments data into continuous packets, each consisting of multiple data words. Packet switching is responsible for routing and exchanging packets between the AI ​​engine and programmable logic. The AI ​​engine kernel retrieves the corresponding input data based on the block it is responsible for computing, satisfying the needs of multiple AI engine kernels for the same data block, as well as the need of a single AI engine kernel for multiple data blocks, thus achieving the transmission and multiplexing of multiple data streams.

[0082] First, the matrix passed from the previous layer is divided into smaller blocks using a matrix partitioning strategy, and these sub-blocks are stored in the corresponding storage units within this layer. Then, based on the computational needs of the AI ​​engine, the AXI stream begins operation, reading data from the storage units and transmitting it to the AI ​​engine via the AXI interface. During transmission, the AXI stream allocates data to different AI engine kernels according to their requirements. Simultaneously, the AXI stream coordinates data transmission between multiple kernels, ensuring both data reuse and real-time performance. After computation, the AXI stream then transmits the result data from the AI ​​engine to the next computational unit or outputs it to a buffer.

[0083] S3: Utilizes an AI engine to achieve high-throughput multiplication operations, such as... Figure 2 As shown, it includes data input preprocessing, core multiplication operations, computation acceleration processing, and final result post-processing and output.

[0084] In some embodiments, step S3 includes the following steps:

[0085] S3-1: Preprocessing of the input matrix data stream. The AI ​​engine has a large amount of local storage as a buffer to store the input matrix data stream. In particular, when using window functions to read the data stream, the matrix needs to be decomposed into smaller data blocks. This decomposition takes into account the vector size and parallel processing capabilities of the AI ​​engine's vector processing units. One processing unit can process data of a limited width simultaneously, so the matrix will be vectorized into multiple smaller blocks. At the same time, sliding window parameters are set to update the data in the cache window. Data flows from the AI ​​engine array to the AI ​​engine kernel, and the data undergoes necessary realignment and format conversion to match the processing requirements of the AI ​​engine.

[0086] S3-2: Core multiplication operations. Performs instruction scheduling, issuing multiple multiplication instructions in parallel within the AI ​​engine's VLIW architecture to fully utilize each processing cycle. Implements parallel execution, executing compound instructions containing multiple multiplication operations to enhance the instruction pipeline processing capability of each core and increase computational density.

[0087] S3-3: Accelerated Processing. In the AI ​​engine array, multi-core synchronized multiplication operations are executed, achieving accelerated processing through parallel operation of the core array. Data flow and control logic are optimized through different jumps and incremental window management, reducing data transmission latency. Furthermore, by iteratively processing the upper-level blocks, data is efficiently transmitted between AI engines using the AXI stream interface and packet management.

[0088] S3-4: Final Result Post-processing and Output. The final processed results are aggregated in local storage, and the results in the accumulator are split and output using window functions, transforming them into a format suitable for output requirements. This process includes converting the accumulator results stored in high-order bits into a fixed-point format for the desired output, including shifting, rounding, and saturation operations, and adjusting the size and arrangement of the output matrix. The results are efficiently transferred to the next processing stage or memory by utilizing the AI ​​engine's DMA controller and AXI streams.

[0089] In summary, traditional FPGA gated logic circuits have slow clock speeds. Parallel processing of large amounts of data with the same operations results in reprogramming significant hardware resources. Furthermore, time multiplexing introduces latency while ensuring the implementation of other processing modules. In contrast, the Versal ACAP platform's AI engine resource is a highly optimized processing unit designed specifically for machine learning and advanced signal processing applications. The VLIW processor within the AI ​​engine can issue up to six instructions per clock cycle, including scalar operations, load / store operations, and fixed-point or floating-point vector operations, achieving higher computational density in a high-clock-cycle AI engine processor.

[0090] Existing matrix multipliers typically perform positional operations on large-scale matrix multiplications directly or use systolic arrays to pipeline matrix multiplication. Direct operations result in low data reuse, high hardware resource consumption, highly serialized operations, high computational latency, and are difficult to deploy on mobile platforms. While systolic array-based matrix multiplication offers high data reuse and relatively low computational latency, it does not fully utilize the parallel capabilities of FPGAs. This invention employs an AI engine array on the Versal ACAP platform to implement block-wise matrix multiplication, conserving parallel FPGA hardware resources while efficiently performing vectorized matrix multiplication operations using an arrayed high-speed dedicated AI engine processor.

[0091] Matrix multiplication involves large amounts of data and high computational complexity. Existing technologies mostly rely on on-chip networks or DMA controllers in FPGAs for communication, and the designed pipelines are difficult to effectively perform efficient communication or data multiplexing. The VersalACAP platform uses AXI streams as a high-speed communication protocol within or between PL and AI engine arrays. The data scheduling strategy designed based on this protocol in this invention can efficiently and orderly manage the large amount of data streams generated during matrix block multiplication.

[0092] Compared with the prior art, the present invention has at least the following advantages and beneficial effects:

[0093] (1) This invention utilizes the AI ​​engine resources on the selected Versal ACAP platform VCK190 computing card to design a multi-level matrix partitioning strategy for fixed-point matrix multiplication and a corresponding data scheduling and multiplexing pipeline. This enables parallel and high-speed large-scale matrix multiplication calculations, significantly accelerating inference for models with dense matrix multiplication. Based on the Versal ACAP platform's AXI streaming protocol and AI engine array, this solution implements multi-level partitioning of matrix multiplication, allowing for efficient use of hardware resources, improved data reuse, and high parallelism. High computational speed is achieved under the high-speed clock of the AI ​​engine.

[0094] (2) This invention effectively utilizes the AXI stream of the Versal ACAP platform to transmit the input and output data of the block matrix between the AI ​​engine and the programmable logic, coordinating the data needs of different computing units. By optimizing the usage strategy of the AXI stream, the data reuse rate and resource utilization efficiency are further improved. By designing reasonable packet sizes and data scheduling strategies, the latency of data transmission is reduced, and by reusing the same input data stream and aggregating multiple output data streams, the utilization efficiency of AXI stream resources is improved. These optimization measures give this invention the advantages of low latency and high throughput in data transmission.

[0095] (3) This invention leverages the advantages of the Versal ACAP platform's AI engine resources. The combination of SIMD and VLIW technologies in the AI ​​engine provides powerful computing capabilities and efficient data processing performance. SIMD is a technology that allows a single operation instruction to operate on multiple data simultaneously. In the AI ​​engine, the SIMD vector processor can perform the same operation on a set of data within each clock cycle, thereby achieving data-level parallel processing. This is particularly important for vector and matrix operations in machine learning applications, as they typically involve a large number of data points and require fast and repetitive computation. VLIW technology, on the other hand, allows multiple independent instructions to be executed within a single clock cycle. This instruction-level parallelism enables the AI ​​engine to efficiently process vectorized matrix multiplication operations. Therefore, fixed-point block matrix multiplication operations based on the AI ​​engine architecture can be compressed into a few clock cycles to achieve low latency and high throughput.

[0096] This embodiment also provides an implementation device for a general fixed-point matrix multiplier based on a high-performance computing architecture in an FPGA, including:

[0097] At least one processor;

[0098] At least one memory for storing at least one program;

[0099] When the at least one program is executed by the at least one processor, the at least one processor implements Figure 3 The method shown.

[0100] This embodiment provides an implementation device for a general fixed-point matrix multiplier based on a high-performance computing architecture in FPGA. It can execute the implementation method of a general fixed-point matrix multiplier based on a high-performance computing architecture in FPGA provided in the method embodiment of the present invention. It can execute any combination of implementation steps of the method embodiment and has the corresponding functions and beneficial effects of the method.

[0101] This application also discloses a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device can read the computer instructions from the computer-readable storage medium and execute the computer instructions, causing the computer device to perform... Figure 3 The method shown.

[0102] This embodiment also provides a storage medium storing instructions or programs that can execute the implementation method of a general fixed-point matrix multiplier based on a high-performance computing architecture in FPGA provided in the method embodiment of the present invention. When the instructions or programs are run, any combination of implementation steps of the method embodiment can be executed, and the method has the corresponding functions and beneficial effects.

[0103] In some alternative embodiments, the functions / operations mentioned in the block diagrams may not occur in the order shown in the operation diagrams. For example, depending on the functions / operations involved, two consecutively shown blocks may actually be executed substantially simultaneously, or the blocks may sometimes be executed in reverse order. Furthermore, the embodiments presented and described in the flowcharts of this invention are provided by way of example to provide a more comprehensive understanding of the technology. The disclosed methods are not limited to the operations and logic flows presented herein. Alternative embodiments are contemplated in which the order of various operations is altered and sub-operations described as part of a larger operation are executed independently.

[0104] Furthermore, although the invention has been described in the context of functional modules, it should be understood that, unless otherwise stated, one or more of the described functions and / or features may be integrated into a single physical device and / or software module, or one or more functions and / or features may be implemented in a separate physical device or software module. It is also understood that a detailed discussion of the actual implementation of each module is unnecessary for understanding the invention. Rather, given the properties, functions, and internal relationships of the various functional modules in the apparatus disclosed herein, the actual implementation of the module will be understood within the scope of conventional skill of an engineer. Therefore, those skilled in the art can implement the invention as set forth in the claims using ordinary techniques without excessive experimentation. It is also understood that the specific concepts disclosed are merely illustrative and not intended to limit the scope of the invention, which is determined by the full scope of the appended claims and their equivalents.

[0105] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, essentially, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0106] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0107] More specific examples of computer-readable media (a non-exhaustive list) include: electrical connections (electronic devices) having one or more wires, portable computer disk drives (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0108] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0109] In the foregoing description of this specification, references to terms such as "one embodiment," "another embodiment," or "some embodiments" indicate that a specific feature, structure, material, or characteristic described in connection with an embodiment or example is included in at least one embodiment or example of the present invention. In this specification, illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0110] Although embodiments of the invention have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.

[0111] The above is a detailed description of the preferred embodiments of the present invention. However, the present invention is not limited to the above embodiments. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the present invention. All such equivalent modifications or substitutions are included within the scope defined by the claims of this application.

Claims

1. A method for implementing a general-purpose fixed-point matrix multiplier based on a high-performance computing architecture in FPGA, characterized in that, Includes the following steps: Based on the parallelism of the AI ​​engine array resources of the Versal ACAP platform, matrix partitioning strategies at different levels are designed. Data scheduling and multiplexing of data packets and data packet exchange based on AI engine and AXI stream, accelerating the kernel through matrix block multiplication, and designing data scheduling and multiplexing strategies; A pipeline for high-throughput vectorized matrix multiplication operations is implemented on the vector processor of an AI engine; The Versal ACAP platform comprises five scales and four levels of blocks, including off-chip, on-chip, AI engine array, AI engine kernel and vector processor. The off-chip cached data is the complete input matrix that is not yet divided into blocks. A multi-level matrix partitioning strategy based on the Versal architecture includes: Based on the hardware resources, matrix operation algorithms and corresponding data scheduling strategies are designed for each level of block partitioning other than chip partitioning. Data from the PS end is synchronized and stored in an off-chip buffer, and this data is then divided into the outermost first block. Different blocks are transmitted from off-chip to on-chip, further divided into lower-level blocks according to the local buffer capacity of the AI ​​engine array, and then transmitted to the AI ​​engine array in the form of an AXI stream according to the blocks. The AI ​​engine array organizes data into windows using packet streams and packet exchange, and the underlying AI engine controls the movement of these windows and performs calculations. The final layer uses an AI engine vector processor to perform high-throughput multiplication operations, and aggregates the calculation results from the lowest level in a preset format and sends them back to the PS.

2. The implementation method of a general fixed-point matrix multiplier based on a high-performance computing architecture in FPGA according to claim 1, characterized in that, To ensure the parallelism of the AI ​​engine array resources based on the Versal ACAP platform, different levels of matrix partitioning strategies are designed, including: The large matrix is ​​divided into smaller sub-blocks by matrix partitioning, and the product of these sub-blocks is calculated separately. By performing different levels of block processing and data reuse on the matrix, dedicated hardware resources are allocated to each sub-block. By utilizing the parallelism of FPGA and AI engine, multiple processing units are designed to calculate different sub-blocks simultaneously, so that the product of each sub-block can be calculated independently. Finally, the results are accumulated to form the final product matrix. Off-chip buffer timing multiplexing: the address generation unit generates addresses corresponding to the buffer according to the time order of different blocks and the time order of iteration within the blocks, and transmits them to the data reading unit. On-chip buffer timing multiplexing: The PL terminal also serves to divide the entire AI engine array into blocks. Different blocks are passed into the AI ​​engine array in different time slots to achieve on-chip multiplexing. AI engine array spatial reuse caches data from the on-chip buffer into different local cache units according to the needs of a single AI engine kernel. Data transmission to the AI ​​engine array is processed in a windowed manner. Data from the AXI stream is cached in local caches around a single AI engine kernel according to the window size. The AI ​​engine kernel obtains new data needed for computation by moving the window. The AI ​​engine kernel uses time-series multiplexing, introducing a final layer of matrix blocks into the matrix multiplication operation at the lowest level of the kernel. The vector processor reads these blocks sequentially, performs multiplication operations, and accumulates the calculation results to pass them to higher levels through the AXI stream interface.

3. The implementation method of a general fixed-point matrix multiplier based on a high-performance computing architecture in FPGA according to claim 1, characterized in that, The data packet stream is designed to segment data into continuous data packets, each consisting of a header and multiple data words. These packets are routed and exchanged between the AI ​​engine and the programmable logic via an AXI stream. The data packet exchange is responsible for routing and exchanging data packets between the AI ​​engine and the programmable logic. The AI ​​engine kernel retrieves the corresponding input data according to the block it is responsible for computing, thus satisfying the needs of multiple AI engine kernels for the same block of data, as well as the needs of a single AI engine kernel for multiple blocks of data, thereby realizing the transmission and multiplexing of multiple data streams.

4. The implementation method of a general fixed-point matrix multiplier based on a high-performance computing architecture in FPGA according to claim 1, characterized in that, The data scheduling and multiplexing of the data packet stream and data packet exchange based on the AI ​​engine and AXI stream are accelerated by matrix block multiplication. The data scheduling and multiplexing strategy is designed, including: The matrix is ​​divided into smaller blocks by using a matrix partitioning strategy, and these sub-blocks are stored in the corresponding storage units of the current layer. Based on the computational needs of the AI ​​engine, the AXI stream begins to work, reading data from the storage unit and transmitting the data to the AI ​​engine through the AXI interface; During transmission, AXI streams distribute data to different AI engine kernels according to their needs; at the same time, AXI streams are responsible for coordinating data transmission between multiple kernels. After the calculation is completed, the AXI stream then transmits the result data from the AI ​​engine to the next processing unit or outputs it to the buffer.

5. The implementation method of a general fixed-point matrix multiplier based on a high-performance computing architecture in FPGA according to claim 1, characterized in that, The pipeline for implementing high-throughput vectorized matrix multiplication operations on the AI ​​engine vector processor includes: The input matrix data stream is preprocessed by using the local storage buffer of the AI ​​engine and decomposing the matrix into smaller data blocks through a window function to adapt to the vector size and parallel capabilities of the processing unit. Perform core multiplication operations, utilize the VLIW architecture to issue multiplication instructions in parallel, and execute compound instructions to enhance instruction pipeline execution capabilities; By leveraging the parallelism of the core array, multi-core synchronous multiplication operations are achieved, optimizing data flow and control logic, and reducing transmission latency; The processing results are post-processed and output, aggregated in local storage, and then transformed and output using window functions.

6. The implementation method of a general fixed-point matrix multiplier based on a high-performance computing architecture in FPGA according to claim 1, characterized in that, The matrix multiplication pipeline includes input data preprocessing, core multiplication operations, computation acceleration processing, and final result post-processing and output, as detailed below: Input data preprocessing: The AI ​​engine has a large amount of local storage as a buffer to store the input matrix data stream. It uses window functions to read the data stream and decompose the matrix into smaller data blocks. The sliding window parameters are set to update the data in the cache window. The data flows from the AI ​​engine array to the AI ​​engine kernel, and the data is re-aligned and formatted to match the processing needs of the AI ​​engine. Core multiplication operations: Instruction scheduling is performed, and multiple multiplication instructions are issued in parallel within the VLIW architecture of the AI ​​engine to make full use of each processing cycle; parallel execution is implemented to execute compound instructions containing multiple multiplication operations to enhance the instruction pipeline processing capability of each core and improve computational density. Accelerated processing: In the AI ​​engine array, the execution of multiplication operations is synchronized across multiple cores, enabling parallel processing through the core array; data flow and control logic are optimized through different jumps and incremental window management, reducing data transmission latency; in addition, by iteratively processing the upper-level blocks, data is efficiently transmitted between AI engines by effectively utilizing the AXI stream interface and data packet management. Final result post-processing and output: The final results are aggregated in local storage, and the results in the accumulator are split and output through window functions, and converted into a format suitable for output requirements; by utilizing the DMA controller and AXI stream of the AI ​​engine, the results are transferred to the next processing stage or memory.

7. An implementation device for a general-purpose fixed-point matrix multiplier based on a high-performance computing architecture in FPGA, characterized in that, include: At least one processor; At least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements the method according to any one of claims 1-6.

8. A computer-readable storage medium storing a processor-executable program, characterized in that, The processor-executable program, when executed by the processor, is used to perform the method as described in any one of claims 1-6.

Images