FPGA sparse operator acceleration method based on StarCoder2 code model

Abstract

This invention discloses an FPGA sparse operator acceleration method based on the StarCoder2 code model. The method involves inputting the target operator's code into the StarCoder2 model, which intelligently identifies its sparse pattern and key sparse dimensions. Based on the identified sparse pattern, the optimal FPGA sparse storage format is adaptively selected, and the weights and input data are converted and stored in the FPGA's on-chip memory. Customized FPGA computing cores are generated based on the key sparse dimensions: parallel computing units with fixed data paths for structured sparse designs and dynamically scheduled processing units for unstructured sparse designs. The storage format and computing core structure are iteratively optimized through hardware simulation until performance targets are met. The optimized design is deployed to actual hardware, and sparsity change data is collected during operation and fed back to the StarCoder2 model to continuously update its identification logic, achieving end-to-end iterative optimization. This method automates and intelligently accelerates sparse operator design, significantly improving the efficiency, flexibility, and resource utilization of FPGAs when handling sparse computations.

Description

Technical Field

[0001] This invention relates to the field of FPGA sparse operator acceleration technology, specifically to an FPGA sparse operator acceleration method based on the StarCoder2 code model. Background Technology

[0002] With the rapid development of artificial intelligence and big data applications, sparse operators (such as sparse matrix multiplication and sparse convolution) are widely used in tasks such as deep learning inference, image processing, and scientific computing. Since most elements in sparse operators are zero, traditional full-scale computation methods, when executed on hardware platforms such as FPGAs, will perform a large amount of invalid computation, leading to wasted computing resources, reduced energy efficiency, and increased latency.

[0003] FPGAs, with their reconfigurability, high parallelism, and low power consumption, have become an ideal platform for accelerating sparse operators. However, efficiently implementing sparse operator acceleration on FPGAs still faces the following key technical challenges:

[0004] Sparse pattern recognition is difficult: different operators have different sparse structures (such as structured sparsity and unstructured sparsity). Traditional methods rely on manual analysis or simple heuristic rules, resulting in low recognition accuracy and poor adaptability.

[0005] Inflexible storage format selection: Using a fixed sparse storage format (such as CSR, CSC, COO, etc.) cannot be dynamically optimized according to the actual sparse characteristics, resulting in insufficient storage bandwidth utilization or high access latency.

[0006] The design of computational cores lacks adaptability: the design of computational units for different sparsity patterns is relatively rigid, making it difficult to switch flexibly between structured and unstructured sparsity, which affects computational parallelism and resource utilization.

[0007] Lack of dynamic feedback and continuous optimization mechanisms: Most existing FPGA acceleration solutions are statically designed and cannot be adjusted according to changes in sparsity during runtime after deployment, making it difficult to adapt to dynamic load and data distribution changes. Summary of the Invention

[0008] To address this, this invention provides an FPGA sparse operator acceleration method based on the StarCoder2 code model, which solves the technical problem that existing technologies struggle to automatically identify sparse patterns and have low computational efficiency.

[0009] To achieve the above objectives, the embodiments of the present invention provide the following technical solutions:

[0010] According to a first aspect of the present invention, an FPGA sparse operator acceleration method based on the StarCoder2 code model is provided, the method comprising:

[0011] Input the code of the target operator into the StarCoder2 model to obtain the sparse mode and key sparse dimension of the operator output by the model;

[0012] Based on the sparse pattern identified by the StarCoder2 model, the corresponding FPGA sparse storage format is selected, and the weights and input data of the operator are converted into the format and stored in the FPGA on-chip memory.

[0013] Based on the key sparse dimension output by the StarCoder2 model, an FPGA computing core for generating sparse operators is generated: if the sparse mode is structured sparse, a parallel computing unit with a fixed data path is designed; if the sparse mode is unstructured sparse, a dynamically scheduled processing unit is designed.

[0014] The sparse storage module and the computing core are integrated into the FPGA design for hardware simulation. Performance indicators are statistically analyzed. If the preset target is not met, the storage format or the computing core structure is adjusted.

[0015] The optimized FPGA design is deployed to the target hardware, runs the actual task and collects sparsity change data, which is fed back to the StarCoder2 model to update its sparse pattern recognition logic, and the accelerated design of subsequent operators is optimized in a loop.

[0016] Furthermore, the code for the target operator is input into the StarCoder2 model, specifically including:

[0017] The source code or pseudocode of the target operator is preprocessed and organized into a standardized plain text format.

[0018] Input the preprocessed code through the StarCoder2 model's API interface or command-line tools;

[0019] The model performs inference based on the Transformer architecture and outputs structured data containing descriptions of the sparse patterns of the operators and indexes of key sparse dimensions.

[0020] Furthermore, the sparsity mode includes structured sparsity and unstructured sparsity; the key sparsity dimension is determined by calculating the sparsity along each dimension and selecting the dimension index corresponding to the maximum value.

[0021] Furthermore, the FPGA sparse storage format includes compressed sparse rows (CSR), compressed sparse columns (CSC), or coordinates (COO) format; the corresponding storage format is selected based on the sparsity index calculated according to the sparse mode and a preset threshold.

[0022] Further, based on the sparse pattern identified by the StarCoder2 model, a corresponding FPGA sparse storage format is selected, and the operator weights and input data are converted to this format and stored in the FPGA on-chip memory, including:

[0023] Extract non-zero elements and their coordinate information from the sparse mask matrix, and construct the corresponding value array, index array, and pointer array;

[0024] Calculate the memory footprint of the converted data and perform address allocation and alignment in the Block Random Access Memory (BRAM) of the FPGA;

[0025] The format-converted data is written to the on-chip memory of the FPGA via the Direct Memory Access (DMA) interface.

[0026] Furthermore, a parallel computing unit with a fixed data path is designed, including:

[0027] The input data and weight matrix are divided into blocks of fixed size;

[0028] Multiple parallel processing units (PEs) are configured, each PE independently processes a data block and performs accumulation calculations through a multiply-accumulate tree.

[0029] Furthermore, a dynamic scheduling processing unit is designed, including:

[0030] Weight data is stored using the Compressed Sparse Line (CSR) format.

[0031] Design a dynamic index scheduler to calculate the storage address of input data in real time based on the column index of non-zero elements and control the execution order of multiplication and addition operations.

[0032] Furthermore, the performance metrics include computation latency, floating-point operation rate (FLOPS), and logic resource utilization. The hardware simulation is performed using a standard sparse matrix test set. If the performance metrics fail to meet the standards, iterative optimization is performed by switching the storage format or adjusting the parallelism and pipeline depth of the computation cores.

[0033] Furthermore, the collection of sparsity variation data specifically includes:

[0034] When the FPGA is running actual tasks, the number of non-zero elements and the total dimension of the data tensor are collected in real time through the monitoring module, and the temporal sparsity sequence and its rate of change are calculated.

[0035] The rate of change data is input as a feature vector into the sparse pattern recognition module of the StarCoder2 model.

[0036] Furthermore, the feedback is fed back to the StarCoder2 model to update its sparse pattern recognition logic, specifically including:

[0037] A training sample set is constructed based on the collected sparsity variation data and the corresponding actual sparsity patterns.

[0038] The sample set is used to perform supervised training on the sub-network responsible for sparse pattern recognition in the StarCoder2 model, and its model parameters are updated.

[0039] The updated model parameters are applied to sparse pattern recognition and accelerated design generation of subsequent new operators.

[0040] The embodiments of the present invention have the following advantages:

[0041] This invention inputs the target operator's code into the StarCoder2 model, intelligently identifying its sparse pattern and key sparse dimensions. Based on the identified sparse pattern, it adaptively selects the optimal FPGA sparse storage format and converts and stores the weights and input data into the FPGA's on-chip memory. Customized FPGA computing cores are generated based on the key sparse dimensions: parallel computing units with fixed data paths for structured sparse designs and dynamically scheduled processing units for unstructured sparse designs. The storage format and computing core structure are iteratively optimized through hardware simulation until performance meets the requirements. The optimized design is deployed to actual hardware, and sparsity change data is collected during operation and fed back to the StarCoder2 model to continuously update its identification logic, achieving end-to-end iterative optimization. This invention automates and intelligently accelerates sparse operator design, significantly improving the efficiency, flexibility, and resource utilization of FPGAs in handling sparse computations. Attached Figure Description

[0042] To more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely exemplary, and those skilled in the art can derive other embodiments based on the provided drawings without creative effort.

[0043] The structures, proportions, sizes, etc. illustrated in this specification are only for the purpose of assisting those skilled in the art in understanding and reading the content disclosed herein, and are not intended to limit the conditions under which the present invention can be implemented. Therefore, they have no substantial technical significance. Any modifications to the structure, changes in the proportions, or adjustments to the size, without affecting the effects and objectives that the present invention can produce, should still fall within the scope of the technical content disclosed in the present invention.

[0044] Figure 1A flowchart illustrating an FPGA sparse operator acceleration method based on the StarCoder2 code model provided in an embodiment of the present invention;

[0045] Figure 2 This is a schematic diagram illustrating the application process of an FPGA sparse operator acceleration method based on the StarCoder2 code model, provided in an embodiment of the present invention. Detailed Implementation

[0046] The following specific embodiments illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0047] In today's digital age, code models are being used more and more widely, especially in fields such as artificial intelligence and big data analytics. StarCoder2, as an advanced code generation model, can generate high-quality code based on the input context, greatly improving code development efficiency. However, as the model size continues to increase, its computational and storage requirements also increase dramatically, leading to slower computation speeds and making it difficult to meet the needs of some application scenarios with high real-time requirements.

[0048] Sparse operators have important applications in many computational tasks, such as sparse matrix operations in deep learning and sparse representations in signal processing. However, the computation of sparse operators is often complex, and traditional computational methods are inefficient. FPGAs, with their reconfigurability and strong parallel computing capabilities, can be flexibly configured according to specific computational tasks, thereby achieving efficient acceleration of sparse operators.

[0049] The main problem that FPGA sparse operator acceleration methods need to solve is how to improve the computational efficiency of FPGA for sparse operators and reduce the resource consumption and time cost in the computation process. The core goal is to enable FPGA to complete tasks faster and with less resources when processing sparse operators.

[0050] In the field of FPGA sparse operator acceleration methods, there are several common related technical solutions. Traditional full-data computation methods perform calculations on all data regardless of its sparsity, leading to a large amount of invalid computation, wasting FPGA computing resources and time, and resulting in very low efficiency. Another method involves manually identifying sparsity patterns, requiring manual analysis of operator sparsity, which is not only time-consuming and labor-intensive but also prone to errors, especially with complex operators. Additionally, some fixed storage format methods use a fixed storage format regardless of the actual sparsity pattern of the operator, failing to adapt well to different sparsity situations and wasting storage resources. Finally, there are methods lacking feedback optimization mechanisms, using the design directly after completion without adjustment based on actual operating conditions, and unable to cope with dynamic changes in data sparsity.

[0051] To address the technical problems mentioned above, such as the difficulty in automatically identifying sparse patterns and low computational efficiency of existing technologies.

[0052] refer to Figure 1 and Figure 2 This invention discloses an FPGA sparse operator acceleration method based on the StarCoder2 code model, comprising: inputting the code of the target operator into the StarCoder2 model, and obtaining the sparse mode and key sparse dimension of the operator output by the model;

[0053] Based on the sparse pattern identified by the StarCoder2 model, the corresponding FPGA sparse storage format is selected, and the weights and input data of the operator are converted into the format and stored in the FPGA on-chip memory.

[0054] Based on the key sparse dimension output by the StarCoder2 model, an FPGA computing core for generating sparse operators is generated: if the sparse mode is structured sparse, a parallel computing unit with a fixed data path is designed; if the sparse mode is unstructured sparse, a dynamically scheduled processing unit is designed.

[0055] The sparse storage module and the computing core are integrated into the FPGA design for hardware simulation. Performance indicators are statistically analyzed. If the preset target is not met, the storage format or the computing core structure is adjusted.

[0056] The optimized FPGA design is deployed to the target hardware, runs the actual task and collects sparsity change data, which is fed back to the StarCoder2 model to update its sparse pattern recognition logic, and the accelerated design of subsequent operators is optimized in a loop.

[0057] 1) Input the code of the target operator into the StarCoder2 model and obtain the sparse pattern and key sparse dimensions of the operator output by the model. The specific implementation method of this step is as follows:

[0058] Prepare the input code for the target operator. Organize the operator's source code or pseudocode snippets into plain text format, such as a Python function or C++ kernel code, describing the operator's computational logic. The code should include input tensor declarations and operation details, such as matrix multiplication or convolution operations. Input the code data using the StarCoder2 API or command-line tools. During input, ensure code formatting is standardized, remove comments and irrelevant characters, and load as a plain string. The model requires input length not exceeding 4096 tokens; truncation or segmentation may be necessary. For large operators, prioritize using a representative subset of the code, such as retaining only the core loop.

[0059] The StarCoder2 model performs inference based on the Transformer architecture. The model loads pre-trained weights and parses the code's syntax and semantic structure using a self-attention mechanism. The input code is embedded as a high-dimensional vector sequence and processed through multiple Transformer blocks. The model's output layer generates a structured JSON or text response, containing a description of the operator's sparse pattern and key sparse dimension indices. The sparse pattern represents the distribution format of non-zero elements in the tensor processed by the operator, such as Compressed Sparse Lines (CSR) or block sparse format. Output parsing involves regular expression matching or a JSON parser to extract key fields.

[0060] After obtaining the model output, extract and verify the sparse pattern and key sparse dimensions. The sparse pattern is defined as the operator input tensor. The storage scheme is designed as follows: Dimensions Global sparsity is calculated as the proportion of non-zero elements:

[0061]

[0062] in Indicates the size of the set. Key sparse dimension. Indexes identified as maximizing sparsity along dimensions:

[0063]

[0064] final

[0065] Dimensional sparsity This quantifies the proportion of non-empty slices in this dimension; a higher value indicates greater sparsity. The output is stored as a tuple ( ), for subsequent optimization.

[0066] 2) Based on the sparse pattern identified by StarCoder2, select the corresponding FPGA sparse storage format, convert the operator weights and input data to this format, and store them in the FPGA's on-chip memory. The specific implementation method for this step is as follows:

[0067] The output of the identified sparse pattern is obtained from StarCoder2, which is usually a binary mask matrix. Alternatively, a list of coordinates of non-zero elements, representing the weight matrix. and input data Non-zero positions in the model. StarCoder2 generates this after analyzing the model structure. ,in express or If non-zero, then 0; otherwise, 0. Sparsity. As a key indicator in calculations, for example, for weights:

[0068]

[0069] in It represents the total number of non-zero elements. This is similar to calculating the input data. .

[0070] Based on sparsity and Select the optimal sparse memory format for the FPGA. Common formats include CSR (Compressed Sparse Row), CSC (Compressed Sparse Column), or COO (Coordinate). The decision threshold is set as follows: if... Prioritize CSR for optimized row access efficiency; otherwise, consider CSC or COO. This selection is based on FPGA hardware characteristics, such as BRAM bandwidth and compute unit parallelism. The conversion process begins... and Transform from dense format to the selected format. For CSR, define three arrays: an array of non-zero values. Column index array and row pointer array . The initialization formula is Then, recursively:

[0071]

[0072] for ,in It is the first Non-zero numbers in the row. Treat them the same way. This ensures that the conversion algorithm is parallelized to leverage the FPGA pipeline.

[0073] Calculate the memory footprint of the converted data to allocate on-chip memory. Estimate the storage size using the FPGA's BRAM resources. For the weighted CSR format, the total byte requirement is:

[0074]

[0075] in It is the size of a floating-point number (e.g., 4 bytes). It is an integer size (e.g., 4 bytes). Similar to computational input Finally, data is written via the FPGA's AXI DMA interface, with the address mapping formula as follows:

[0076]

[0077] in It is a data block index. This is the on-chip memory start address. Optimized alignment is used to match the BRAM block size, ensuring low-latency access. The entire process is script-controlled on the FPGA soft-core processor, with an average execution time of less than 1ms.

[0078] 3) FPGA computing cores for generating sparse operators based on the key sparse dimensions output by StarCoder2: For structured sparseness, design parallel computing units with fixed data paths; for unstructured sparseness, design dynamically scheduled processing units. The specific implementation method for this step is as follows:

[0079] Analyze the key sparse dimension information in the StarCoder2 model output, including sparse type (structured or unstructured) and sparsity. (Defined as the proportion of non-zero elements), block size (If structured) and the dimensions of the weight matrix Determine the sparsity type using the parsing results: if the block size... Existence and Within a specific threshold (e.g.) If the weights are within a certain range (e.g., 0.5%), they are classified as structured sparsity; otherwise, they are considered unstructured sparsity. This judgment is based on the weight distribution analysis after model training to ensure matching with hardware constraints.

[0080] For structured sparse computing, design parallel computing units with fixed data paths. The input vector... and sparse weight matrix Divided into fixed sizes The number of blocks Each processing element (PE) independently computes an output block. Efficient summation is achieved using a multiply-accumulator tree. The data path is fixed: input... and weight Input to PE directly through a predefined pipeline, with no dynamic scheduling overhead. Parallelism is achieved by... Decision to use Multiple PEs operate simultaneously. Formula derivation starts from the original calculation: overall output. After sparsification, the output block is calculated as follows: .in It is a sparse submatrix, with an optimized summation structure: Let the input block... element weighted blocks element ,but Internally, PE implements it as an accumulator. The clock cycle is optimized to FPGA resource allocation utilizes LUTs and DSP units to reduce control logic to 5% with a fixed data flow.

[0081] For unstructured sparse data, a dynamically scheduled processing unit is designed. The storage weights use Compressed Sparse Row (CSR) format, including an array of values. Column index and row pointer The processing unit contains an index scheduler that dynamically calculates the addresses of non-zero elements: the scheduler reads... Generate input data address ,in This refers to the data step size. The scheduler iterates through the rows based on a state machine, controlling data acquisition and performing multiplication-accumulation operations. The output calculation is as follows:

[0082]

[0083] Coherent Derivation of the Formula: From the Original Sparse Matrix-Vector Multiplication Extend to non-zero element index Calculation part and State machine state transition Optimize latency by reducing access time through a prefetching mechanism. The FPGA implementation uses BRAM to store CSR data, the dynamic scheduler hardware occupies less than 10% of logic resources, and supports high sparsity. Scenario. Finally, Verilog code is generated to integrate the computational core and verify that the timing constraints meet the FPGA clock frequency. .

[0084] 4) Integrate the sparse memory module and computing core into the FPGA design for hardware simulation and performance evaluation. If the target is not met, return to step two to adjust the memory format or step three to adjust the computing core structure. The specific implementation method for this step is as follows:

[0085] Configure the sparse storage module to store matrix data in a compressed sparse row format, including the row pointer array. Column index array Sum array The computational core is designed as a multi-parallel unit to perform sparse matrix-vector multiplication operations, and is synthesized using the Vivado HLS tool with a hardware description language. A bitstream is loaded onto an FPGA target board, such as a Kintex-7, and the simulation environment is set up using the Xilinx SDK. Standard test matrices, such as the SuiteSparse dataset, are input for real-time computation. During simulation, signal timing is monitored, and the delay data of the complete input-to-output path is recorded. Performance metrics include computational latency. Floating-point operation speed (FLOPS) and logic resource utilization percentage. The computational model is based on the sparse matrix dimension. Non-zero element number The delay formula is as follows ,in It is the average clock cycle for processing each non-zero element. This refers to the FPGA clock frequency. The FLOPS metric is calculated as follows: Assume that each element corresponds to one multiplication and one addition operation. If the actual test... Exceeding the preset threshold or FLOPS below Then return to step two to optimize the storage format, such as switching to the ELLPACK structure to reduce... Occupying resources, or returning to step three to adjust the computational core parallelism, increase pipeline depth, or decrease it. The evaluation is automated through iterative processes using Python scripts to ensure convergence to the target performance parameters, while resource usage reports are recorded for design validation. This process is repeated until all metrics meet the targets. Optimizing storage format affects memory access efficiency, and adjusting the core structure improves computational throughput.

[0086] 5) Deploy the optimized FPGA design to the target hardware, run the actual task and collect sparsity variation data, feed this data back to StarCoder2 to update the sparse pattern recognition logic, and iteratively optimize subsequent operators to accelerate the design. The specific implementation method for this step is as follows:

[0087] First, deploy the optimized FPGA bitstream file to the target hardware platform, such as a Xilinx Zynq or Intel Stratix FPGA board. Load the bitstream using JTAG or PCIe interfaces, ensuring timing convergence and resource utilization meet constraints. Run real-world tasks such as neural network inference or matrix computation loads, and monitor the sparsity changes of the data tensors in real time using monitoring modules on the FPGA (such as embedded logic analyzers or performance counters). Define sparsity metrics. ,in Indicates the time step. It is a vector The number of non-zero elements, It's the overall dimension. It involves collecting data sequences from multiple task instances and calculating the rate of change. They are then aggregated into a feature vector input dataset.

[0088] Collected Data is transmitted to the StarCoder2 system via Ethernet or USB interface. This system's sparse pattern recognition model is based on a deep neural network structure with the following parameters: Input features Mapping to the predicted output:

[0089]

[0090] in It is a non-linear activation function such as ReLU. The loss function is defined as follows:

[0091]

[0092] The aim is to minimize prediction error. The model parameters are updated using the gradient descent optimization algorithm.

[0093]

[0094] in It's the learning rate. Calculated through backpropagation.

[0095] Feedback after the update At the StarCoder2 logic layer, the sparse pattern recognition rules are adjusted. Based on the new rules, optimized operator acceleration designs are generated iteratively, such as reconstructing sparse matrix multiplication kernels or reducing redundant operations, and repeatedly deployed to the FPGA for verification. The entire process is iterative until the sparsity variation converges or reaches a preset performance threshold, ensuring end-to-end latency and energy consumption optimization.

[0096] This invention combines the StarCoder2 code model with FPGA technology. By analyzing and optimizing the sparse operators involved in the StarCoder2 code model, an acceleration algorithm suitable for FPGA implementation is designed. On the FPGA hardware platform, the parallel computing resources are utilized to process the sparse operators in parallel, greatly improving the computation speed. The core idea is to use the StarCoder2 code model to assist in accelerating FPGA sparse operators. The process involves first inputting the target operator code into the StarCoder2 model to obtain the operator's sparse pattern and key sparse dimensions; then, selecting a suitable FPGA sparse storage format based on the identified sparse pattern, converting the operator's weights and input data, and storing them in the FPGA's on-chip memory; next, generating corresponding FPGA computing cores based on the key sparse dimensions, designing parallel computing units with fixed data paths for structured sparseness, and dynamically scheduled processing units for unstructured sparseness; then, integrating the storage module and computing cores into the FPGA design for hardware simulation, adjusting as needed if performance is unsatisfactory; finally, deploying the optimized design to the target hardware, collecting sparsity change data to feed back to StarCoder2 to update the identification logic, and iteratively optimizing subsequent designs. The highlight is that it utilizes the intelligent recognition capabilities of the StarCoder2 model, which can more accurately handle different sparse patterns and achieve targeted acceleration.

[0097] This acceleration method has broad application prospects in many fields. In software development, it can speed up code generation and improve development efficiency; in the field of artificial intelligence, it can accelerate the training and inference process of models and improve the real-time response capability of the system; in big data analysis, it can quickly process large-scale sparse data and extract valuable information from it.

[0098] This invention is applied to scenarios requiring efficient code processing and FPGA operator acceleration, thereby improving code processing efficiency and FPGA operator execution speed. Utilizing FPGA sparse operator acceleration technology based on the StarCoder2 code model, in software development, when developers need to optimize FPGA-related code to achieve faster algorithm execution, this technology can accelerate sparse operators in the code, reducing computation time. In the field of artificial intelligence chip design, for FPGA-based chips, this technology can accelerate on-chip operator operations, improving the overall chip performance. This invention has wide applications in software development, artificial intelligence chip design, and other fields.

[0099] This invention utilizes StarCoder2 to automatically identify sparse patterns, avoiding the hassle and errors of manual identification and improving accuracy and efficiency. It can select appropriate storage formats based on different sparse patterns, avoiding resource waste associated with fixed storage formats. Furthermore, it features a feedback optimization mechanism that continuously adjusts and optimizes based on actual operating conditions, adapting to dynamic changes in data sparsity.

[0100] When using the method disclosed in the embodiments of this invention, the computational efficiency of FPGA for sparse operators can be significantly improved, resource consumption can be reduced, and computation time can be shortened. In practical applications, for scenarios requiring a large amount of sparse computation, such as deep learning inference and big data processing, the system performance can be greatly improved. The embodiments of this invention solve the problems of inaccurate identification of sparse patterns, inflexible storage format, and lack of dynamic optimization in traditional methods, while also solving the problems of low computational efficiency, serious resource waste, and inability to adapt to dynamic changes when FPGA processes sparse operators.

[0101] Although the present invention has been described in detail above with general descriptions and specific embodiments, modifications or improvements can be made to it, which will be obvious to those skilled in the art. Therefore, all such modifications or improvements made without departing from the spirit of the present invention fall within the scope of protection claimed by the present invention.

Claims

1. A method for FPGA sparse operator acceleration based on StarCoder2 code model, characterized in that, The method includes: Input the code of the target operator into the StarCoder2 model to obtain the sparse mode and key sparse dimension of the operator output by the model; Based on the sparse pattern identified by the StarCoder2 model, the corresponding FPGA sparse storage format is selected, and the weights and input data of the operator are converted into the format and stored in the FPGA on-chip memory. Based on the key sparse dimension output by the StarCoder2 model, an FPGA computing core for generating sparse operators is generated: if the sparse mode is structured sparse, a parallel computing unit with a fixed data path is designed; if the sparse mode is unstructured sparse, a dynamically scheduled processing unit is designed. The sparse storage module and the computing core are integrated into the FPGA design for hardware simulation. Performance indicators are statistically analyzed. If the preset target is not met, the storage format or the computing core structure is adjusted. The optimized FPGA design is deployed to the target hardware, runs the actual task and collects sparsity change data, which is fed back to the StarCoder2 model to update its sparse pattern recognition logic, and the accelerated design of subsequent operators is optimized in a loop.

2. The FPGA sparse operator acceleration method based on the StarCoder2 code model as described in claim 1, characterized in that, Input the code of the target operator into the StarCoder2 model, specifically including: The source code or pseudocode of the target operator is preprocessed and organized into a standardized plain text format. Input the preprocessed code through the StarCoder2 model's API interface or command-line tools; The model performs inference based on the Transformer architecture and outputs structured data containing descriptions of the sparse patterns of the operators and indexes of key sparse dimensions.

3. The FPGA sparse operator acceleration method based on the StarCoder2 code model as described in claim 1, characterized in that, The sparsity modes include structured sparsity and unstructured sparsity; The key sparse dimension is determined by calculating the sparsity along each dimension and selecting the dimension index corresponding to the maximum value.

4. The FPGA sparse operator acceleration method based on the StarCoder2 code model as described in claim 1, characterized in that, The FPGA sparse storage format includes Compressed Sparse Row (CSR), Compressed Sparse Column (CSC), or Coordinate COO format; the corresponding storage format is selected based on the sparsity index calculated according to the sparse mode and a preset threshold.

5. The FPGA sparse operator acceleration method based on the StarCoder2 code model as described in claim 1, characterized in that, Based on the sparse pattern identified by the StarCoder2 model, the corresponding FPGA sparse storage format is selected, and the operator weights and input data are converted to this format and stored in the FPGA on-chip memory, including: Extract non-zero elements and their coordinate information from the sparse mask matrix, and construct the corresponding value array, index array, and pointer array; Calculate the memory footprint of the converted data and perform address allocation and alignment in the Block Random Access Memory (BRAM) of the FPGA; The format-converted data is written to the on-chip memory of the FPGA via the Direct Memory Access (DMA) interface.

6. The FPGA sparse operator acceleration method based on the StarCoder2 code model as described in claim 1, characterized in that, Design a parallel computing unit with a fixed data path, including: The input data and weight matrix are divided into blocks of fixed size; Multiple parallel processing units (PEs) are configured, each PE independently processes a data block and performs cumulative calculations through a multiply-accumulator tree.

7. The FPGA sparse operator acceleration method based on the StarCoder2 code model as described in claim 1, characterized in that, The design includes a dynamic scheduling processing unit, comprising: Weight data is stored using the Compressed Sparse Line (CSR) format. Design a dynamic index scheduler to calculate the storage address of input data in real time based on the column index of non-zero elements and control the execution order of multiplication and addition operations.

8. The FPGA sparse operator acceleration method based on the StarCoder2 code model as described in claim 1, characterized in that, The performance metrics include computation latency, floating-point operation rate (FLOPS), and logic resource utilization. The hardware simulation is performed using a standard sparse matrix test set. If the performance indicators do not meet the standards, iterative optimization is performed by switching the storage format or adjusting the parallelism and pipeline depth of the computing cores.

9. The FPGA sparse operator acceleration method based on the StarCoder2 code model as described in claim 1, characterized in that, The collection of sparsity variation data specifically includes: When the FPGA is running an actual task, the number of non-zero elements and the total dimension of the data tensor are collected in real time through the monitoring module, and the temporal sparsity sequence and its rate of change are calculated. The rate of change data is input as a feature vector into the sparse pattern recognition module of the StarCoder2 model.

10. The FPGA sparse operator acceleration method based on the StarCoder2 code model as described in claim 9, characterized in that, Feedback is fed back to the StarCoder2 model to update its sparse pattern recognition logic, specifically including: A training sample set is constructed based on the collected sparsity variation data and the corresponding actual sparsity patterns. The sample set is used to perform supervised training on the sub-network responsible for sparse pattern recognition in the StarCoder2 model, and its model parameters are updated. The updated model parameters are applied to sparse pattern recognition and accelerated design generation of subsequent new operators.

Images