Method and device for realizing calculation acceleration based on CGRA and storage medium

Abstract

The invention provides a method and device for achieving calculation acceleration based on CGRA and a storage medium, and relates to the technical field of machine learning, and the method comprises the steps: defining a deep learning model through a deep learning framework, and generating a calculation graph; converting the calculation graph into an intermediate representation, and generating an intermediate representation file; performing operator division by using a graph division algorithm based on the intermediate representation file, and mapping the operators based on a cost model to generate a data flow chart; generating a hardware configuration file based on the data flow chart, and configuring the CGRA architecture through an FPGA tool; loading the hardware configuration file into the CGRA, executing a task, and monitoring at the same time; collecting a hardware execution result based on a DMA mechanism; and transmitting the hardware execution result back to the deep learning framework based on an RPC framework. Optimization can be carried out according to the structural characteristics of the deep learning network, and a customized acceleration effect is provided.

Description

Technical Field

[0001] The present invention relates to the field of machine learning technology, and in particular to a method, device and storage medium for realizing computing acceleration based on CGRA. Background Art

[0002] Deep learning models usually require a lot of computing resources, such as convolution operations, matrix multiplication, etc. Traditional CPUs and GPUs may have performance bottlenecks when processing these tasks, so more efficient accelerators are needed to improve computing speed and power efficiency. At present, most deep learning tasks are accelerated using GPUs. GPUs speed up model training and reasoning through parallel computing capabilities and dedicated deep learning libraries such as CUDA and cuDNN. When using GPUs to accelerate Pytorch and TensorFlow codes, problems such as low resource utilization and complex programming will be encountered, as well as high power consumption. Another alternative is to use field-programmable gate arrays (FPGAs), which can perform specific computing tasks by reconfiguring hardware circuits. FPGAs can generally provide higher energy efficiency than GPUs for specific tasks, but require higher programming complexity and design time. In addition, ASICs have poor flexibility and are difficult to modify once the design is completed. The development cycle of ASICs is long and the cost is high.

[0003] Therefore, how to achieve more efficient deep learning model computing acceleration has become a technical problem that needs to be solved urgently. Summary of the invention

[0004] The present invention aims to solve at least one of the technical problems existing in the prior art or related technology, and discloses a method, device and storage medium for realizing computing acceleration based on CGRA. Based on CGRA (reconfigurable computing array), efficient support for specific algorithms is realized, and it can be optimized according to the characteristics of the network structure to provide customized acceleration effects.

[0005] The first aspect of the present invention discloses a method for realizing computing acceleration based on CGRA, comprising: defining a model: defining a deep learning model using a deep learning framework and generating a computational graph; intermediate representation: converting the computational graph into an intermediate representation and generating an intermediate representation file; operator partitioning and mapping: performing operator partitioning based on the intermediate representation file using a graph partitioning algorithm, and mapping the operators based on a cost model to generate a data flow graph; hardware configuration: generating a hardware configuration file based on the data flow graph, and configuring the CGRA architecture through an FPGA tool; CGRA execution and monitoring: loading the hardware configuration file into CGRA and executing tasks while monitoring; collecting execution results: collecting hardware execution results based on a DMA mechanism; and returning results: returning the hardware execution results to the deep learning framework based on an RPC framework.

[0006] In this technical solution, the computing task is treated as a deterministic computing process. By configuring the mapper, the original code is mapped and transformed using a customized hardware accelerator to improve the computing efficiency. The mapping transformation is the core part of CGRA (reconfigurable computing array) to achieve acceleration. It determines the type of acceleration object and inputs the transformed data into the computing unit. The computing unit uses parallel processing and pipeline technology to perform computing processing, and finally outputs the calculation results and optimizes the execution efficiency.

[0007] According to the method for realizing computing acceleration based on CGRA disclosed in the present invention, preferably, it also includes: intermediate representation optimization: performing DAG optimization, constant folding optimization or operator fusion optimization on the intermediate representation file before performing operator partitioning and mapping to generate an optimized intermediate representation file.

[0008] According to the method for realizing computing acceleration based on CGRA disclosed in the present invention, preferably, the deep learning framework includes: PyTorch or TensorFlow.

[0009] According to the method for realizing computing acceleration based on CGRA disclosed in the present invention, preferably, the deep learning model includes: a convolutional neural network model.

[0010] According to the method for realizing computing acceleration based on CGRA disclosed in the present invention, preferably, the computation graph is used to define the data structure and the computation process.

[0011] The second aspect of the present invention discloses a device for realizing computing acceleration based on CGRA, comprising: a memory for storing program instructions; a processor for calling the program instructions stored in the memory to realize a method for realizing computing acceleration based on CGRA as any of the above technical solutions.

[0012] A third aspect of the present invention discloses a computer-readable storage medium storing a program code, wherein the program code is used to implement a method for realizing computing acceleration based on CGRA as in any of the above technical solutions.

[0013] The beneficial effects of the present invention include at least: the present invention regards the computing task as a deterministic computing process, and maps and transforms the original code by configuring the mapper and adopting a customized hardware accelerator, thereby improving the computing efficiency. The contradiction between computing performance and the requirements of complex computing tasks in the prior art is solved, which greatly facilitates the execution and optimization of calculations. The coarse-grained reconfigurability of CGRA enables it to be optimized for specific deep learning tasks, thereby achieving higher computing efficiency. Compared with FPGA, the configuration of CGRA is more static, which can reduce power consumption while ensuring performance. This makes CGRA more advantageous in application scenarios that require low power consumption. Compared with ASIC, the coarse-grained reconfigurability of CGRA enables it to adapt to more application scenarios and new technology trends. This makes CGRA have better scalability and adaptability in the field of deep learning. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 A schematic flow chart of a method for realizing computational acceleration based on CGRA according to an embodiment of the present invention is shown.

[0015] Figure 2 A schematic block diagram of a device for implementing computational acceleration based on CGRA according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0016] In order to more clearly understand the above-mentioned objects, features and advantages of the present invention, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0017] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0018] In the present invention, CGRA is a hardware accelerator architecture that performs specific computing tasks by configuring large-scale computing units and storage units. It allows customized execution of specific algorithms and computing modes at the hardware level to improve computing efficiency. PyTorch and TensorFlow are deep learning frameworks that provide high-level APIs and computing graph concepts for defining and training neural network models.

[0019] Example 1

[0020] like Figure 1 As shown, according to an embodiment of the present invention, a method for realizing computing acceleration based on CGRA is disclosed, comprising:

[0021] PyTorch / TensorFlow code input: Define the deep learning model according to PyTorch / TensorFlow and generate the computational graph of PyTorch / TensorFlow;

[0022] High-level parser: converts PyTorch / TensorFlow computational graphs into intermediate representations;

[0023] Intermediate representation optimization: optimization through graph optimization, constant folding, and operator fusion;

[0024] The specific process of graph optimization, constant folding, and algorithm fusion includes:

[0025] Build the computation graph:

[0026] In this step, you first need to convert the model in Pytorch or TensorFlow into a computational graph. The computational graph consists of nodes (representing computational operations) and edges (representing data flow). Nodes may include various mathematical operations (such as addition, multiplication), activation functions (such as ReLU, sigmoid), data input and output operations, etc. Edges connect these nodes, indicating that data is passed from one node to another.

[0027] Identify optimization opportunities:

[0028] In the computational graph, by analyzing the relationships between nodes and edges, various optimization opportunities can be identified, such as constant folding (replacing constant expressions with their results), operator fusion (merging multiple independent operators into a composite operator), etc. In addition, redundant nodes and edges (such as unnecessary calculations or data duplication) and nodes that can be executed in parallel can also be identified.

[0029] Application optimization technology:

[0030] For the identified optimization opportunities, the corresponding optimization techniques are applied. For constant folding, the value of the constant expression can be directly calculated during compilation and the result can be used to replace the original expression. For operator fusion, multiple independent operators can be combined into a composite operator to reduce the data transmission overhead and improve the parallel processing efficiency. At the same time, the structure of the computational graph can also be adjusted to better adapt to the parallel processing capabilities and data reuse characteristics of CGRA.

[0031] Generate the optimized computational graph:

[0032] After the above optimization steps, an optimized computational graph is generated. Compared with the original computational graph, this computational graph has fewer nodes and edges, higher computational efficiency and lower memory usage.

[0033] High-level graph optimizer and partitioning: operator splitting and reorganization, operator mapping and allocation, and data flow graph generation; the specific process of computation graph operator splitting and reorganization, operator mapping and allocation, and data flow graph generation includes:

[0034] Operator Splitting and Recombining

[0035] Splitting: Splitting complex operators into simpler sub-operators for more refined optimization. For example, in deep learning, a complex convolution operation can be split into multiple simple matrix multiplication operations. Reorganization: Merging multiple simple operations into more complex operators to reduce computational overhead and improve parallelism. For example, merging multiple small matrix multiplication operations into a large matrix multiplication operation can more effectively utilize the parallel computing power of CGRA.

[0036] Operator Mapping and Assignment

[0037] Mapping: Mapping operators in the high-level graph to the underlying hardware or software execution units. This usually requires considering factors such as the type of operator, the amount of computation, and data dependencies. Allocation: Allocating operators to specific hardware resources, such as specific PEs (processing units) in CGRA, to maximize computational efficiency and resource utilization.

[0038] Data flow diagram generation

[0039] A data flow diagram is a graphical representation that describes the flow and processing of data in a system. In high-level graph optimization, data flow diagrams are used to show the data dependencies and computational processes between operators. And the subsequent software-level configuration generation of CGRA. By generating a data flow diagram, the overhead of data transmission and storage is reduced, and the optimal configuration path of CGRA is achieved.

[0040] Operator library selection and scheduling: selection of optimal operator implementation, hardware-specific optimization, and application of scheduling strategies;

[0041] The process of selecting the optimal operator implementation includes:

[0042] On different hardware and software platforms, the same operator may have multiple implementations. Selecting the optimal operator implementation can significantly improve computational efficiency and accuracy. This usually requires considering factors such as the operator's computational complexity, memory usage, and parallelism, and evaluating it in combination with the specific hardware and software environment.

[0043] The process of hardware specific optimization includes:

[0044] Customized optimization of operators for specific hardware platforms can further improve computing performance. For example, in edge computing, edge devices are usually subject to power and heat dissipation limitations, so low-power design is the key to edge CGRA optimization. By optimizing circuit design and algorithms, unnecessary computing overhead and energy consumption can be reduced; in cloud scenarios, large-scale data processing and complex computing tasks are usually involved, such as big data analysis, machine learning, etc. Therefore, it is necessary to optimize the computing performance and parallelism of CGRA. In the above different scenarios, operators need to be customized and optimized, and different operator implementation methods should be selected to achieve the optimal processing efficiency.

[0045] The process of applying the scheduling strategy includes:

[0046] The scheduling strategy is used to determine the execution order and resource allocation of operators. A reasonable scheduling strategy can significantly improve the throughput and response time of the system. Common scheduling strategies include static scheduling, dynamic scheduling, heuristic scheduling, etc. In practical applications, it is necessary to select a suitable scheduling strategy based on specific scenarios and requirements.

[0047] Low-level compilation and optimization: code generation and optimization, assembly and linking, binary generation;

[0048] Hardware generation and configuration management: generate hardware configuration files, configure CGRA architecture, and perform hardware initialization and testing;

[0049] CGRA execution: running tasks, real-time task management and monitoring;

[0050] Result return: return the calculation result;

[0051] Result return and collection: Return the calculation results to the framework.

[0052] Example 2

[0053] According to another embodiment of the present invention, a specific implementation process of accelerating the computation of a convolutional neural network (CNN) using the above-mentioned method for realizing computation acceleration based on CGRA is disclosed:

[0054] 1. Model definition and export:

[0055] Define a simple CNN model and export it to ONNX format.

[0056] Steps: Define a simple convolutional neural network (CNN) model using a high-level framework such as PyTorch or TensorFlow. Export the model to ONNX (Open Neural Network Exchange) format for model exchange between different frameworks and tools.

[0057] Input: Neural network model definition code (such as PyTorch or TensorFlow script).

[0058] Output: Model file in ONNX format.

[0059] Code Example (PyTorch):

[0060] import torch

[0061] import torch.nn as nn

[0062] import torch.onnx

[0063] class SimpleCNN(nn.Module):

[0064] def __init__(self):

[0065] super(SimpleCNN, self).__init__()

[0066] self.conv1 = nn.Conv2d(1, 6, 3)

[0067] self.conv2 = nn.Conv2d(6, 16, 3)

[0068] self.fc1 = nn.Linear(16*6*6, 120)

[0069] self.fc2 = nn.Linear(120, 84)

[0070] self.fc3 = nn.Linear(84, 10)

[0071] def forward(self, x):

[0072] x = self.conv1(x)

[0073] x = torch.relu(x)

[0074] x = self.conv2(x)

[0075] x = torch.relu(x)

[0076] x = x.view(-1, 16*6*6)

[0077] x = self.fc1(x)

[0078] x = torch.relu(x)

[0079] x = self.fc2(x)

[0080] x = torch.relu(x)

[0081] x = self.fc3(x)

[0082] return x

[0083] model = SimpleCNN()

[0084] dummy_input = torch.randn(1, 1, 32, 32)

[0085] torch.onnx.export(model, dummy_input, "simple_cnn.onnx")

[0086] 2. Model conversion and optimization:

[0087] Convert ONNX models to MLIR format and perform intermediate representation optimization.

[0088] Operation steps: Use the MLIR compiler framework to convert the ONNX model to the MLIR format. Perform intermediate representation optimization on the model under the MLIR framework, such as operator fusion and memory layout optimization.

[0089] Input: Model file in ONNX format.

[0090] Output: Optimized model in MLIR format.

[0091] Conversion command:

[0092] onnx-mlir simple_cnn.onnx -o simple_cnn.mlir

[0093] Optimized code:

[0094] from my_dag_optimizer import optimize_dag

[0095] from polyhedral_optimizer import fuse_operations

[0096] # Load the MLIR model

[0097] with open("simple_cnn.mlir", "r") as f:

[0098] ir = f.read()

[0099] # Optimize DAG

[0100] optimized_ir = optimize_dag(ir)

[0101] # Perform operator fusion

[0102] fused_ir = fuse_operations(optimized_ir)

[0103] # Save optimized MLIR

[0104] with open("optimized_simple_cnn.mlir", "w") as f:

[0105] f.write(fused_ir)

[0106] 3. Operator partitioning and mapping:

[0107] Operator partitioning is performed using a graph partitioning algorithm and mapping is performed based on a cost model.

[0108] Operation steps: Use the graph partitioning algorithm to divide the operators in the optimized MLIR model into multiple subgraphs, each of which can be mapped to different parts of the CGRA. Select the best mapping scheme based on the cost model (such as latency, power consumption, resource occupancy, etc.).

[0109] Input: Optimized model in MLIR format.

[0110] Output: operator partitioning results and mapping scheme.

[0111] Code example:

[0112] from metis import part_graph

[0113] from cost_model import map_operations

[0114] # Operator Partitioning

[0115] parts = part_graph(fused_ir, nparts=2)

[0116] Operator Mapping

[0117] mapped_ir = map_operations(fused_ir, cost_model, parts)

[0118] # Save the mapped MLIR

[0119] with open("mapped_simple_cnn.mlir", "w") as f:

[0120] f.write(mapped_ir)

[0121] 4. Generate hardware configuration file:

[0122] Use Verilog to generate hardware configuration files and use the CGRA tool to perform hardware configuration.

[0123] Operation steps: Generate CGRA hardware configuration file using hardware description language such as Verilog. Compile the hardware configuration file through CGRA tool to generate bitstream file.

[0124] Input: Operator partitioning results and mapping scheme. CGRA hardware description file.

[0125] Output: CGRA bitstream file.

[0126] Verilog code example:

[0127] module simple_cnn(

[0128] input wire clk,

[0129] input wire rst,

[0130] input wire [31:0] input_data,

[0131] output wire [31:0] output_data );

[0133] / / Verilog hardware description code

[0134] endmodule

[0135] 5. CGRA Implementation and Monitoring

[0136] Load the generated bitstream file into CGRA and execute the task while monitoring.

[0137] Operation steps: Load the generated bitstream file to CGRA. Load the optimized model data to the memory of CGRA. Start CGRA to execute the task and monitor the task execution in real time through the monitoring unit.

[0138] Input: CGRA bitstream file. Optimized model data.

[0139] Output: Task execution results (stored in CGRA memory). Monitoring log (for debugging and performance analysis).

[0140] Code example (C language):

[0141] #include<papi.h>

[0142] void load_configuration(const char *config_file);

[0143] void execute_task();

[0144] int main() {

[0145] PAPI_library_init(PAPI_VER_CURRENT);

[0146] / / Load the hardware configuration file

[0147] load_configuration("simple_cnn.bit");

[0148] / / Execute the task and monitor performance

[0149] int EventSet = PAPI_NULL;

[0150] long long values[1];

[0151] PAPI_create_eventset(&EventSet);

[0152] PAPI_add_event(EventSet, PAPI_TOT_CYC);

[0153] PAPI_start(EventSet);

[0154] execute_task();

[0155] PAPI_stop(EventSet, values);

[0156] printf("Total cycles: %lld\n", values[0]);

[0157] return 0;

[0158] }

[0159] 6. Result collection and return:

[0160] Use DMA technology to collect hardware execution results and return them to the high-level framework.

[0161] Operation steps: Use DMA (Direct Memory Access) technology to collect the task execution results on CGRA into the host memory. Process and organize the collected results for subsequent use.

[0162] Input: Task execution results on CGRA.

[0163] Output: Result data in host memory.

[0164] Code example (C language):

[0165] void start_dma_transfer(void *src, void *dst, size_t size);

[0166] void wait_for_dma_completion();

[0167] void collect_results() {

[0168] / / Assume that the execution result is saved at hardware address 0x80000000

[0169] void *hardware_address = (void *)0x80000000;

[0170] void *cpu_address = malloc(result_size);

[0171] / / Start DMA transfer

[0172] start_dma_transfer(hardware_address, cpu_address, result_size);

[0173] / / Wait for DMA to complete

[0174] wait_for_dma_completion();

[0175] / / Process the result (e.g., return via RPC)

[0176] }

[0177] 7. Return the results to the high-level framework:

[0178] Use RPC to pass the results back to PyTorch / TensorFlow.

[0179] Operation steps: Use RPC (Remote Procedure Call) technology to transfer the result data from the host memory back to the high-level framework (such as PyTorch or TensorFlow). Post-process and analyze the results in the high-level framework.

[0180] Input: Result data in host memory.

[0181] Output: Result objects in a high-level framework (can be used for subsequent analysis or visualization).

[0182] Code example (Python):

[0183] import xmlrpc.client

[0184] proxy = xmlrpc.client.ServerProxy("http: / / localhost:8000 / ")

[0185] result = proxy.get_result()

[0186] print("Execution result:", result)

[0187] Example 3

[0188] like Figure 2 As shown, according to another embodiment of the present invention, a device 200 for realizing computing acceleration based on CGRA is also disclosed, including: a memory 201, used to store program instructions; a processor 202, used to call the program instructions stored in the memory to implement the method for realizing computing acceleration based on CGRA as in the above embodiment.

[0189] Example 4

[0190] According to another embodiment of the present invention, a computer-readable storage medium is disclosed. The computer-readable storage medium stores program code, and the program code is used to implement the method for realizing computing acceleration based on CGRA as in the above embodiment.

[0191] All or part of the steps in the various methods of the above embodiments can be completed by controlling the relevant hardware through a program, and the program can be stored in a readable storage medium, and the storage medium includes a read-only memory (ROM), a random access memory (RAM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electronically erasable programmable read-only memory (EEPROM), a compact disc (CD-ROM) or other optical disc storage, magnetic disk storage, magnetic tape storage, or any other readable medium that can be used to carry or store data.

[0192] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for realizing computing acceleration based on CGRA, characterized in that: include: Define model: Use deep learning framework to define deep learning model and generate computational graph; Intermediate representation: convert the computational graph into an intermediate representation and generate an intermediate representation file; Operator partitioning and mapping: performing operator partitioning based on the intermediate representation file using a graph partitioning algorithm, and mapping the operators based on a cost model to generate a data flow graph; Hardware configuration: Generate a hardware configuration file based on the data flow diagram, and configure the CGRA architecture through FPGA tools; CGRA execution and monitoring: loading the hardware configuration file into CGRA and executing the task while monitoring; Collect execution results: collect hardware execution results based on DMA mechanism; Result feedback: The hardware execution result is transmitted back to the deep learning framework based on the RPC framework.

2. The method for realizing computing acceleration based on CGRA according to claim 1, characterized in that: Also includes: Intermediate representation optimization: Before performing operator partitioning and mapping, DAG optimization, constant folding optimization or operator fusion optimization is performed on the intermediate representation file to generate an optimized intermediate representation file.

3. The method for realizing computing acceleration based on CGRA according to claim 1, characterized in that: The deep learning framework includes: PyTorch or TensorFlow.

4. The method for realizing computing acceleration based on CGRA according to claim 1, characterized in that: The deep learning model includes: a convolutional neural network model.

5. The method for realizing computational acceleration based on CGRA according to any one of claims 1 to 4, characterized in that: The computational graph is used to define the data structure and computational process.

6. A device for realizing computing acceleration based on CGRA, characterized in that: include: A memory for storing program instructions; A processor, configured to call the program instructions stored in the memory to implement the method for realizing computing acceleration based on CGRA as described in any one of claims 1 to 5.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a program code, and the program code is used to implement the method for realizing computing acceleration based on CGRA according to any one of claims 1 to 5.

Images