Method for compiling and running RISC-V and GPNPU cooperative architecture based on TVM framework

Abstract

The application discloses a RISC-V and GPNPU collaborative architecture compiling and running method, device, medium and program based on a TVM framework, which comprises the following steps: an open neural network exchange format file is used to parse operator types, tensor shapes and connection relationships, and the operator types, the tensor shapes and the connection relationships are converted into TVM operator types, TVM tensor shapes and TVM connection relationships under the TVM framework, and the parameters are optimized; according to the optimized TVM operator types, the TVM tensor shapes and the TVM connection relationships, main central processor engineering codes, reduced instruction set architecture codes and general neural network processing unit codes are correspondingly generated; the main central processor engineering codes, the reduced instruction set architecture codes and the general neural network processing unit codes are compiled to obtain executable files, architecture instruction files and instruction files; when data scheduling is performed on the general neural network processing unit, the reduced instruction set architecture is used to complete data scheduling processing of the main central processor by using the executable files, the architecture instruction files and the instruction files.

Description

Technical Field

[0001] This application relates to the field of computer compilation technology, particularly to the field of TVM-based GPNP heterogeneous computing technology, specifically to a compilation and execution method, apparatus, device, and medium for a TVM-based RISC-V and GPNPU collaborative architecture. Background Technology

[0002] With the rapid development of artificial intelligence technology, neural network models have been widely used in fields such as image recognition and speech processing. To meet the requirements of real-time performance and energy efficiency, the RISC-V architecture (Reduced Instruction Set Computing architecture) combined with the GPNPU (General-Purpose Neural Network Processing Unit) to form the GPNPU heterogeneous architecture has become the mainstream solution for edge computing. As an open-source instruction set, the RISC-V architecture has become a key component in heterogeneous computing due to its customizability and high energy efficiency. In particular, when working with accelerators such as GPUs and NPUs, it can optimize performance for specific scenarios.

[0003] However, existing compilation schemes for the GPNPU architecture have the following shortcomings: weak multi-target code generation capability, existing tools are unable to generate multiple types of target code that are compatible with Host, RISC-V and GPNPU at the same time, and usually rely on multiple toolchains to compile step by step, resulting in a fragmented process and poor compatibility; insufficient instruction scheduling coordination, RISC-V and GPNPU lack a unified scheduling mechanism, and often adopt remote scheduling on the Host side, which leads to high communication latency and is prone to hardware resource idleness or conflict, reducing overall computing efficiency.

[0004] Therefore, there is an urgent need for a compilation method that can achieve efficient parsing of ONNX (Open Neural Network Exchange Format) models, customized graph optimization, automatic generation of multi-target code, and unified scheduling of heterogeneous instructions. Summary of the Invention

[0005] The purpose of this application is to overcome the shortcomings of existing technologies and provide a compilation and execution method, apparatus, device, and medium based on a RISC-V and GPNPU collaborative architecture using the TVM framework. This application aims to solve the problems of difficult multi-object code generation and poor instruction scheduling coordination in existing solutions by constructing a full-stack compilation process and a near-data scheduling mechanism to improve model compilation efficiency and heterogeneous architecture execution efficiency.

[0006] To achieve the above objectives, the first technical solution adopted in this application is: a compilation and execution method for a RISC-V and GPNPU collaborative architecture based on the TVM framework, which includes: parsing operator types, tensor shapes, and connection relationships using an Open Neural Network Exchange Format file; converting operator types, tensor shapes, and connection relationships into TVM operator types, TVM tensor shapes, and TVM connection relationships under the TVM architecture; optimizing the operator cache addresses, data storage locations, and instruction scheduling corresponding to the TVM operator types, TVM tensor shapes, and TVM connection relationships to obtain optimized TVM operator types and TVM tensor shapes. The TVM connection relationships are determined; based on the optimized TVM operator type, TVM tensor shape, and TVM connection relationships, the corresponding main CPU project code, RISC-V code, and GPNPU code are generated; using the LLVM architecture, the main CPU project code, RISC-V code, and GPNPU code are compiled to obtain the executable files, architecture instruction files, and instruction files corresponding to the main CPU, RISC-V, and GPNPU, respectively; using the executable files, architecture instruction files, and instruction files, RISC-V completes the main CPU's data scheduling processing during data scheduling in the GPNPU.

[0007] Optionally, the operator cache address, data storage location, and instruction scheduling corresponding to the TVM operator type, TVM tensor shape, and TVM connection relationship are optimized to obtain the optimized TVM operator type, TVM tensor shape, and TVM connection relationship. This includes: establishing a virtual memory mapping table to simulate the storage lifecycle of the static random access memory inside the GPNPU; determining the working order of the processing operators executing the GPNPU according to the virtual memory mapping table; and mapping the output cache address of the currently executing operator to the input address of the next layer operator for processing operators that execute the GPNPU consecutively.

[0008] Optionally, the operator cache address, data storage location, and instruction scheduling corresponding to the TVM operator type, TVM tensor shape, and TVM connection relationship are optimized to obtain the optimized TVM operator type, TVM tensor shape, and TVM connection relationship. This includes: adjusting the location of the TVM tensor shape stored in memory according to the storage format of the TVM tensor shape; and allocating corresponding independent storage areas in static random access memory for RISC-V and GPNPU.

[0009] Optionally, the operator cache address, data storage location, and instruction scheduling corresponding to the TVM operator type, TVM tensor shape, and TVM connection relationship are optimized to obtain the optimized TVM operator type, TVM tensor shape, and TVM connection relationship. This includes: determining the hardware allocation strategy of the hardware resources for executing the computation work of the corresponding processing operators according to the working order of the GPNPU's processing operators; and generating the optimized TVM operator type, TVM tensor shape, and TVM connection relationship with a tag containing the hardware allocation strategy according to the hardware allocation strategy.

[0010] Optionally, based on the optimized TVM operator type, TVM tensor shape, and TVM connection relationship, corresponding main CPU project code, RISC-V code, and GPNPU code are generated, including: reserving a communication interface for communication with GPNPU in the RISC-V code; and converting the GPNPU's dedicated operators into functions that call the GPNPU driver interface by setting the GPNPU code.

[0011] Optionally, using the executable file, architecture instruction file, and instruction file, RISC-V is used to complete the data scheduling process of the main CPU during data scheduling by the GPNPU. This includes: the main CPU loading the architecture instruction file into the corresponding memory area of ​​RISC-V for instruction caching, and loading the instruction file into the corresponding memory area of ​​the general-purpose GPNPU for instruction storage; and when the GPNPU issues a data call instruction, RISC-V, based on the shared register flag relationship with the main CPU, sends the address of the corresponding instruction in the instruction file and the start signal to the corresponding memory area of ​​the GPNPU via the control bus to read the data.

[0012] Optionally, a virtual memory mapping table is established to simulate the storage lifecycle of the static random access memory inside the GPNPU, and the working order of the processing operators executing the GPNPU is determined according to the virtual memory mapping table, including: storing the calculation results of the processing operators into DDR memory when the storage space of the static random access memory is insufficient or when the storage lifecycle of the static random access memory ends.

[0013] The second technical solution adopted in this application is: a compilation and execution device based on a TVM framework RISC-V and GPNPU collaborative architecture, comprising: a parsing module, which uses an Open Neural Network Exchange Format file to parse the operator type, tensor shape, and connection relationship, and converts the operator type, tensor shape, and connection relationship into TVM operator type, TVM tensor shape, and TVM connection relationship under the TVM architecture; an optimization module, which optimizes the operator cache address, data storage location, and instruction scheduling corresponding to the TVM operator type, TVM tensor shape, and TVM connection relationship to obtain the optimized TVM operator type, TVM tensor shape, and TVM connection relationship; and an intermediate generation module. The code generation module generates the main CPU project code, RISC-V code, and GPNPU code based on the optimized TVM operator type, TVM tensor shape, and TVM connection relationships. The compilation module uses the LLVM architecture to compile the main CPU project code, RISC-V code, and GPNPU code to obtain the executable files, architecture instruction files, and instruction files corresponding to the main CPU, RISC-V, and GPNPU, respectively. The scheduling and execution module uses the executable files, architecture instruction files, and instruction files to perform data scheduling for the main CPU using RISC-V during data scheduling by the GPNPU.

[0014] The third technical solution adopted in this application is: a computer device, including a memory, a processor and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the compilation and running method of the TVM-based RISC-V and GPNPU co-architecture of Solution 1.

[0015] The fourth technical solution adopted in this application is: a computer-readable storage medium storing a computer program thereon, which, when loaded by a processor, is able to execute the compilation and running method of the RISC-V and GPNPU co-architecture based on the TVM framework described in Solution 1 above.

[0016] This application features full-stack heterogeneous code automatic generation capability, implementing a mechanism of one-time parsing and three-end generation. By integrating TVM and LLVM, users only need to input a model file to automatically generate three sets of code adapted to the Host (main central processing unit), RISC-V (Reduced Instruction Set Computing), and GPNPU (General Neural Network Processing Unit), solving the problems of fragmented toolchains and the difficulty of manual integration in heterogeneous development. Simultaneously, it achieves extreme memory access efficiency. Through a "zero-copy" graph optimization strategy combined with the caching characteristics of GPNPU, it maximizes the reuse of data between operators on the on-chip SRAM, cutting off the path of intermediate data writing back to DDR, thus breaking through the memory bandwidth bottleneck of heterogeneous computing. Furthermore, this application features efficient fine-grained unified scheduling, innovatively constructing a two-level scheduling system between the Host and RISC-V, delegating high-frequency data scheduling rights to the "near-data" RISC-V, thereby significantly reducing communication latency and achieving pipelined parallelism of RISC-V scalar instructions and GPNPU vector instructions, significantly improving hardware resource utilization. Finally, this application features a scalable dedicated instruction compilation stream, enabling dedicated neural network processors to have standardized compilation capabilities and flexibly adapt to the iteration and expansion of the GPNPU instruction set. Attached Figure Description

[0017] Figure 1 This is a schematic diagram of a specific implementation of the compilation and execution method of the RISC-V and GPNPU co-architecture based on the TVM framework of this application; Figure 2 This is a schematic diagram of the compilation process for the compilation and execution method of the RISC-V and GPNPU co-architecture based on the TVM framework in this application. Figure 3 This is a schematic diagram of the compilation and execution method of the RISC-V and GPNPU co-architecture based on the TVM framework in this application, showing the runtime loading process. Figure 4 This is a schematic diagram of a specific implementation of a compilation and execution device based on a TVM framework and a RISC-V and GPNPU co-architecture according to this application. Detailed Implementation

[0018] The preferred embodiments of this application will now be described in detail with reference to the accompanying drawings, so that the advantages and features of this application can be more easily understood by those skilled in the art, thereby providing a clearer and more definite definition of the scope of protection of this application.

[0019] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes the element.

[0020] TVM, or Tensor Virtual Machine, is an open-source machine learning compiler framework designed to automatically compile deep learning models into efficient code executable on various hardware platforms, thereby optimizing inference performance. TVM imports models from multiple deep learning frameworks, such as TensorFlow, PyTorch, and ONNX, and transforms these learning frameworks into a unified intermediate representation. Then, it utilizes high-level and low-level Inference Frameworks (IR) for graph optimization processes, including operator fusion, quantization, and constant folding, to improve computational efficiency. Subsequently, TVM generates optimized low-level code based on the hardware characteristics of devices such as CPUs, GPUs, and TPUs, and further accelerates the code through scheduling and automatic tuning tools such as Auto TVM or Auto Scheduler.

[0021] Figure 1 This is a schematic diagram of a specific implementation of the compilation and execution method of the RISC-V and GPNPU co-architecture based on the TVM framework of this application.

[0022] like Figure 1As shown, the compilation and running method of the RISC-V and GPNPU collaborative architecture based on the TVM framework in this application includes the following steps: Step S101, parsing the operator type, tensor shape, and connection relationship using the Open Neural Network Exchange Format file, and converting the operator type, tensor shape, and connection relationship into TVM operator type, TVM tensor shape, and TVM connection relationship under the TVM architecture; Step S102, optimizing the operator cache address, data storage location, and instruction scheduling corresponding to the TVM operator type, TVM tensor shape, and TVM connection relationship to obtain the optimized TVM operator type, TVM tensor shape, and TVM connection relationship; Step S10 3. Based on the optimized TVM operator type, TVM tensor shape, and TVM connection relationship, generate the corresponding main CPU project code, RISC-V code, and GPNPU code; Step S104: Using the LLVM architecture, compile the main CPU project code, RISC-V code, and GPNPU code to obtain the executable file, architecture instruction file, and instruction file corresponding to the main CPU, RISC-V, and GPNPU, respectively; Step S105: Using the executable file, architecture instruction file, and instruction file, when the GPNPU performs data scheduling, use RISC-V to complete the data scheduling processing of the main CPU.

[0023] Specifically, the core of the heterogeneous architecture combining RISC-V with GPUs and NPUs lies in task optimization through hardware abstraction and dynamic scheduling. This architecture integrates RISC-V processors, GPUs, and NPUs, managing resources uniformly through a hardware abstraction layer, shielding underlying differences, and allowing tasks to be allocated to the optimal unit as needed. For example, RISC-V handles system scheduling and data preprocessing, GPUs handle parallel-intensive tasks such as image rendering, and NPUs focus on AI inference. The three components achieve low-latency communication through high-speed interconnects such as PCIe or CXL, reducing data transmission overhead. In collaborative computing, dynamic load balancing and energy efficiency optimization are key. The system dynamically adjusts task allocation based on real-time load, power consumption, and latency data. For example, in smart cockpits or mobile devices, lightweight AI models are preferentially processed by the NPU; if the load is too high, it switches to the GPU's Tensor Cores, while voltage-frequency scaling and data locality optimization are used to improve energy efficiency.

[0024] like Figure 1 The compilation and running method of the RISC-V and GPNPU collaborative architecture based on the TVM framework shown includes step S101, parsing the operator type, tensor shape and connection relationship using the Open Neural Network Exchange Format file, and converting the operator type, tensor shape and connection relationship into TVM operator type, TVM tensor shape and TVM connection relationship under the TVM architecture.

[0025] In one specific embodiment of this application, after converting the connection relationships into TVM operator types, TVM tensor shapes, and TVM connection relationships under the TVM architecture, corresponding quantization processing is performed on the TVM operator types, TVM tensor shapes, and TVM connection relationships, and corresponding conventional graph optimization based on nodes and data flows is performed. Specifically, as shown... Figure 2 As shown, TVM reads the input ONNX model file, parses out the operator type, tensor shape and connection relationship, and converts the parsing result into a unified computation graph representation within TVM. It also performs quantization and conventional graph optimization processing on the converted operator type, tensor shape and connection relationship.

[0026] For example, a user loads a YOLOv8n model in ONNX format via a TVM frontend on the host side. The compiler's graph optimization module identifies consecutive convolutional (Conv), sigmoid (Sigmoid) activation, and dot product (Mul) layers, and the output data of the convolutional (Conv) layer is one input data of the sigmoid activation and one input data of the dot product (Mul).

[0027] like Figure 1 The compilation and running method of the RISC-V and GPNPU collaborative architecture based on the TVM framework shown includes step S102, which optimizes the operator cache address, data storage location and instruction scheduling corresponding to the TVM operator type, TVM tensor shape and TVM connection relationship to obtain the optimized TVM operator type, TVM tensor shape and TVM connection relationship.

[0028] In one specific embodiment of this application, a virtual memory mapping table is established to simulate the storage lifecycle of the static random access memory inside the general neural network processing unit; and, based on the virtual memory mapping table, the working order of the processing operators of the general neural network processing unit is determined; and, for the processing operators of the general neural network processing unit that are executed consecutively, the output cache address of the currently executing operator is mapped to the input address of the next layer operator.

[0029] In one specific embodiment of this application, a virtual memory mapping table is established to simulate the storage lifecycle of the static random access memory inside the general neural network processing unit, and the working order of the processing operators of the general neural network processing unit is determined according to the virtual memory mapping table, including: storing the calculation results of the processing operators into DDR memory when the storage space of the static random access memory is insufficient or when the storage lifecycle of the static random access memory ends.

[0030] In one specific embodiment of this application, the location of the TVM tensor shape stored in memory is adjusted according to the storage format of the TVM tensor shape; and corresponding independent storage areas are allocated in static random access memory for the reduced instruction set architecture and the general neural network processing unit. In another specific embodiment of this application, a hardware allocation strategy for the hardware resources that perform the computation of the corresponding processing operators is determined according to the working order of the processing operators of the general neural network processing unit; and, according to the hardware allocation strategy, an optimized TVM operator type, TVM tensor shape, and TVM connection relationship containing the label of the hardware allocation strategy are generated.

[0031] Specifically, based on the TVM graph optimization results, and considering the heterogeneous characteristics of the GPNPU, the computation graph is deeply optimized. This involves establishing a virtual memory mapping table to simulate the lifecycle of the GPNPU's internal SRAM. For operators continuously executed within the GPNPU, the output cache address of the previous layer operator is forcibly mapped directly to the input address of the next layer operator. DDR transfer instructions are only inserted when SRAM space is insufficient or the data lifecycle ends, thereby reducing DDR access. Through a "zero-copy" graph optimization strategy, combined with static simulation of the GPNPU's cache characteristics, data reuse between operators is maximized in on-chip SRAM, cutting off the path of intermediate data writing back to DDR and overcoming the memory bandwidth bottleneck of heterogeneous computing. Zero-copy optimization achieves this by reusing operator inputs and outputs.

[0032] Zero-copy optimization is a technique that improves data transfer performance by reducing the number of system calls and memory data copies, and is particularly suitable for scenarios such as file transfer and network services. Its core idea is to minimize unnecessary data copying and switching between user mode and kernel mode, thereby reducing CPU overhead and increasing throughput. The core principle of zero-copy optimization lies in reducing context switching and data copying. Traditional data transfer, such as using the read and write system calls, requires multiple user mode and kernel mode switches: each system call involves two switches, one from user mode to kernel mode and the other from kernel mode to user mode. This number of switches increases dramatically, especially when processing small blocks of data, causing CPU time to be wasted on saving and restoring context. Simultaneously, data needs to be copied multiple times through user buffers, kernel buffers, etc., increasing the CPU load. Zero-copy reduces these overheads through merging operations, such as using mmap to directly map the kernel buffer to user space, avoiding data copying during data reading, or using system calls like sendfile to transfer data directly from the file descriptor to the buffer in kernel mode, reducing the number of system calls and data movement.

[0033] For example, by querying and determining the preset GPNPU hardware parameters, the SRAM size is 512KB. The compiler finds that the intermediate feature map size is 224KB, which is smaller than the SRAM capacity. Therefore, when generating intermediate code, the compiler points the output address of Conv, the input address of Sigmoid, and one of the input addresses of Mul to the same physical address in the SRAM, such as 0x200000. This configuration not only saves memory but also avoids the latency and bandwidth consumption caused by multiple DDR read / write operations.

[0034] The computation graph is deeply optimized based on the TVM graph optimization results and the heterogeneous characteristics of GPNPU. This includes memory layout optimization, which adjusts the memory layout of tensors in the computation graph according to the tensor storage format of GPNPU, thereby reducing the number of runtime address jumps; and allocating independent memory regions for RISC-V and GPNPU to avoid conflicts in data or instruction access and read / write operations.

[0035] Deep optimization of the computation graph based on TVM graph optimization results and the heterogeneous characteristics of GPNPU includes: instruction scheduling pre-planning, which generates an optimized computation graph containing hardware allocation markers based on the execution order of hardware resource planning operators and hardware allocation strategies. Instruction scheduling is a compiler backend and a key optimization step for modern superscalar, out-of-order execution processors. Its goal is to rearrange the execution order of instructions without changing the semantics to maximize instruction-level parallelism, thereby reducing pipeline stalls caused by data dependencies, control dependencies, and hardware resource conflicts. Pre-planning refers to a series of analyses, modeling, and decision-making processes performed before actual instruction sequencing. It provides information and strategies for the final "scheduler" at both macro and micro levels.

[0036] like Figure 1 The compilation and running method of the RISC-V and GPNPU collaborative architecture based on the TVM framework shown includes step S103, which generates the main central processing unit project code, RISC-V code and GPNPU code according to the optimized TVM operator type, TVM tensor shape and TVM connection relationship.

[0037] In one specific embodiment of this application, a communication interface for communicating with a general neural network processing unit is reserved in the reduced instruction set architecture code; and, by setting the general neural network processing unit code, the dedicated operators of the general neural network processing unit are converted into functions that call the general neural network processing unit driver interface.

[0038] Specifically, based on the optimized computation graph, the system automatically generates C++ project code for the Host, RISC-V, and GPNPU architectures, respectively. The Host CPU project code handles initialization, start / stop control, and instruction file loading logic; the RISC-V project code implements loop control and data transfer logic, and reserves a communication interface with the GPNPU; the GPNPU project code converts dedicated operators into functions that call the GPNPU driver interface, such as matrix operation startup and result reading. By integrating TVM and LLVM, users only need to input a single model file to automatically generate three sets of code adapted for the Host, RISC-V, and GPNPU architectures, solving the problems of fragmented toolchains and the difficulty of manual integration in heterogeneous development.

[0039] like Figure 1 The compilation and running method of the RISC-V and GPNPU collaborative architecture based on the TVM framework shown includes step S104, which uses the LLVM architecture to compile the main CPU project code, RISC-V code and GPNPU code to obtain the executable file, architecture instruction file and instruction file corresponding to the main CPU, RISC-V and GPNPU respectively.

[0040] Specifically, the main CPU project code, RISC architecture code, and general neural network processing unit (GNRF) code are compiled to obtain executable files, architecture instruction files, and instruction files corresponding to the main CPU, RISC architecture, and GNRF, respectively. In other words, the compilation generates three files: host_project, riscv_project, and gpnpu_project, which are the executable file, architecture instruction file, and instruction file, respectively. A custom compiler built based on the LLVM framework generates multiple object files from the three parts of the C++ project code. The LLVM compiler calls x86_64-gcc, riscv64-unknown-elf-gcc, and the custom gpnpu-llvm-backend, ultimately generating host.exe, riscv.po, and gpnpu.po. LLVM is an open-source framework and toolchain for building compilers. It not only provides a highly optimized intermediate representation but also supports a wide range of optimization techniques and backend code generation.

[0041] The Host project is compiled into host.exe and linked with the runtime library; the RISC-V project is compiled into a binary instruction file riscv.po that conforms to the RISC-VISA specification; based on the LLVM Target API extension, the GPNPU project is converted into dedicated instructions. During this process, instruction slot filling optimization is performed, and dependency-free configuration instructions and computation instructions are packaged to generate the binary instruction file gpnpu.po.

[0042] This embodiment implements a scalable dedicated instruction compilation stream, namely a GPNPU compilation backend based on the LLVM Target API extension, which enables the dedicated neural network processor to have standardized compilation capabilities and flexibly adapt to the iteration and expansion of the GPNPU instruction set.

[0043] Specifically, configuration and optimization include loading the GPNPU target architecture configuration and setting optimization options such as O3. This section provides solutions based on general compiler and hardware optimization practices to address the need for loading the GPNPU target architecture configuration and setting O3 optimization options. Target architecture configuration loading: Loading the GPNPU target architecture typically involves specifying the architecture identifier in the compiler or model conversion tool. For example, when using frameworks like LLVM or Apache TVM, architecture support can be enabled via command-line parameters such as -target=GPNPU or similar instructions. If the GPNPU is based on the RISC-V instruction set, ensure the toolchain includes RISC-V NPU extension support and verify configuration details with the hardware vendor's architecture manual. O3 optimization option settings: Setting the O3 optimization level enables advanced compiler optimizations, including loop unrolling, function inlining, and vectorization. However, note that O3 may increase compilation time and affect debugging. Preferably, for NPU-specific optimizations, check the architecture documentation to see if O3 automatically triggers hardware acceleration instructions; otherwise, manually add target-specific modifiers.

[0044] Specifically, when encountering the aforementioned special compilation instructions, RISC-V does not perform complex task offloading operations. Instead, it writes the offset address of the task in gpnpu.po, such as 0x0040, to the GPNPU's command register and sets the "Start" bit. After detecting the "Start" bit, the GPNPU hardware directly prefetches the instruction at address 0x0040 from the instruction memory and executes it. During this period, the RISC-V processor continues to execute subsequent data prefetch instructions, achieving parallel computation and data handling. When the GPNPU completes the computation, it sets the "Done" register. RISC-V detects this bit flip, confirms the data validity, and continues to the next stage of processing.

[0045] like Figure 1The compilation and running method of the RISC-V and GPNPU collaborative architecture based on the TVM framework shown includes step S105, using the executable file, architecture instruction file and instruction file, to complete the data scheduling processing of the main central processing unit when the GPNPU performs data scheduling.

[0046] In one specific embodiment of this application, the main central processing unit loads the architecture instruction file into the storage area corresponding to the reduced instruction set architecture for instruction caching, and loads the instruction file into the storage area corresponding to the general neural network processing unit for instruction storage; and when the general neural network processing unit issues a data call instruction, the reduced instruction set architecture, based on the shared register flag relationship with the main central processing unit, sends the address of the corresponding instruction in the instruction file and the start signal to the storage area corresponding to the general neural network processing unit via the control bus to read the data.

[0047] Specifically, such as Figure 3 As shown, after loading the target file, a two-level scheduling system of Host-Offload and RISC-V-Dispatch is constructed through unified RISC-V scheduling. After the calculation is completed, the final result is output. Furthermore, the Host runs host.exe, which writes riscv.po and gpnpu.po to a designated reserved area in DDR via PCIe. After the RISC-V processor is reset, it reads riscv.po from DDR and begins execution.

[0048] The host's hardware initialization process, running host.exe, includes loading riscv.po into the RISC-V instruction cache via PCIe or AXI and loading gpnpu.po into the GPNPU instruction storage unit. The unified RISC-V scheduling process includes the RISC-V processor starting the scheduling module. For instructions calling RISC-V, execution is performed directly within the RISC-V core. For GPNPU call instructions, RISC-V sends the address of the corresponding instruction in gpnpu.po and a start signal via a dedicated control bus. This application achieves nanosecond-level synchronization and pipelined parallelism by establishing a lightweight handshake mechanism between RISC-V and GPNPU, i.e., shared register flags. After GPNPU completes its scheduling process, it sends a signal to RISC-V, which then reads the result from the corresponding storage location and continues execution.

[0049] This embodiment achieves efficient fine-grained unified scheduling and innovatively constructs a two-level scheduling system of Host-Offload and RISC-V-Dispatch, which pushes high-frequency scheduling authority down to the "near-data" RISC-V processor. Compared with the traditional Host scheduling mode, it significantly reduces communication latency, realizes pipelined parallelism of RISC-V scalar instructions and GPNPU vector instructions, and significantly improves hardware resource utilization.

[0050] like Figure 4 As shown, this application also provides a compilation and execution device for a RISC-V and GPNPU collaborative architecture based on the TVM framework, including: a parsing module 401, which uses an Open Neural Network Exchange Format (ONS) file to parse the operator type, tensor shape, and connection relationship, and converts the operator type, tensor shape, and connection relationship into TVM operator type, TVM tensor shape, and TVM connection relationship under the TVM architecture; an optimization module 402, which optimizes the operator cache address, data storage location, and instruction scheduling corresponding to the TVM operator type, TVM tensor shape, and TVM connection relationship to obtain the optimized TVM operator type, TVM tensor shape, and TVM connection relationship; and an intermediate code generation module. 403 is used to generate the main CPU project code, RISC-V code, and GPNPU code according to the optimized TVM operator type, TVM tensor shape, and TVM connection relationship; 404 is used to compile the main CPU project code, RISC-V code, and GPNPU code using the LLVM architecture to obtain the executable file, architecture instruction file, and instruction file corresponding to the main CPU, RISC-V, and GPNPU, respectively; 405 is used to utilize the executable file, architecture instruction file, and instruction file to complete the main CPU's data scheduling processing using RISC-V when the GPNPU performs data scheduling.

[0051] In one specific embodiment of this application, a computer device is also provided, including a memory, a processor, and a computer program stored in the memory. The processor executes the computer program to implement the steps in an embodiment of a compilation and execution method for a RISC-V and GPNPU co-architecture based on a TVM framework. Specific implementation methods can be found in the above method embodiments, and will not be repeated here.

[0052] This application also provides a computer-readable storage medium containing instructions on which a computer program is stored. For example, a memory containing instructions that can be executed by a processor of a computer device to perform the above-described method. When executed by the processor, the computer program can implement the steps in an embodiment of a compilation and execution method based on a TVM-based RISC-V and GPNPU co-architecture.

[0053] Those skilled in the art will understand that embodiments of this application can provide methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0054] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0055] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0056] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0057] The above are merely embodiments of this application and do not limit the scope of this patent application. Any equivalent structural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of this application.

Claims

1. A compilation and execution method for a RISC-V and GPNPU co-architecture based on the TVM framework, characterized in that, include: The operator type, tensor shape, and connection relationship are parsed using the Open Neural Network Exchange Format file, and the operator type, tensor shape, and connection relationship are converted into TVM operator type, TVM tensor shape, and TVM connection relationship under the TVM architecture. The TVM operator type, TVM tensor shape, and TVM connection relationship are optimized to obtain the optimized TVM operator type, TVM tensor shape, and TVM connection relationship. Based on the optimized TVM operator type, TVM tensor shape, and TVM connection relationship, the corresponding main central processing unit (CPU) project code, RISC-V code, and GPNPU code are generated. Using the LLVM architecture, the main CPU project code, the RISC-V code, and the GPNPU code are compiled to obtain the executable file, architecture instruction file, and instruction file corresponding to the main CPU, RISC-V, and GPNPU, respectively. Using the executable file, the architecture instruction file, and the instruction file, the RISC-V is used to complete the data scheduling process of the main central processing unit when the GPNPU performs data scheduling.

2. The compilation and execution method of the framework-based RISC-V and GPNPU co-architecture according to claim 1, characterized in that, The optimization of the TVM operator type, TVM tensor shape, and TVM connection relationship corresponding to the operator cache address, data storage location, and instruction scheduling, to obtain the optimized TVM operator type, TVM tensor shape, and TVM connection relationship, includes: A virtual memory mapping table is established to simulate the storage lifecycle of the static random access memory inside the GPNPU; and, based on the virtual memory mapping table, the execution order of the processing operators of the GPNPU is determined; and, For processing operators that are continuously executed by the GPNPU, the output buffer address of the currently executing operator is mapped to the input address of the next layer operator.

3. The compilation and execution method of the RISC-V and GPNPU co-architecture based on the TVM framework according to claim 1, characterized in that, The optimization of the TVM operator type, TVM tensor shape, and TVM connection relationship corresponding to the operator cache address, data storage location, and instruction scheduling, to obtain the optimized TVM operator type, TVM tensor shape, and TVM connection relationship, includes: Adjust the location in memory where the TVM tensor shape is stored according to the storage format of the TVM tensor shape; and, Allocate corresponding independent storage areas in static random access memory for the RISC-V and the GPNPU.

4. The compilation and execution method of the RISC-V and GPNPU co-architecture based on the TVM framework according to claim 1, characterized in that, The optimization of the TVM operator type, TVM tensor shape, and TVM connection relationship corresponding to the operator cache address, data storage location, and instruction scheduling, to obtain the optimized TVM operator type, TVM tensor shape, and TVM connection relationship, includes: Based on the working order of the processing operators of the GPNPU, a hardware allocation strategy for the corresponding hardware resources used to execute the computational work of the processing operators is determined; and, Based on the hardware allocation strategy, an optimized TVM operator type, TVM tensor shape, and TVM connection relationship are generated, which include the tags of the hardware allocation strategy.

5. The compilation and execution method of the RISC-V and GPNPU co-architecture based on the TVM framework according to claim 1, characterized in that, The step of generating the main CPU project code, RISC-V code, and GPNPU code based on the optimized TVM operator type, TVM tensor shape, and TVM connection relationships includes: The RISC-V code reserves a communication interface for communicating with the GPNPU; and by setting the GPNPU code, the GPNPU's dedicated operators are converted into functions that call the GPNPU driver interface.

6. The compilation and execution method of the RISC-V and GPNPU co-architecture based on the TVM framework according to claim 1, characterized in that, The method of utilizing the executable file, the architecture instruction file, and the instruction file to perform data scheduling on the GPNPU, and using the RISC-V architecture to complete the data scheduling process of the main CPU, includes: The main central processing unit loads the architecture instruction file into the memory area corresponding to the RISC-V for instruction caching, and loads the instruction file into the memory area corresponding to the general-purpose GPNPU for instruction storage; and, When the GPNPU issues a data retrieval instruction, the RISC-V, based on the shared register flag relationship with the main central processing unit, sends the address of the corresponding instruction in the instruction file and the start signal to the corresponding storage area of ​​the GPNPU via the control bus to read the data.

7. The compilation and execution method of the framework-based RISC-V and GPNPU co-architecture according to claim 2, characterized in that, The steps of establishing a virtual memory mapping table to simulate the storage lifecycle of the static random access memory inside the GPNPU, and determining the execution order of the processing operators of the GPNPU based on the virtual memory mapping table, include: When the storage space of the static random access memory is insufficient or the storage working life of the static random access memory ends, the calculation result of the processing operator is stored in the DDR memory.

8. A compilation and execution device for a RISC-V and GPNPU co-architecture based on the TVM framework, characterized in that, include: The parsing module uses the Open Neural Network Exchange Format file to parse out the operator type, tensor shape, and connection relationship, and converts the operator type, tensor shape, and connection relationship into TVM operator type, TVM tensor shape, and TVM connection relationship under the TVM architecture. An optimization module is used to optimize the operator cache address, data storage location, and instruction scheduling corresponding to the TVM operator type, the TVM tensor shape, and the TVM connection relationship, so as to obtain the optimized TVM operator type, TVM tensor shape, and TVM connection relationship. The intermediate code generation module is used to generate the main central processing unit (CPU) project code, RISC-V code, and GPNPU code according to the optimized TVM operator type, TVM tensor shape, and TVM connection relationship. The compilation module is used to compile the main CPU project code, the RISC-V code, and the GPNPU code using the LLVM architecture to obtain the executable file, architecture instruction file, and instruction file corresponding to the main CPU, RISC-V, and GPNPU, respectively. The scheduling execution module is used to utilize the executable file, the architecture instruction file, and the instruction file to complete the data scheduling processing of the main central processing unit using the RISC-V when the GPNPU performs data scheduling.

9. A computer device, comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the compilation and running method of the RISC-V and GPNPU co-architecture based on the TVM framework as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is loaded by the processor, it executes the processing steps of the compilation and running method of the RISC-V and GPNPU co-architecture based on the TVM framework as described in any one of claims 1 to 7.

Images