RISC-V computing and running method based on heterogeneous computing feature intermediate representation

Abstract

The application provides a RISC-V operation method and device based on a heterogeneous computing feature intermediate representation, the application proposes that the heterogeneous computing feature intermediate representation is constructed based on an MLIR framework, the architecture features of a RISC-V CIM chip are uniformly described from five dimensions of calculation, concurrency, communication, synchronization and data reuse, and on this basis, the abstract expression and feature description of different customized CIM chips and their extended instruction semantics are realized, so as to provide unified support for instruction analysis, optimization scheduling and target code generation of a compilation system, reduce the compilation adaptation cost of the CIM heterogeneous chip, and fully tap the hardware computing potential.

Description

Technical Field

[0001] This invention relates to the fields of computer compilation technology and heterogeneous computing chip technology, and particularly to a design method, apparatus, electronic device, computer-readable storage medium, and computer program product for a heterogeneous computing feature description intermediate representation for RISC-V in-memory extended instructions. Background Technology

[0002] With the explosive growth of artificial intelligence (AI) technology, the demands of AI algorithms on hardware computing power and energy efficiency continue to rise. Traditional von Neumann architecture chips suffer from a severe "memory wall" bottleneck due to the physical separation of computing and storage units, making data transfer overhead a major obstacle to performance improvement. Computing-in-Memory (CIM) chips, by integrating storage and computing units, can significantly reduce data transfer, becoming a key technological approach to overcoming this bottleneck. The RISC-V instruction set architecture, with its open-source nature and support for flexible instruction extensions, has become the mainstream control architecture for CIM chips.

[0003] The trend of deep customization in the field of AI has led to significant heterogeneity and fragmentation issues in existing RISC-V in-memory computing heterogeneous chips, resulting in poor software portability and high development costs.

[0004] On the one hand, in-memory computing chips are inherently heterogeneous. The chip contains acceleration components such as tensor cores for accelerating matrix operations and vector cores for accelerating vector calculations. Adapting to the computational characteristics of the RISC-V general-purpose core and heterogeneous acceleration components, controlling the computational flow of the RISC-V general-purpose core and heterogeneous acceleration components through synchronous instructions, and achieving task concurrency of multiple acceleration components are the challenges.

[0005] On the other hand, although different organizations design chips based on the RISC-V instruction set, their in-memory computing chips have very different architectural features, resulting in different designs for internal interconnects, memory access methods, etc., which leads to fragmentation and poses significant challenges to user programming and program optimization, becoming an international problem in the current field of artificial intelligence chips.

[0006] Existing mainstream AI programming frameworks and compilation systems, such as TVM and Triton, are mainly designed for general-purpose architectures such as CPUs and GPUs. Their intermediate representations mainly revolve around general-purpose parallel computing instructions and hardware characteristics, lacking specific support for RISC-V CIM extension instructions. They cannot accurately capture the semantic features of CIM extension instructions and the architectural details of heterogeneous hardware. Although the IREE compilation system provides support for RISC-V target architectures based on the Multi-Level Intermediate Representation (MLIR) framework, its intermediate representation is mainly oriented towards the RISC-V basic instruction set. It lacks sufficient ability to describe the characteristics of RISC-V CIM extension instructions and dedicated CIM units for AI acceleration design. The compilation system cannot fully understand the core semantics of CIM extension instructions and hardware adaptation requirements, making it difficult to leverage the computing power advantages of CIM chips.

[0007] In existing technologies, the AI ​​model compilation process typically adopts a degraded approach of "high-level model—general intermediate representation—target code generation," and its traditional compilation process is as follows: Figure 1 As shown, the current approach primarily targets CPU, GPU, or traditional NPU architectures, lacking the ability to express the characteristics of RISC-V in-memory computing heterogeneous architectures. During the model's progressive degradation process, data reuse relationships in tensor computation, on-chip storage hierarchies, synchronization control, and in-memory computing co-operation semantics are easily lost, making it difficult for the compiler to effectively map high-level AI semantics to RISC-V CIM extension instructions. Furthermore, existing LLVM backends lack support for CIM extension instructions, local memory management, and heterogeneous execution models, making it difficult to achieve automatic code generation and optimization for in-memory computing chips. Summary of the Invention

[0008] To address the problems in existing technologies, such as the lack of a unified feature description method for RISC-V CIM extended instructions, the difficulty in accurately depicting the semantics of extended instructions and the characteristics of heterogeneous hardware architectures through intermediate representations, leading to difficulties in compilation adaptation, low instruction mapping efficiency, and underutilization of CIM chip computing power, this invention proposes a heterogeneous computing feature description intermediate representation for RISC-V CIM chips (RISC-V CIM IR). This intermediate representation, built on the MLIR framework, provides a unified characterization of the architectural features of RISC-V CIM chips from five dimensions: computation, concurrency, communication, synchronization, and data reuse. Based on this, it achieves abstract expression and feature description of the semantics and extended instructions of different customized CIM chips, thereby providing unified support for instruction parsing, optimization scheduling, and target code generation in the compilation system, reducing the compilation adaptation cost of heterogeneous CIM chips, and fully tapping the potential of hardware computing power.

[0009] Specifically, addressing the shortcomings of existing technologies, such as Figure 5 As shown, this invention proposes a RISC-V in-memory computing chip operation method based on intermediate representation of heterogeneous computing features, including:

[0010] The initial step is to export the AI ​​model as a unified tensor computation representation file; the AI ​​model can be for image recognition, large language models, etc.

[0011] The conversion step involves converting the abstract value semantic tensor representation in the unified tensor computation representation file into a memory-level buffer representation with reference semantics, thus obtaining an intermediate computation representation file and realizing the conversion from a high-level abstract intermediate representation to a memory-aware intermediate representation.

[0012] The construction steps specify the data type and computation precision for the hardware-aware operations in the intermediate computation representation file to obtain the intermediate representation of heterogeneous computing features, so as to adapt to the hardware computing characteristics of the in-memory computing array of the in-memory computing chip.

[0013] The optimization steps involve compiling and optimizing the intermediate representation of the heterogeneous computing feature description based on five dimensions: computation, communication, concurrency, data reuse, and collaborative synchronization, to obtain the CIM feature description. This compilation and optimization includes: dividing the tensor computation in the intermediate representation of the heterogeneous computing feature description into multiple Tile instances according to the parallel architecture of the in-memory computing chip, and distributing them to multiple CIM engines for parallel execution; allocating private on-chip SRAM to each instance according to the storage hierarchy of the in-memory computing chip, and combining asynchronous DMA transfer to allow instance data to reside in on-chip storage during computation, achieving data reuse; defining the collaborative execution and synchronization semantics between the general-purpose core and the NPU acceleration core in the in-memory computing chip in the intermediate representation of the heterogeneous computing feature description; and merging adjacent computational operations into composite operators based on the data dependencies in the intermediate representation of the heterogeneous computing feature description to reduce intermediate result storage and off-chip memory access overhead.

[0014] The code generation steps involve converting the CIM feature description into a standard LLVM IR, where LLVM IR is a low-level platform-independent intermediate representation for the compiler backend; converting general computational operations in LLVM IR into RISC-V basic instructions for execution by the RISC-V general-purpose core; converting LLVM Intrinsic built-in functions in LLVM IR for in-memory computing extension operations into RISC-V CIM extension instructions for execution by the in-memory computing NPU acceleration core by extending the instruction selection, lowering, and machine code generation modules in the LLVM RISC-V backend; and generating the target AI model program based on the RISC-V basic instructions and RISC-V CIM extension instructions.

[0015] The aforementioned RISC-V in-memory computing chip operation method based on heterogeneous computing feature intermediate representation includes:

[0016] The deployment and execution steps involve cross-compiling and linking the target AI model program to generate an executable file, and then deploying it to a RISC-V heterogeneous chip platform or FPGA prototype verification platform with an in-memory computing chip to obtain the inference output results of the target AI model program. The inference output results can correspond to the AI ​​model as image recognition results, large language model response results, etc.

[0017] Among them, the RISC-V general-purpose processor core is used for program control, task scheduling and data transfer control, while the in-memory computing NPU acceleration core is used to perform matrix multiplication, convolution calculation and vector calculation tasks.

[0018] After the model is executed, the output results of the target platform are numerically compared with the output results of the PyTorch Golden Reference Model to verify the correctness of computational semantics, data flow semantics and target code generation during the conversion of the high-level AI model into the RISC-V in-memory computing target program, and to evaluate the execution performance of the generated code on the target platform.

[0019] The aforementioned RISC-V in-memory computing chip operation method based on heterogeneous computing feature intermediate representation includes: a basic module, a data loading and storage module, a memory management module, a shape transformation and type conversion module, a computing module, and a communication and collaboration module;

[0020] This basic module operates on function definitions, return control, and constant references, ensuring the integrity of function logic and the legality of data references;

[0021] This data loading and storage module operates to facilitate data interaction between main memory and on-chip SRAM, and between the CIM computing unit and SRAM. It adapts to the data access requirements of the in-memory computing architecture, clearly defines memory addresses, indexes, and data types, and ensures the correctness of data reading and writing.

[0022] This memory management module is used to allocate and release memory, adapt to the hierarchical storage requirements of data under the in-memory computing architecture, distinguish memory types through memory space parameters, and improve the utilization of storage resources.

[0023] This shape transformation and type conversion module is used for adjusting the dimensions and converting the precision of tensor data to adapt to the data layout requirements of attention mechanisms, convolution calculations and matrix operations.

[0024] This computing module is used for the abstract mapping of RISC-V standard instructions, vector operations and in-memory computing extended operations. It supports element-level calculations of basic arithmetic, logic and comparison operations, directly maps to the underlying hardware instructions, and covers matrix operations, convolution calculations and heterogeneous collaborative computing scenarios.

[0025] This communication and coordination module is used for synchronous control between the RISC-V main processor and the in-memory computing accelerator, accelerates core startup, ensures that the collaborative execution of multiple computing units is free of data conflicts, and adapts to the parallel scheduling requirements under the heterogeneous in-memory computing architecture.

[0026] like Figure 6 As shown, this invention also proposes a RISC-V in-memory computing chip operating device B based on the intermediate representation of heterogeneous computing features, which includes:

[0027] The initial module exports the AI ​​model as a unified tensor computation representation file;

[0028] The conversion module converts the tensor representation of abstract value semantics in the unified tensor computation representation file into a memory-level buffer representation with reference semantics, thus obtaining an intermediate computation representation file and realizing the conversion from high-level abstract intermediate representation to memory-aware intermediate representation.

[0029] The module specifies the data type and computation precision for the hardware-aware operations in the intermediate computation representation file, and obtains the intermediate representation of heterogeneous computing features to adapt to the hardware computing characteristics of the in-memory computing array of the in-memory computing chip.

[0030] The optimization module, based on five dimensions including computation, communication, concurrency, data reuse, and collaborative synchronization, compiles and optimizes the intermediate representation of the heterogeneous computing feature description to obtain the CIM feature description. This compilation and optimization includes: dividing the tensor computation in the intermediate representation of the heterogeneous computing feature description into multiple Tile instances according to the parallel architecture of the in-memory computing chip, and distributing them to multiple CIM engines for parallel execution; allocating private on-chip SRAM to each instance according to the storage hierarchy of the in-memory computing chip, and combining asynchronous DMA transfer to allow instance data to reside in on-chip storage during computation, achieving data reuse; defining the collaborative execution and synchronization semantics between the general core and the NPU acceleration core in the in-memory computing chip in the intermediate representation of the heterogeneous computing feature description; and merging adjacent computational operations into composite operators based on the data dependencies in the intermediate representation of the heterogeneous computing feature description to reduce intermediate result storage and off-chip memory access overhead.

[0031] The code generation module converts the CIM feature description into standard LLVM IR, where LLVM IR is a low-level platform-independent intermediate representation for the compiler backend; it converts general computational operations in LLVM IR into RISC-V basic instructions for execution by the RISC-V general-purpose core; it converts LLVM Intrinsic built-in functions in LLVM IR for in-memory computing extension operations into RISC-V CIM extension instructions for execution by the in-memory computing NPU acceleration core by extending the instruction selection, lowering, and machine code generation modules in the LLVM RISC-V backend; and it generates the target AI model program based on the RISC-V basic instructions and RISC-V CIM extension instructions.

[0032] The aforementioned RISC-V in-memory computing chip operating device based on heterogeneous computing feature intermediate representation includes:

[0033] The deployment and execution module generates an executable file from the target AI model program through cross-compilation and linking, and deploys it to a RISC-V heterogeneous chip platform or FPGA prototype verification platform with an in-memory computing chip to obtain the inference output results of the target AI model program.

[0034] Among them, the RISC-V general-purpose processor core is used for program control, task scheduling and data transfer control, while the in-memory computing NPU acceleration core is used to perform matrix multiplication, convolution calculation and vector calculation tasks.

[0035] After the model is executed, the output results of the target platform are numerically compared with the output results of the PyTorch Golden Reference Model to verify the correctness of computational semantics, data flow semantics and target code generation during the conversion of the high-level AI model into the RISC-V in-memory computing target program, and to evaluate the execution performance of the generated code on the target platform.

[0036] The RISC-V in-memory computing chip operating device based on the intermediate representation of heterogeneous computing features includes: a basic module, a data loading and storage module, a memory management module, a shape transformation and type conversion module, a computing module, and a communication and collaboration module.

[0037] This basic module operates on function definitions, return control, and constant references, ensuring the integrity of function logic and the legality of data references;

[0038] This data loading and storage module operates to facilitate data interaction between main memory and on-chip SRAM, and between the CIM computing unit and SRAM. It adapts to the data access requirements of the in-memory computing architecture, clearly defines memory addresses, indexes, and data types, and ensures the correctness of data reading and writing.

[0039] This memory management module is used to allocate and release memory, adapt to the hierarchical storage requirements of data under the in-memory computing architecture, distinguish memory types through memory space parameters, and improve the utilization of storage resources.

[0040] This shape transformation and type conversion module is used for adjusting the dimensions and converting the precision of tensor data to adapt to the data layout requirements of attention mechanisms, convolution calculations and matrix operations.

[0041] This computing module is used for the abstract mapping of RISC-V standard instructions, vector operations and in-memory computing extended operations. It supports element-level calculations of basic arithmetic, logic and comparison operations, directly maps to the underlying hardware instructions, and covers matrix operations, convolution calculations and heterogeneous collaborative computing scenarios.

[0042] This communication and coordination module is used for synchronous control between the RISC-V main processor and the in-memory computing accelerator, accelerates core startup, ensures that the collaborative execution of multiple computing units is free of data conflicts, and adapts to the parallel scheduling requirements under the heterogeneous in-memory computing architecture.

[0043] The present invention also proposes a client for implementing any of the RISC-V in-memory computing chip operating devices based on heterogeneous computing feature intermediate representation.

[0044] The present invention also proposes an electronic device, including the RISC-V in-memory computing chip operating device based on the intermediate representation of heterogeneous computing features. The electronic device may be connected to an information display device, which is used to display the target AI model program with user-set display parameters, attributes or through an artificial intelligence model.

[0045] The present invention also proposes a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the RISC-V in-memory computing chip operation method based on the intermediate representation of heterogeneous computing features.

[0046] The present invention also proposes a computer program product, including a computer program, wherein when the computer program is executed by a processor, it implements the steps of the RISC-V in-memory computing chip operation method based on the intermediate representation of heterogeneous computing features.

[0047] As can be seen from the above solutions, the advantages of the present invention are:

[0048] Precise Feature Description: This invention targets the architectural characteristics of RISC-V CIM chips and systematically characterizes heterogeneous computing features from five dimensions: computation, concurrency, communication, synchronization, and data reuse. It can simultaneously reflect the hardware architecture characteristics and their corresponding instruction semantic relationships, thereby achieving accurate modeling of CIM computing behavior and solving the problems of insufficient characterization and incomplete semantic expression of existing intermediate representations of CIM chips.

[0049] High adaptability: Through a unified abstraction and description mechanism oriented towards architectural features, it achieves unified modeling and representation of RISC-V CIM chips customized by different organizations, effectively shielding the differences between different chips in extended instruction formats, functional implementations, and hardware organization structures. Adaptation to multiple CIM chips can be completed without modifying the intermediate representation core design, significantly reducing compilation adaptation and application migration costs.

[0050] High instruction mapping efficiency: This invention establishes a direct link between computational semantics and hardware execution characteristics in the IR, enabling the compilation system to efficiently map CIM accelerated operations to corresponding RISC-V extended instructions based on the architectural information and computational characteristics depicted in the IR. This reduces redundant conversion and instruction combination overhead, thereby improving instruction generation efficiency and fully leveraging the computing power and energy efficiency advantages of the CIM chip.

[0051] Support for compilation optimization: The characterization of key architectural features of the CIM chip in IR provides a unified and accurate hardware semantic foundation for compilation optimization. It can directly support the implementation of optimization strategies such as operator fusion, block optimization, data reuse and parallel scheduling, reduce intermediate data storage and memory access times, reduce data transfer overhead, and improve the overall execution efficiency of AI models on the CIM chip. Attached Figure Description

[0052] Figure 1 Flowchart for traditional downgrade of existing technologies;

[0053] Figure 2 Flowchart for building and compiling applications for RISC-V CIM IR;

[0054] Figure 3 Flowchart for generating target code for RISC-V CIM IR;

[0055] Figure 4 This is a diagram of the RISC-V CIM heterogeneous computing chip architecture.

[0056] Figure 5 This is a flowchart of the method of the present invention;

[0057] Figure 6 This is a block diagram of the device of the present invention;

[0058] Figure 7 This is a schematic diagram of the structure of the first electronic device of the present invention;

[0059] Figure 8 This is a schematic diagram of the application environment structure of the first electronic device of the present invention;

[0060] Figure 9 This is a schematic diagram of the structure of the second electronic device of the present invention.

[0061] Figure label:

[0062] A - First electronic device;

[0063] B-RISC-V in-memory computing chip operating device based on heterogeneous computing feature intermediate representation;

[0064] C-Data acquisition equipment;

[0065] D-Information display device;

[0066] 1000 - Second electronic device;

[0067] Ⅰ-Computational Unit;

[0068] II-ROM;

[0069] III-RAM;

[0070] N-bus;

[0071] V-Interface;

[0072] VI - Input Unit;

[0073] VII - Output Unit;

[0074] VIII - Storage medium;

[0075] IX - Communication Unit. Detailed Implementation

[0076] It should be noted that, in this application, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus.

[0077] In the absence of further restrictions, an element defined by the phrase "comprising a..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0078] The processor described in this invention is the control center of an electronic device. It can be a single processor or a collective term for multiple processing elements. For example, it can be one or more central processing units (CPUs), application-specific integrated circuits (ASICs), or one or more integrated circuits configured to implement embodiments of this invention, such as one or more digital signal processors (DSPs), or one or more field-programmable gate arrays (FPGAs).

[0079] Alternatively, the processor can perform various functions of the electronic device by running or executing software programs stored in memory and by calling data stored in memory.

[0080] In a specific implementation, as one example, the processor may include one or more CPUs. Each of these processors may be a single-core processor or a multi-core processor. Here, "processor" can refer to one or more devices, circuits, and / or processing cores for processing data (e.g., computer program instructions). Electronic devices may include servers, desktop computers, laptops, smartphones, tablets, embedded computers, etc., where the embedded computer includes vehicles and robots, etc.

[0081] The memory is used to store the software program that executes the solution of the present invention, and the execution is controlled by the processor. For specific implementation methods, please refer to the above method embodiments, which will not be repeated here.

[0082] It should be noted that the structure of the electronic device shown in the accompanying drawings of this invention does not constitute a limitation thereof. The actual knowledge structure recognition device may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0083] The above embodiments can be implemented, in whole or in part, by software, hardware (such as circuits), firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. A semiconductor medium can be a solid-state drive.

[0084] It should also be understood that the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. A and B can be singular or plural. Additionally, the character " / " in this article generally indicates an "or" relationship between the preceding and following related objects, but it can also represent an "and / or" relationship. Please refer to the context for a more accurate understanding.

[0085] In this invention, "at least one" means one or more, and "more than one" means two or more. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of a single item or a plurality of items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be a single item or multiple items.

[0086] It should also be understood that, in various embodiments of the present invention, the order of the above-mentioned processes does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0087] In the several embodiments provided by this invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0088] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0089] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0090] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0091] Existing AI models and intermediate representations fail to simultaneously depict key features such as computation patterns, storage levels, parallel execution, and synchronization relationships. They lack a unified syntax structure and operational semantics to carry the information of in-memory computing extended instructions, resulting in problems such as semantic loss and inefficient instruction mapping during the compilation process.

[0092] This invention proposes a method for constructing a dedicated intermediate representation based on the MLIR framework. It abstracts the characteristics of the CIM chip architecture from five dimensions: computational expression, concurrent execution, data communication, synchronization control, and data reuse. Furthermore, it constructs a dual-path instruction generation mechanism to achieve the separation and mapping of general-purpose computing and in-memory accelerated computing.

[0093] The technical solution proposed in this invention is as follows:

[0094] The RISC-V CIM IR of this invention is designed based on the MLIR framework. As the core abstraction carrier connecting the tensor representation of high-level AI models and the RISC-V CIM chip, it is constructed using a dedicated dialect to achieve unified modeling and accurate expression of the CIM chip architecture features and its extended instruction semantics. Its core design and implementation are as follows:

[0095] The RISC-V CIM IR supports multiple data types, including INT4, INT8, INT16, FP8, FP16, BF16, and FP32, to adapt to the needs of CIM chips in AI computing scenarios with different precision. Based on this, the computational and execution characteristics of the RISC-V CIM heterogeneous chip are characterized from the following five core dimensions, and a unified expression of extended instruction semantics is achieved within this feature description framework:

[0096] 1. Computational Representation Dimension: This dimension describes the core computational behaviors supported by the CIM chip, including tensor operations (matrix multiplication, matrix-vector multiplication), vector operations (vector-vector, vector-scalar operations, and reduction operations), etc. Through abstract modeling of computational modes and operational logic, it achieves a unified expression of the computational semantics of different CIM extended instructions, supports differentiated descriptions of general-purpose computation and CIM accelerated computation, and provides a semantic foundation for subsequent instruction mapping.

[0097] 2. Concurrent Execution Dimension: This dimension characterizes the parallel architecture features of the RISC-V CIM chip, including the parallel execution capabilities of heterogeneous computing resources such as CIM computing units, tensor computing cores, and vector computing cores. It describes the semantics of concurrent startup and independent execution of multiple computing units, adapts to the differences in the number and organization of computing resources of different chips, and provides support for parallel scheduling in compilation optimization.

[0098] 3. Data Communication Dimension: This dimension describes the multi-layered storage structure in the CIM chip, including the data transfer semantics between on-chip SRAM and external main memory, and between CIM computing units and on-chip SRAM. Under the RISC-V CIM IR framework, it uniformly expresses extended instruction semantics such as data loading (TLD) and data storage (TSD), and clarifies data communication paths and behaviors.

[0099] 4. Synchronization Dimension: This dimension defines the collaborative execution and synchronization semantics between the RISC-V general-purpose core and the CIM acceleration unit. It describes the processes of task assignment, execution waiting, and result synchronization, and corresponds to the synchronization control mechanism in the extended instructions to ensure the correctness of the execution order and data consistency between heterogeneous computing units.

[0100] 5. Data Reuse Dimension: Combining the storage hierarchy of the CIM chip, the data residency and reuse characteristics in the on-chip SRAM and internal storage of the CIM unit are characterized, supporting explicit expression of high-frequency access data, providing hardware feature basis for block optimization, data reuse and memory access optimization strategies, thereby reducing data transfer overhead and memory access energy consumption.

[0101] The RISC-V CIM IR construction process and application flow are adapted to the layered architecture of the compilation system, as detailed below:

[0102] The front end converts the high-level AI model into an MLIR tensor-level intermediate representation (Linalg-on-Tensors Dialect), and maps it to the RISC-V CIM IR of this invention through a dedicated conversion mechanism, realizing the conversion from general tensor computation semantics to CIM chip architecture feature description; this IR, as the core carrier of compilation optimization and instruction mapping, provides unified semantic and hardware feature support for operator fusion, block optimization and task scheduling.

[0103] During the backend instruction generation stage, based on the CIM chip architecture features and extended instruction semantics depicted in the IR, IR operations are mapped to target hardware instructions: general computing operations are converted into RISC-V basic instructions, and CIM accelerated computing operations are mapped to corresponding RISC-V CIM extended instructions according to their semantic features, thereby achieving efficient adaptation to different customized CIM extended instruction systems.

[0104] To make the above-mentioned features and effects of the present invention clearer and easier to understand, specific embodiments are described below in conjunction with the accompanying drawings. This specification discloses one or more embodiments incorporating the features of the present invention. The disclosed embodiments are merely illustrative. The scope of protection of the present invention is not limited to the disclosed embodiments, but is defined by the appended claims.

[0105] This invention constructs a complete compilation and application chain around the RISC-V CIM chip, and its overall process is as follows: Figure 2As shown in the diagram. This process is based on the MLIR multi-level intermediate representation system. It converts high-level AI models into tensor computation semantics that do not reflect the hardware architecture through the Torch-MLIR toolchain. Then, it maps these semantics to the underlying hardware execution code level by level through MLIR and RISC-V CIM, achieving lossless semantic transfer from model representation to target instruction generation. The overall architecture is divided into three layers: a front-end abstraction layer, a mid-end adaptation layer, and a back-end code generation layer.

[0106] At the front-end abstraction layer, PyTorch is used as the user modeling interface to convert AI models into TorchScript IR, and further reduce them to the MLIR tensor-level intermediate representation Linalg-on-Tensors Dialect, thus forming a hardware-independent tensor computation representation. This stage mainly completes the standardized expression of model computation semantics, providing a unified input for subsequent mapping to the CIM architecture.

[0107] The mid-level adaptation layer introduces the RISC-V CIM IR proposed in this invention, serving as the core abstract carrier connecting high-level tensor computation and the underlying CIM chip. This mid-level representation is built on the MLIR framework, precisely characterizing the architectural features of the RISC-V CIM chip from five dimensions: computation, concurrency, communication, synchronization, and data reuse. Under this unified feature description framework, it achieves the abstract expression and unified modeling of the extended instruction semantics of the CIM chip, enabling high-level tensor computation to naturally map to heterogeneous hardware execution semantics, while providing structured support for compilation optimization and task scheduling.

[0108] The backend code generation layer executes a multi-path instruction mapping strategy based on the architectural features and computational semantics depicted in the RISC-V CIM IR. On one hand, it progressively reduces general computational operations to MLIR standard dialects (such as Affine, Arith, and MemRef) and generates RISC-V basic instructions via LLVM IR. On the other hand, it maps CIM accelerated computational operations to LLVMIntrinsic functions and generates machine code for CIM extended instructions by extending modules such as instruction selection, Lowering, and TableGen description in the LLVM RISC-V backend, ultimately generating an executable program. This multi-path mapping mechanism is as follows: Figure 2 As shown.

[0109] LLVM Intrinsics are an internal function mechanism within the LLVM compiler framework, specially identified and processed by the compiler. They are used to describe low-level hardware semantics and special operations that are difficult to express in ordinary functions, such as vector computation, memory operations, atomic operations, and processor extension instructions. During compilation, Intrinsics are not executed as ordinary function calls but are further mapped to machine instructions or optimization operations corresponding to the target platform.

[0110] Based on the aforementioned RISC-V CIM IR overall framework, in order to achieve unified modeling and accurate expression of CIM chip architecture features, this invention further systematically designs the syntax structure, data type system, and operation set of the intermediate representation, enabling it to carry computational semantics and hardware feature information in a structured manner within the MLIR framework.

[0111] Specifically, in terms of syntax structure, the RISC-V CIM IR of this invention adopts a modular organization, with Module as the top-level container. Each module is identified by a globally unique symbol and internally encapsulates functions, constants, and global memory objects, forming an independent compilation unit. Module definitions follow the syntax specifications below:

[0112] symbol_name := `@` identifier

[0113] func.module @symbol_name {

[0114] <items>*

[0115] }

[0116] Here, `symbol_name` is a globally unique identifier for the module, and `items` can contain function definitions, constant declarations, global memory objects, etc. For example, a module containing ResNet18 convolution operators can be defined as `func.module @resnet18_conv { ...}`, ensuring that the computational logic within the module is decoupled from external resources.

[0117] Functions (Kernels) are the core carriers of RISC-V CIM IR computation logic. They are divided into two categories: general functions and in-memory acceleration functions. Both are defined using the func keyword and require explicit function signatures (parameter types, return types) and function bodies (operation sequences).

[0118] ssa_name := `%` identifier

[0119] function_signature ::=<function_parameter> *

[0120] function_parameter ::=<ssa_name> `:` <type>

[0121] <kernel_definition> ::= `func` @kernel_name `(`<function_signature> `)` `->`<function_type> {

[0122] <kernel_body>

[0123] }

[0124] The function body consists of a series of operations, all of which are represented using SSA (Static Single Assignment), allowing data dependencies to be explicitly expressed and tracked during compilation. For example, the form "%0 = riscv.matmul %a, %b :memref<16x16xfp16> -> memref<16x16xfp16>" clearly describes the tensor computation process and result dependencies, thus providing a foundation for subsequent compilation optimization.

[0125] Regarding the data type system, this invention constructs a unified scalar and tensor type system. It supports multiple precisions, including INT4, INT8, INT16, FP8, FP16, BF16, and FP32, to adapt to AI computing scenarios with varying precision. Tensor data is uniformly represented using the memref type, describing multidimensional data structures and their storage layout through the form "shape × element_type". For example, memref<16x16xfp16> represents a 16×16 half-precision floating-point tensor. This type system not only characterizes data precision but also expresses storage structure information, corresponding to the on-chip storage and array structure of the CIM chip.

[0126] The RISC-V CIM IR designed in this invention is constructed using the MLIR-specific dialect and combines a unified modeling approach for control flow and data flow, enabling the simultaneous expression of program logic control relationships and parallel computing relationships. Based on this, the architectural characteristics of the RISC-V CIM chip are systematically characterized from the following five core dimensions:

[0127] In the computational representation dimension, operations such as `riscv.matmul`, `riscv.batch_matmul`, and `riscv.conv2d` are defined to provide a unified abstract model for tensor and vector computations, enabling the expression of CIM computational patterns. Simultaneously, general computational operations such as `riscv.add`, `riscv.mul`, and `riscv.sub` are defined to express basic arithmetic and logical operations. In this dimension, RISC-VCIM IR not only describes computational logic but also establishes a mapping between computational semantics and hardware execution characteristics, allowing general-purpose computation and CIM-accelerated computation to achieve differentiated expressions at the same level. In the specific implementation, logic control operations are mapped to the CPU execution path, while intensive computational operations are mapped to LLVM Intrinsic functions, and corresponding CIM extended instructions are further generated in the backend, thus avoiding the performance loss caused by combining general instructions.

[0128] In terms of concurrent execution, the SPMD (Single Program Multiple Data) parallel execution model is constructed using `riscv.launch async tile (%tx, %ty)`. This divides the computational task into multiple tile instances, each with its own independent index domain and data view in the RISC-V CIM heterogeneous architecture, characterizing a "single program, multiple data blocks" execution approach. Compared to SIMD (Single Instruction Multiple Data) and SIMT (Single Instruction Multiple Threads) models, SPMD better aligns with the execution mode in the RISC-V CIM heterogeneous architecture, where the CPU handles control and CIM units handle computation. The RISC-V CIM IR, through explicit descriptions of the parallel domain, data partitioning, and execution boundaries, enables the compiler to automatically map tiles to different CIM computational units and generate corresponding concurrent scheduling and synchronization control logic. Furthermore, this parallel semantics can be further mapped to the MLIR's `scf.parallel` structure, achieving a smooth transition from abstract parallelism to concrete execution mechanisms.

[0129] In terms of data communication, via riscv.load memref<?x?xi32, memspace> Operations such as `riscv.store` provide a unified description of data transfer between on-chip SRAM and external main memory, and between the CIM computing unit and on-chip SRAM, with the data transfer path explicitly defined by the `memspace` address space parameter. In this dimension, data loading and storage semantics correspond to extended instructions such as TLDs and TSDs in the hardware, thereby achieving unified modeling of data communication behavior.

[0130] In terms of synchronization, an explicit dependency graph based on async tokens is constructed using `riscv.launch` and `riscv.wait_all async [%dependency1, %dependency2]`. `riscv.launch` initiates asynchronous computations or transfers to the CIM computing unit and generates synchronization tokens, while `riscv.wait_all` waits for all dependencies to complete. Based on the SSA (Synchronous Execution Architecture) dependency expression, necessary synchronization operations are inserted only before the results are actually used, thus achieving fine-grained and minimal synchronization control. This mechanism ensures the correctness of the execution order and data consistency between the CPU and CIM units, while avoiding unnecessary blocking and improving overall execution efficiency.

[0131] In terms of data reuse, by managing data blocks at the tile level and modeling the storage hierarchy, frequent data movement between off-chip and on-chip storage is reduced, significantly lowering memory access overhead and improving computational efficiency. The `riscv.alloc` and `riscv.dealloc` functions are designed to explicitly manage the lifecycle of data in on-chip SRAM. `riscv.alloc` allocates on-chip SRAM for tile data, while `riscv.dealloc` controls the end of the data lifecycle. These functions enable data to persist and be reused across multiple computational operations after loading. Operations such as `riscv.reshape` and `riscv.transpose` are combined to adjust the data layout to adapt to the CIM array computation mode, reducing data movement overhead. Data precision conversion is achieved through `riscv.quantization` and `riscv.dequantization` operations to adapt to low-precision computational needs and reduce energy consumption.

[0132] Furthermore, in terms of program structure and basic support, this invention implements function definition, return control and constant expression through operations such as riscv.func, riscv.return and riscv.constant, ensuring that IR has complete program semantic expression capabilities; through the coordinated cooperation of the above basic operations with computation, communication and synchronization operations, a complete intermediate representation operation system is formed.

[0133] Building upon the aforementioned unified intermediate representation, this invention further designs a dual-path instruction mapping mechanism: for general computational operations, it converts them to the MLIR standard dialect via `-convert-riscv-to-affine` and gradually reduces them to RISC-V basic instructions; for CIM-accelerated computational operations, it maps them to LLVM Intrinsic functions (such as `@llvm.riscv.matmul`, `@llvm.riscv.conv.drv`, etc.) via `-convert-riscv-to-cim`, and generates corresponding CIM extended instructions through an extended LLVM backend. Instruction encoding rules are defined using TableGen to achieve efficient mapping from IR semantics to hardware instructions.

[0134] Combination Figure 3 The RISC-V CIM heterogeneous chip architecture shown in this invention utilizes a unified intermediate representation layer modeling of key chip structural characteristics, starting from the organization of computation execution units, memory access patterns, and data flow characteristics. This method not only achieves accurate characterization of the CIM chip architecture but also provides a unified and scalable technical foundation for compilation optimization and efficient instruction generation, thereby significantly improving the performance and energy efficiency of RISC-V CIM chips in AI computing scenarios.

[0135] The RISC-V CIM IR in this invention is specifically designed into a basic module, a data loading and storage module, a memory management module, a shape transformation and type conversion module, a computing module, and a communication and collaboration module. Its operation design is detailed in Tables 1-7.

[0136] Table 1 shows the basic module operations, which mainly implement function definition, return control, and constant reference. They are the foundational support for IR programs, ensuring the integrity of function logic and the legality of data references.

[0137] Table 1: Basic Module Operations Operation Name Function Description riscv.func(riscv::FuncOp) Function: Defines a function containing a single SSA control flow graph (SSA CFG) region to encapsulate complete computational logic; supports specifying function name, parameter list, and return type, and is the core carrier for performing computational tasks. Example: riscv.func @add(%a:i32, %b: i32) -> (i32) {%0 = riscv.add % a, % b : i32riscv.return %0 : i32 / / Returns the computation result} riscv.return(riscv::ReturnOp) Function: Function return operation, used to end function execution and return a result (if any); it must match the return type defined in the function definition. Operands can be omitted if there is no return value, ensuring the normal termination of the function execution flow. Syntax: operation ::= `riscv.return` attr-dict ($operands^ `:` type ($operands))? Example: riscv.returnriscv.func @foo() -> (i32, f8) {return %0, %1 : i32, f8} riscv.constant(riscv::ConstantOp) Function: Generates constant values, supporting the referencing of function symbols or direct specification of constant data; commonly used to define fixed parameters, function pointers, etc., ensuring that values ​​determinable during the compilation phase can be efficiently referenced and optimized. Syntax: operation ::= `riscv.constant` attr-dict $value `:` type(results) Example: %3 = riscv.constant10 : i32%2 = riscv.constant @myfn : (memref <16xf32>, f32) -> memref <16xf32>%3 = "riscv.constant"() { value = @myfn} : () -> ((memref <16xf32>, f32)-> memref <16xf32>)

[0138] The data loading and storage module operations in Table 2 enable data interaction between main memory and on-chip SRAM, and between the CIM computing unit and SRAM. This adapts to the data access requirements of the in-memory computing architecture. Memory addresses, indexes, and data types must be clearly defined to ensure the correctness and efficiency of data reading and writing.

[0139] Table 2: Data Loading and Storage Module Operation Operation Name Function Description riscv.load(riscv::LoadOp) Function: Data loading operation, enabling data interaction between main memory and on-chip SRAM, and between the CIM computing unit and SRAM. The destination address for data transfer is specified by the memref address space parameter memspace. Syntax: operation ::= `riscv.load` $memref `[` $indices `]` attr-dict `:` type($memref) Example: %0 =riscv.load %A[%a, %b] : memref<8x?xi32, #layout, memspace0> riscv.store (riscv::StoreOp) Function: Data storage operation, writes data to the target address of a specified memory location (memref type); supports multi-dimensional memory indexing, requiring matching of memory dimensions and data types, commonly used for storing computational results, intermediate feature maps, etc. Syntax: operation ::= `riscv.store` $memref `[` $indices `]` attr-dict `:`type($memref) Example: %0 = riscv.store %A[%a, %b] : memref<8x?xi32, #layout,memspace0>

[0140] Table 3 shows the memory management module operations for memory allocation and release, adapting to the tiered storage requirements of LocalMemory and Global Memory in an in-memory computing architecture. Local Memory stores data that requires frequent access during the current computation phase; Global Memory stores single data sets, complete model parameters, and data from activated data that cannot fully reside in Local Memory during the current computation phase. The system differentiates between different storage types through memory space parameters, achieving tiered data management and improved storage resource utilization.

[0141] Table 3: Memory Management Module Operation Name Function Description riscv.alloc Function: Memory allocation, via the memref address space parameter (#memory_space) <0> for GlobalMemory, #memory_space <1> For Local Memory, frequently accessed weight data is allocated to Local Memory, while intermediate feature maps are stored in Global Memory. An example of efficient storage resource management is shown below: %weight_mem = riscv.alloc :memref<256x256xfp16, LMEM> %feat_mem = riscv.alloc :memref<256x256xfp16, GMEM> riscv.dealloc Function: Used to reclaim memory objects allocated by riscv.alloc to prevent memory leaks. This operation determines when the data's lifecycle ends; a memory object (memref type) to be freed must be specified, and the memory object cannot be accessed after being freed; Example: riscv.dealloc %weight_mem : memref<256x256xfp16, LMEM> riscv.dealloc %feat_mem:memref<256x256xfp16, GMEM>

[0142] The shape transformation and type conversion modules in Table 4 enable the adjustment of tensor data dimensions and the conversion of data precision, adapting to the data form requirements of scenarios such as attention mechanisms and convolution calculations under the in-memory computing architecture, reducing data transfer overhead and improving computational efficiency.

[0143] Table 4: Shape Transformation and Type Conversion Module Operation Name Function Description riscv.transpose(riscv::TransposeOp) Functionality: Dynamic matrix transpose operation, supports dimension reconstruction, integrates a hardware-level transpose module, completes key matrix transpose during the data loading phase, avoids redundant off-chip memory transfer, and reduces memory access latency for attention mechanism operators. Supports specifying dimension swapping rules through permutation parameters to meet the tensor form requirements of different computing scenarios. Syntax: Operation::= `riscv.transpose` $memref `,` $permutation attr-dict `:` type($memref)`to` type($result) Example: %1 = "riscv.transpose"(%0) {transp=[1,0]} : (memref <2x3xf64>) -> (memref <3x2xf64>) The `riscv.reshape(riscv::ReshapeOp)` method performs a matrix shape transformation. Functionality: Matrix shape transformation operation used to adjust the dimensional combination of tensors (e.g., reshaping a 2x4x8 tensor into an 8x8 tensor) without changing the total number of elements in the tensor; adapts to the parallel computing dimensionality requirements of in-memory compute arrays, adjusting the tensor shape to match the parallelism of the in-memory compute array, improving hardware resource utilization; supports specifying the target dimension through shape parameters, which must satisfy the constraint that "the product of all dimensions equals the total number of elements in the input tensor". Syntax: Operation ::= `riscv.reshape` $source `(` $shape `)` attr-dict `:`functional-type(operands, result) Example: %dest = "riscv.reshape"(%src, %shape): (memref <2x3xf32>, memref <2xi32>) -> memref <3x2xf32> riscv.quantization(riscv::QuantizationOp) Functionality: Tensor quantization operation, converting high-precision tensor data (e.g., FP32) into low-precision data (e.g., INT4 / INT8), adapting to the low-precision computing needs of in-memory computing architectures, reducing storage overhead and computing power consumption; the quantization process follows the "linear quantization" rule, mapping the original data to a finite number of quantization levels (NoOfLevels) by calculating the quantization step size, and rounding the mapping result; supports specifying the quantization range (max / min) and the number of quantization levels (NoOfLevels) to meet scenarios with different precision requirements. Example: %quant = riscv.quantization %orig, 5, -5, 16 :memref<5xf32> -> memref <5xi4> riscv.dequantization(riscv::DequantizationOp) Function: Tensor dequantization operation, restoring low-precision quantized data (INT4 / INT8) to high-precision data (e.g., FP32), for result restoration after quantization or subsequent high-precision processing; the dequantization process is reversible with the quantization process, mapping the quantization back to the original data range through the quantization step size (step) and quantization range (max / min); it must be consistent with the quantization parameters (max / min / NoOfLevels) of the riscv.quantization operation to ensure the accuracy of the original data. Example: %recovered =riscv.dequantization %quant, 5, -5, 16 : memref <5xi4> -> memref <5xf32>

[0144] Tables 5 and 6 define two categories of core operations based on the computational characteristics of in-memory computing chips: general-purpose computing and operations specific to the in-memory computing architecture. General-purpose computing operations implement an abstract mapping between RISC-V standard instructions and vector operations, supporting element-level calculations of basic arithmetic, logic, and comparison operations. Adapted to general-purpose computing scenarios, they can be directly associated with LLVM standard instructions to ensure execution efficiency. Operations specific to the in-memory computing architecture are designed for the hardware characteristics of SRAM-based in-memory computing architectures, directly mapping to underlying hardware instructions. These operations cover core scenarios such as matrix operations, convolution calculations, and heterogeneous collaboration, fully leveraging the computational advantages of in-memory computing convergence.

[0145] Table 5: General Computing Modules Operation Name Function Description riscv.{add,sub}riscv::{AddOp,SubOp} Features: Supports scalar / vector dual-mode computation; scalar mode handles single numerical operations, while vector mode supports element-wise tensor operations; the result type is consistent with the input operand type, meeting general arithmetic computation needs. Example: %int_scalar = "riscv.add"(%a, %b) : (i32, i32) -> i32 %int_float = "riscv.sub"(%a, %b) : (f32, f32) -> f32 %int_memref = "riscv.add"(%t_a, %t_b) :(memref <2x3xi32>, memref <2x3xi32>) -> memref <2x3xi32> riscv.{mul,div}riscv::{MulOp,DivOp} Functionality: Integer multiplication and floating-point division operations, supporting scalar / vector bimodal computation; integer multiplication supports unsigned / signed integers (implicitly depending on the type), and floating-point division conforms to the IEEE floating-point standard; adaptable to general numerical computation scenarios, such as weight multiplication of input and feature map normalization in neural networks. Example: %int_scalar = "riscv.mul"(%a,%b) : (i32, i32) -> i32 %int_memref = "riscv.div"(%t_a, %t_b) : (memref <2x3xi32>, memref <2x3xi32>) -> memref <2x3xi32> riscv.{min,max}riscv::{MinOp,MaxOp} Functionality: Element-level extremum operations (minimum / maximum value), compatible with INT / FP series data types, supports scalar / vector dual-mode computation; in vector mode, it is compatible with the parallel comparison logic of in-memory arrays, and can batch process tensor element extrema, suitable for scenarios such as pooling layers and activation function boundary constraints in neural networks; it supports batch processing mode, allowing multiple sets of data to be processed in a single operation. Example: %int_scalar = "riscv.min"(%a, %b) : (i32, i32) -> i32 %int_memref = "riscv.max"(%t_a, %t_b) : (memref <2x3xi32>, memref <2x3xi32>) -> memref <2x3xi32> riscv.{andi,ori, xori,noti}riscv::{AndIOp,OriOp,XoriOp,NotiOp} Features: Adapts to scalar / vector dual-mode computation; logical operations follow binary bit manipulation rules, commonly used in scenarios such as data masking and bit-level feature extraction; in vector mode, it supports parallel bit-by-bit operations on tensor elements, improving processing efficiency. Example: %a= riscv.xori %b, %c : i64 %f = riscv.ori %g, %h : memref <4xi32> riscv.{cmpi,cmpf}riscv::{CmpIOp,CmpfOp} Functionality: Comparison operations (integer comparison / cmpi, floating-point comparison / cmpf), supporting multiple comparison predicates (such as less than / slt, equal to / eq), outputting a boolean result (i1); in scalar mode, it is used for control flow branch condition judgment, and in vector mode, it is used for batch data filtering, condition mask generation, and other scenarios; comparison predicates must be specified through attributes to ensure the explicitness of the comparison logic. Example: %0 = "riscv.cmpi"(%lhs, %rhs) {predicate = "slt"} : (i32, i32)-> i1 %1 = "riscv.cmpi"(%a, %b) {predicate = "eq"}: (memref <4xi64>,memref <4xi64>) -> memref <4xi1> %2 = "riscv.cmpf"(%lhs, %rhs) {predicate= "slt"} : (f32, f32) -> i1

[0146] Table 6: Calculations that can be mapped to CIM Operation Name Function Description riscv.matmulriscv::MatmulOp Function: Matrix-to-matrix multiplication operation, directly mapping to the hardware multiply-accumulate instructions of in-memory compute arrays, supporting INT / FP series data types (such as i8 / fp16); the dimension of the input matrix must be specified (ensuring matrix multiplication dimension compatibility: number of columns in the first matrix = number of rows in the second matrix), and the dimension of the calculated result is "number of rows in the first matrix × number of columns in the second matrix"; suitable for scenarios such as fully connected layers and convolutional layers (after unfolding) in neural networks, fully utilizing the parallel computing power of in-memory compute arrays. Example: %2 = "riscv.matmul"(%0, %1): (memref<2x3xf64>, memref <3x4xf64>) -> memref <2x4xf64> riscv.batch_matmulriscv::BatchMatmulOp Functionality: Batch matrix-matrix multiplication operation, directly mapping to the hardware batch multiplication and accumulation instructions of the CIM array, supporting INT / FP series data types (such as i8 / fp16); executes multiple independent matrix multiplication operations in parallel in batches, ensuring that each matrix within a batch meets the dimension compatibility rule (number of columns in the first matrix = number of rows in the second matrix); the dimension of the calculation result is batch × number of rows in the first matrix × number of columns in the second matrix; suitable for scenarios such as batch inference, multi-head attention, and batch fully connected layers in neural networks. Example: %2 ="riscv.batch_matmul"(%0, %1) : (memref<8x2x3xf64>, memref<8x3x4xf64>) ->memref<8x2x4xf64> riscv.matvecriscv::MatvecOp Function: Matrix-vector multiplication operation, directly mapping to the hardware instructions of in-memory compute arrays, supporting INT / FP series data types; input is a matrix (tile type) and a vector (1-dimensional tile type), requiring "number of matrix columns = vector length", the calculation result is a vector (length = number of matrix rows); suitable for feature mapping, linear transformation, and other scenarios in neural networks, avoiding the redundant expansion of matrix-vector operations to matrix-matrix operations, and improving computational efficiency. Example: %2 = "riscv.matvec"(%0, %1) :(memref <4x5xf32>, memref <5xf32>) -> memref <4xf32> riscv.conv2driscv::Conv2dOp Functionality: Performs 2D convolution operations, directly calling the in-memory array to complete vector-matrix multiplication calculations with fixed weights. Supports variable stride and padding configurations. Inputs are feature maps and convolution kernels. Supports specifying stride and padding via attributes to adapt to convolution requirements of feature maps of different sizes. Example: %output = "riscv.conv2d"(% stride, %padding) : (memref <4x4xf32>, memref <2x2xf32>) -> memref <3x3xf32> riscv.transpose(riscv::TransposeOp) Syntax: Operation ::= `riscv.transpose` $memref `,` $permutation attr-dict `:`type($memref) `to` type($result) Example: %1 = "riscv.transpose"(%0) {transp=[1,0]} : (memref <2x3xf64>) -> (memref <3x2xf64>)

[0147] The communication and collaboration module in Table 7 implements functions such as synchronous control between the RISC-V main processor and the in-memory computing accelerator, and accelerates the startup of the core, ensuring that the collaborative execution of multiple computing units is free of data conflicts and adapting to the parallel scheduling requirements under the heterogeneous in-memory computing architecture.

[0148] Table 7: Communication and Collaboration Modules Operation Name Function Description riscv.wait_all Function: Enables synchronous control between the RISC-V main processor and in-memory accelerators, ensuring that the main processor continues execution only after all started accelerator tasks have completed. It supports asynchronous flags (async) and can be associated with preceding synchronization events (such as %e0) to achieve multi-stage synchronization. Suitable for in-memory collaborative computing scenarios, it avoids computational errors caused by unprepared data. Example: %e0 = riscv.wait_all async %e1 = riscv.wait_all async [%e0] riscv.launch Function: Used to launch in-memory computing accelerator cores. Input parameters (such as configuration parameters and data addresses) can be specified for the accelerator core, triggering the accelerator to execute a specified computation task. It supports associating memory objects (such as `memref` type) to pass data addresses to the accelerator, enabling data interaction between the main processor and the accelerator. It is suitable for task scheduling in heterogeneous in-memory computing architectures and can launch multiple accelerator cores in batches for parallel execution. Example: `riscv.launch async tile (%tx, %ty) in (%c0, %c1)args(%A , %B):memref<16x16xf16>, memref<16x16x f16> {…}`

[0149] In one specific embodiment, the runtime environment of the compilation system of the present invention includes the following hardware and software configuration:

[0150] Hardware environment: Employs a RISC-V in-memory computing heterogeneous chip platform, such as... Figure 3 As shown, the chip includes a RISC-V general-purpose computing core and an NPU acceleration core based on an SRAM-based in-memory computing architecture. The NPU acceleration core integrates an in-memory array, vector processing units, and on-chip local storage, and supports more than twenty RISC-V extension instructions for AI computing. The system can be deployed on an FPGA prototyping platform with a clock frequency of approximately 200MHz to verify the correctness and performance of the code generated by the compilation system. Functional correctness is verified by comparing the output results of the PyTorch golden reference model with the results of the target platform; execution performance is evaluated using metrics such as statistical model inference time, instruction execution cycle, and data transfer overhead.

[0151] Software environment: The compilation system is built on LLVM18.0 and the MLIR framework. It supports model input for the PyTorch deep learning framework through the Torch-MLIR front-end, runs on the Linux operating system, and has a complete compilation toolchain, which can complete model compilation, code generation and deployment execution.

[0152] The following uses the compilation process of the Llama 3.2 (1B) AI model as an example to illustrate the complete process of this invention:

[0153] Step 1: Obtain the unified tensor computation representation file for the model.

[0154] Download the Llama 3.2 (1B) edge AI model file from the Hugging Face open-source platform, and export the model as an intermediate representation file in TorchScript IR format based on the PyTorch framework; use the Torch-MLIR open-source front-end toolchain to convert TorchScript IR into the MLIR standard Linalg-on-Tensors Dialect. This Dialect uses a unified linear algebra computation abstraction to describe the tensor computation process in the model, and finally forms a standardized unified tensor computation representation file.

[0155] Step 2: Tensor to buffer conversion.

[0156] The official MLIR tool mlir-opt is invoked to perform a one-shot bufferization operation, which converts the abstract, value-semantic tensor representation into a memory-level, reference-semantic buffer representation. This realizes the transformation from high-level abstract IR to memory-aware IR, laying the foundation for subsequent hardware adaptation and optimization.

[0157] Step 3: Construct RISC-V CIM IR.

[0158] The `-convert-linalg-to-riscv` conversion pass, implemented based on the MLIR framework, converts the general tensor computation operators in the `Linalg-on-Tensors Dialect` into hardware-aware operations for the RISC-V in-memory computing heterogeneous architecture, and constructs a RISC-V CIM IR. Specifically, high-level linear algebra operators such as `linalg.batch_matmul` and `linalg.conv2d` are mapped to hardware operations such as `riscv.batch_matmul` and `riscv.conv2d`, which correspond to the execution semantics of the underlying in-memory computing array. Simultaneously, based on the configuration of the target RISC-V in-memory computing chip, the converted operations are assigned data types such as INT4 / INT8 / FP16 and computational precision to adapt to the hardware computing characteristics of the in-memory computing array.

[0159] Step 4: Compilation optimization based on five-dimensional features.

[0160] After the RISC-V CIM IR is built, a custom optimization pass is called to optimize the compilation in five dimensions: computation, concurrency, communication, data reuse, and collaborative synchronization, in order to improve the parallel execution efficiency and memory access efficiency of the in-memory computing heterogeneous chip. Specifically, the SPMD parallel domain is defined using `riscv.launch async tile(%tx, %ty)`, dividing large-scale tensor computations into multiple tile instances and distributing them to different CIM engines for parallel execution, thus achieving parallel scheduling optimization. A custom `riscv-tiling` optimization pass is invoked to perform tile block scheduling on computation tasks, dividing tensor computations into fine-grained tile blocks adapted to the on-chip SRAM capacity. Private on-chip SRAM cache space is allocated to tile instances using `riscv.alloc`, combined with asynchronous DMA transfer using `riscv.load / riscv.store`, ensuring that frequently accessed data continuously resides in on-chip storage during computation, thus achieving data reuse optimization. `riscv.wait_all` waits for asynchronous data loading and computation task completion, ensuring the execution order and data consistency between the RISC-V general-purpose core and the in-memory computing NPU acceleration core, thus achieving collaborative synchronization optimization (synchronization control optimization). Simultaneously, a custom `riscv-fusion` pass is invoked, based on RISC-V CIM... Data dependencies in IR merge adjacent computational operations into composite operators to reduce intermediate result storage and off-chip memory access overhead, thereby achieving operator fusion optimization.

[0161] Parallel scheduling optimization: SPMD parallel domains are defined by riscv.launch async tile (%tx, %ty), which divides large-scale tensor computations into multiple tile instances and distributes them to different CIM engines for parallel execution;

[0162] Data reuse optimization: Private on-chip SRAM is allocated for Tile instances via riscv.alloc, and combined with asynchronous DMA transfer via riscv.load / riscv.store, Tile data continues to reside in on-chip storage during computation, supporting multiple reuses;

[0163] Synchronization control optimization: By using riscv.wait_all to wait for asynchronous data loading and computation tasks to complete, the execution order and data consistency between the RISC-V general-purpose core and the NPU acceleration core are guaranteed.

[0164] Operator fusion optimization: A custom RISC-V fusion pass is invoked to merge adjacent operators in the data flow graph into composite operators, reducing intermediate result storage and memory access overhead. The data flow graph is implicitly constructed or extracted based on operator dependencies and data dependencies in RISC-V CIM IR.

[0165] A custom riscv-tiling optimization pass is invoked to enable the tile partitioning optimization method for block scheduling of computation tasks. Large-scale tensor computations are divided into fine-grained tile blocks adapted to the on-chip SRAM capacity. Cache space is allocated in the on-chip SRAM through the riscv.alloc operation, so that frequently accessed tile data resides in the on-chip SRAM during computation, reducing the number of off-chip memory accesses, improving data reuse rate and reducing memory access power consumption.

[0166] Step 5: Generate LLVM IR and convert it to RISC-V instructions. LLVM IR is a low-level platform-independent intermediate representation for compiler backends, used to describe program control flow, data flow, and low-level computational semantics, and supports further conversion to target machine instructions.

[0167] The LLVM-provided mlir-translate downgrade pass is invoked to convert the optimized RISC-V CIM IR into a standard LLVM IR. For general computational operations in the LLVM IR, RISC-V basic instructions are generated using the native LLVM toolchain. For the mapped LLVM Intrinsic functions, corresponding RISC-V CIM extended instructions are generated using the extended LLVM RISC-V backend, ultimately forming an executable instruction sequence containing basic and extended instructions.

[0168] Step 6: Deployment of the target program and execution of the model.

[0169] The generated target program, containing RISC-V basic instructions and in-memory computing extension instructions, is cross-compiled and linked to generate an executable file, which is then deployed to a RISC-V in-memory computing heterogeneous chip platform or an FPGA prototyping platform for execution. The RISC-V general-purpose processor core is responsible for program control, task scheduling, and data transfer control, while the in-memory computing NPU acceleration core is responsible for executing AI computing tasks such as matrix multiplication, convolution, and vector computation. During model execution, on-chip and off-chip data transfer is completed through `riscv.load` / `riscv.store`, and asynchronous task synchronization control is achieved through `riscv.wait_all`, ultimately yielding the AI ​​model inference output. The AI ​​model inference output can also be compared with the PyTorch golden reference model to verify the correctness of computational semantics, dataflow semantics, and target code generation during the conversion of the high-level AI model to the RISC-V in-memory computing target program in steps 1 to 5.

[0170] The following are system embodiments corresponding to the above method embodiments. This embodiment can be implemented in conjunction with the above embodiments. The relevant technical details mentioned in the above embodiments are still valid in this embodiment, and will not be repeated here to reduce repetition. Accordingly, the relevant technical details mentioned in this embodiment can also be applied to the above embodiments.

[0171] like Figure 6 As shown, this invention also proposes a RISC-V in-memory computing chip operating device B based on the intermediate representation of heterogeneous computing features, which includes:

[0172] The initial module exports the AI ​​model as a unified tensor computation representation file;

[0173] The conversion module converts the tensor representation of abstract value semantics in the unified tensor computation representation file into a memory-level buffer representation with reference semantics, thus obtaining an intermediate computation representation file and realizing the conversion from high-level abstract intermediate representation to memory-aware intermediate representation.

[0174] The module specifies the data type and computation precision for the hardware-aware operations in the intermediate computation representation file, and obtains the intermediate representation of heterogeneous computing features to adapt to the hardware computing characteristics of the in-memory computing array of the in-memory computing chip.

[0175] The optimization module, based on five dimensions including computation, communication, concurrency, data reuse, and collaborative synchronization, compiles and optimizes the intermediate representation of the heterogeneous computing feature description to obtain the CIM feature description. This compilation and optimization includes: dividing the tensor computation in the intermediate representation of the heterogeneous computing feature description into multiple Tile instances according to the parallel architecture of the in-memory computing chip, and distributing them to multiple CIM engines for parallel execution; allocating private on-chip SRAM to each instance according to the storage hierarchy of the in-memory computing chip, and combining asynchronous DMA transfer to allow instance data to reside in on-chip storage during computation, achieving data reuse; defining the collaborative execution and synchronization semantics between the general core and the NPU acceleration core in the in-memory computing chip in the intermediate representation of the heterogeneous computing feature description; and merging adjacent computational operations into composite operators based on the data dependencies in the intermediate representation of the heterogeneous computing feature description to reduce intermediate result storage and off-chip memory access overhead.

[0176] The code generation module converts the CIM feature description into standard LLVM IR, where LLVM IR is a low-level platform-independent intermediate representation for the compiler backend; it converts general computational operations in LLVM IR into RISC-V basic instructions for execution by the RISC-V general-purpose core; it converts LLVM Intrinsic built-in functions in LLVM IR for in-memory computing extension operations into RISC-V CIM extension instructions for execution by the in-memory computing NPU acceleration core by extending the instruction selection, lowering, and machine code generation modules in the LLVM RISC-V backend; and it generates the target AI model program based on the RISC-V basic instructions and RISC-V CIM extension instructions.

[0177] The aforementioned RISC-V in-memory computing chip operating device based on heterogeneous computing feature intermediate representation includes:

[0178] The deployment and execution module generates an executable file from the target AI model program through cross-compilation and linking, and deploys it to a RISC-V heterogeneous chip platform or FPGA prototype verification platform with an in-memory computing chip to obtain the inference output results of the target AI model program.

[0179] Among them, the RISC-V general-purpose processor core is used for program control, task scheduling and data transfer control, while the in-memory computing NPU acceleration core is used to perform matrix multiplication, convolution calculation and vector calculation tasks.

[0180] After the model is executed, the output results of the target platform are numerically compared with the output results of the PyTorch Golden Reference Model to verify the correctness of computational semantics, data flow semantics and target code generation during the conversion of the high-level AI model into the RISC-V in-memory computing target program, and to evaluate the execution performance of the generated code on the target platform.

[0181] The RISC-V in-memory computing chip operating device based on the intermediate representation of heterogeneous computing features includes: a basic module, a data loading and storage module, a memory management module, a shape transformation and type conversion module, a computing module, and a communication and collaboration module.

[0182] This basic module operates on function definitions, return control, and constant references, ensuring the integrity of function logic and the legality of data references;

[0183] This data loading and storage module operates to facilitate data interaction between main memory and on-chip SRAM, and between the CIM computing unit and SRAM. It adapts to the data access requirements of the in-memory computing architecture, clearly defines memory addresses, indexes, and data types, and ensures the correctness of data reading and writing.

[0184] This memory management module is used to allocate and release memory, adapt to the hierarchical storage requirements of data under the in-memory computing architecture, distinguish memory types through memory space parameters, and improve the utilization of storage resources.

[0185] This shape transformation and type conversion module is used for adjusting the dimensions and converting the precision of tensor data to adapt to the data layout requirements of attention mechanisms, convolution calculations and matrix operations.

[0186] This computing module is used for the abstract mapping of RISC-V standard instructions, vector operations and in-memory computing extended operations. It supports element-level calculations of basic arithmetic, logic and comparison operations, directly maps to the underlying hardware instructions, and covers matrix operations, convolution calculations and heterogeneous collaborative computing scenarios.

[0187] This communication and coordination module is used for synchronous control between the RISC-V main processor and the in-memory computing accelerator, accelerates core startup, ensures that the collaborative execution of multiple computing units is free of data conflicts, and adapts to the parallel scheduling requirements under the heterogeneous in-memory computing architecture.

[0188] The present invention also proposes a client for implementing any of the RISC-V in-memory computing chip operating devices based on heterogeneous computing feature intermediate representation.

[0189] like Figure 7 As shown, in another embodiment of the present invention, a first electronic device A is also proposed, including the RISC-V in-memory computing chip operating device B based on the intermediate representation of heterogeneous computing features.

[0190] like Figure 8 As shown, the first electronic device A can also be connected to the data acquisition device C and the information display device D through a wired or wireless information transmission scheme. The data acquisition device C is used to collect AI model files on the end side, and the information display device D is used to display the target AI model program obtained by the present invention.

[0191] The information display device D can process and organize the data output by the first electronic device A based on an information display mechanism to improve the readability of the data. This information display mechanism can be manually preset, for example, visualizing the data output by the first electronic device A. It can present the user with the specified key information based on user-defined display parameters and / or attributes, such as the data range and font, color, and scrolling options. Users can access this information more quickly without needing to navigate to secondary pages or scroll through pages, saving them time and effort. Alternatively, the information display mechanism can be an artificial intelligence (AI) display model that learns the user's key information interests based on past usage habits, such as viewing time, click count, and edit count, and automatically presents rich and necessary key information.

[0192] The present invention also provides a computer program product, which includes a computer program that can be stored on a readable storage medium. When the computer program is executed by a processor, the computer can execute the RISC-V in-memory computing chip operation method based on the intermediate representation of heterogeneous computing features provided by the above methods.

[0193] In another embodiment, the present invention also proposes a storage medium VIII for storing a computer program that executes the RISC-V in-memory computing chip operation method based on the intermediate representation of heterogeneous computing features. It should be understood that the storage medium in the embodiments of the present invention can be volatile memory or non-volatile memory, or may include both. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous DRAM (DDR SDRAM), enhanced synchronous DRAM (ESDRAM), synchronous linked DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0194] Figure 9 A schematic block diagram of a second electronic device 1000 that can be used to implement embodiments of the present invention is shown. The second electronic device 1000 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The second electronic device 1000 can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein. The second electronic device 1000 may be the same as or different from the first electronic device A.

[0195] The second electronic device 1000 includes a computing unit I, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory II (ROM) or a computer program loaded from storage medium VIII into random access memory (RAM) III. The RAM III may also store various programs and data required for the operation of the device 1000. The computing unit I, ROM II, and RAM III are interconnected via bus IV. An input / output (I / O) interface V is also connected to bus IV.

[0196] Multiple components in the second electronic device 1000 are connected to I / O interface V, including: input unit VI, such as a keyboard, mouse, etc.; output unit VII, such as various types of displays, speakers, etc.; storage medium VIII, such as a disk, optical disk, etc.; and communication unit IX, such as a network card, modem, wireless transceiver, etc. Communication unit IX allows the second electronic device 1000 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0197] The computing unit I can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of computing unit I include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit I performs the various methods and processes described above, such as method steps S1-S5. For example, in some embodiments, the methods can be implemented as computer software programs tangibly contained in a machine-readable medium, such as storage medium VIII. In some embodiments, part or all of the computer program can be loaded and / or installed on device 1000 via ROM II and / or communication unit IX. When the computer program is loaded into RAM III and executed by computing unit I, one or more steps of the methods described above can be performed. Alternatively, in other embodiments, computing unit I can be configured to perform methods by any other suitable means (e.g., by means of firmware).

[0198] Although embodiments of the present invention have been disclosed above, they are not limited to the applications listed in the specification and embodiments. They can be applied to various fields suitable for the present invention. For those skilled in the art, other modifications can be easily made. Therefore, without departing from the general concept defined by the claims and their equivalents, the present invention is not limited to the specific details and illustrations shown and described herein.< / type> < / items>

Claims

1. A method for operating a RISC-V in-memory computing chip based on intermediate representation of heterogeneous computing features, characterized in that, include: The initial step is to export the AI ​​model as a unified tensor computation representation file; The conversion step involves converting the abstract value semantic tensor representation in the unified tensor computation representation file into a memory-level buffer representation with reference semantics, thus obtaining an intermediate computation representation file and realizing the conversion from a high-level abstract intermediate representation to a memory-aware intermediate representation. The construction steps specify the data type and computation precision for the hardware-aware operations in the intermediate computation representation file to obtain the intermediate representation of heterogeneous computing features, so as to adapt to the hardware computing characteristics of the in-memory computing array of the in-memory computing chip. The optimization steps involve compiling and optimizing the intermediate representation of the heterogeneous computing feature description based on five dimensions: computation, communication, concurrency, data reuse, and collaborative synchronization, to obtain the CIM feature description. This compilation and optimization includes: dividing the tensor computation in the intermediate representation of the heterogeneous computing feature description into multiple Tile instances according to the parallel architecture of the in-memory computing chip, and distributing them to multiple CIM engines for parallel execution; allocating private on-chip SRAM to each instance according to the storage hierarchy of the in-memory computing chip, and combining asynchronous DMA transfer to allow instance data to reside in on-chip storage during computation, achieving data reuse; defining the collaborative execution and synchronization semantics between the general-purpose core and the NPU acceleration core in the in-memory computing chip in the intermediate representation of the heterogeneous computing feature description; and merging adjacent computational operations into composite operators based on the data dependencies in the intermediate representation of the heterogeneous computing feature description to reduce intermediate result storage and off-chip memory access overhead. The code generation steps involve converting the CIM feature description into a standard LLVM IR, where LLVM IR is a low-level platform-independent intermediate representation for the compiler backend; converting general computational operations in LLVM IR into RISC-V basic instructions for execution by the RISC-V general-purpose core; converting LLVM Intrinsic built-in functions in LLVM IR for in-memory computing extension operations into RISC-V CIM extension instructions for execution by the in-memory computing NPU acceleration core by extending the instruction selection, lowering, and machine code generation modules in the LLVM RISC-V backend; and generating the target AI model program based on the RISC-V basic instructions and RISC-V CIM extension instructions.

2. The RISC-V in-memory computing chip operation method based on heterogeneous computing feature intermediate representation as described in claim 1, characterized in that, include: The deployment and execution steps involve cross-compiling and linking the target AI model program to generate an executable file, and then deploying it to a RISC-V heterogeneous chip platform or FPGA prototype verification platform with an in-memory computing chip to obtain the inference output results of the target AI model program. Among them, the RISC-V general-purpose processor core is used for program control, task scheduling and data transfer control, while the in-memory computing NPU acceleration core is used to perform matrix multiplication, convolution calculation and vector calculation tasks. After the model is executed, the output results of the target platform are numerically compared with the output results of the PyTorch Golden Reference Model to verify the correctness of computational semantics, data flow semantics and target code generation during the conversion of the high-level AI model into the RISC-V in-memory computing target program, and to evaluate the execution performance of the generated code on the target platform.

3. The RISC-V in-memory computing chip operation method based on heterogeneous computing feature intermediate representation as described in claim 1, characterized in that, The heterogeneous computing feature description intermediate representation includes: a basic module, a data loading and storage module, a memory management module, a shape transformation and type conversion module, a computing module, and a communication and collaboration module; This basic module operates on function definitions, return control, and constant references, ensuring the integrity of function logic and the legality of data references; This data loading and storage module operates to facilitate data interaction between main memory and on-chip SRAM, and between the CIM computing unit and SRAM. It adapts to the data access requirements of the in-memory computing architecture, clearly defines memory addresses, indexes, and data types, and ensures the correctness of data reading and writing. This memory management module is used to allocate and release memory, adapt to the hierarchical storage requirements of data under the in-memory computing architecture, distinguish memory types through memory space parameters, and improve the utilization of storage resources. This shape transformation and type conversion module is used for adjusting the dimensions and converting the precision of tensor data to adapt to the data layout requirements of attention mechanisms, convolution calculations and matrix operations. This computing module is used for the abstract mapping of RISC-V standard instructions, vector operations and in-memory computing extended operations. It supports element-level calculations of basic arithmetic, logic and comparison operations, directly maps to the underlying hardware instructions, and covers matrix operations, convolution calculations and heterogeneous collaborative computing scenarios. This communication and coordination module is used for synchronous control between the RISC-V main processor and the in-memory computing accelerator, accelerates core startup, ensures that the collaborative execution of multiple computing units is free of data conflicts, and adapts to the parallel scheduling requirements under the heterogeneous in-memory computing architecture.

4. A RISC-V in-memory computing chip operating device based on heterogeneous computing feature intermediate representation, characterized in that, include: The initial module exports the AI ​​model as a unified tensor computation representation file; The conversion module converts the tensor representation of abstract value semantics in the unified tensor computation representation file into a memory-level buffer representation with reference semantics, thus obtaining an intermediate computation representation file and realizing the conversion from high-level abstract intermediate representation to memory-aware intermediate representation. The module specifies the data type and computation precision for the hardware-aware operations in the intermediate computation representation file, and obtains the intermediate representation of heterogeneous computing features to adapt to the hardware computing characteristics of the in-memory computing array of the in-memory computing chip. The optimization module, based on five dimensions including computation, communication, concurrency, data reuse, and collaborative synchronization, compiles and optimizes the intermediate representation of the heterogeneous computing feature description to obtain the CIM feature description. This compilation and optimization includes: dividing the tensor computation in the intermediate representation of the heterogeneous computing feature description into multiple Tile instances according to the parallel architecture of the in-memory computing chip, and distributing them to multiple CIM engines for parallel execution; allocating private on-chip SRAM to each instance according to the storage hierarchy of the in-memory computing chip, and combining asynchronous DMA transfer to allow instance data to reside in on-chip storage during computation, achieving data reuse; defining the collaborative execution and synchronization semantics between the general core and the NPU acceleration core in the in-memory computing chip in the intermediate representation of the heterogeneous computing feature description; and merging adjacent computational operations into composite operators based on the data dependencies in the intermediate representation of the heterogeneous computing feature description to reduce intermediate result storage and off-chip memory access overhead. The code generation module converts the CIM feature description into standard LLVM IR, where LLVM IR is a low-level platform-independent intermediate representation for the compiler backend; it converts general computational operations in LLVM IR into RISC-V basic instructions for execution by the RISC-V general-purpose core; it converts LLVM Intrinsic built-in functions in LLVM IR for in-memory computing extension operations into RISC-V CIM extension instructions for execution by the in-memory computing NPU acceleration core by extending the instruction selection, lowering, and machine code generation modules in the LLVM RISC-V backend; and it generates the target AI model program based on the RISC-V basic instructions and RISC-V CIM extension instructions.

5. The RISC-V in-memory computing chip operating device based on heterogeneous computing feature intermediate representation as described in claim 1, characterized in that, include: The deployment and execution module generates an executable file from the target AI model program through cross-compilation and linking, and deploys it to a RISC-V heterogeneous chip platform or FPGA prototype verification platform with an in-memory computing chip to obtain the inference output results of the target AI model program. Among them, the RISC-V general-purpose processor core is used for program control, task scheduling and data transfer control, while the in-memory computing NPU acceleration core is used to perform matrix multiplication, convolution calculation and vector calculation tasks. After the model is executed, the output results of the target platform are numerically compared with the output results of the PyTorch Golden Reference Model to verify the correctness of computational semantics, data flow semantics and target code generation during the conversion of the high-level AI model into the RISC-V in-memory computing target program, and to evaluate the execution performance of the generated code on the target platform.

6. The RISC-V in-memory computing chip operating device based on heterogeneous computing feature intermediate representation as described in claim 1, characterized in that, The heterogeneous computing feature description intermediate representation includes: a basic module, a data loading and storage module, a memory management module, a shape transformation and type conversion module, a computing module, and a communication and collaboration module; This basic module operates on function definitions, return control, and constant references, ensuring the integrity of function logic and the legality of data references; This data loading and storage module operates to facilitate data interaction between main memory and on-chip SRAM, and between the CIM computing unit and SRAM. It adapts to the data access requirements of the in-memory computing architecture, clearly defines memory addresses, indexes, and data types, and ensures the correctness of data reading and writing. This memory management module is used to allocate and release memory, adapt to the hierarchical storage requirements of data under the in-memory computing architecture, distinguish memory types through memory space parameters, and improve the utilization of storage resources. This shape transformation and type conversion module is used for adjusting the dimensions and converting the precision of tensor data to adapt to the data layout requirements of attention mechanisms, convolution calculations and matrix operations. This computing module is used for the abstract mapping of RISC-V standard instructions, vector operations and in-memory computing extended operations. It supports element-level calculations of basic arithmetic, logic and comparison operations, directly maps to the underlying hardware instructions, and covers matrix operations, convolution calculations and heterogeneous collaborative computing scenarios. This communication and coordination module is used for synchronous control between the RISC-V main processor and the in-memory computing accelerator, accelerates core startup, ensures that the collaborative execution of multiple computing units is free of data conflicts, and adapts to the parallel scheduling requirements under the heterogeneous in-memory computing architecture.

7. A client for implementing any of the RISC-V in-memory computing chip operating devices based on heterogeneous computing feature intermediate representations as described in claims 4-6.

8. An electronic device, characterized in that, The device includes a RISC-V in-memory computing chip operating device based on heterogeneous computing feature intermediate representation as described in claims 4-6. The electronic device may be connected to an information display device, which is used to display the target AI model program with user-set display parameters, attributes, or through an artificial intelligence model.

9. A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the RISC-V in-memory computing chip operation method based on the intermediate representation of heterogeneous computing features as described in any one of claims 1-3.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the RISC-V in-memory computing chip operation method based on the intermediate representation of heterogeneous computing features as described in any of claims 1-3.

Images