Design method for AI compiler of multi-core end-side AI chip

Abstract

The invention discloses a design method of an AI compiler of a multi-core end side AI chip, and belongs to the field of AI chip scheduling systems. According to the method, a multi-core storage allocation strategy, a synchronization mechanism and code generation optimization Pass are set in the compiling process, and efficient scheduling and modular management of the multi-core parallel computing tasks are achieved. The method is based on a multi-stage intermediate representation framework popular in the industry, corresponding functions are easy to develop, a multi-core storage distribution and synchronization mechanism can be defined in a specific Pass, modular management is achieved, the stability and expansibility of a compiler can be improved while the flexibility of a multi-core framework is guaranteed, and the method is suitable for popularization and application. Therefore, the performance and the development efficiency of the end-side AI chip in a complex application scene are improved.

Description

Technical Field

[0001] This invention relates to the field of AI chip scheduling system technology, and in particular to an AI compiler design method for a multi-core edge AI chip. Background Technology

[0002] When front-end AI chips have limited computing power, they typically use self-developed AI compiler frameworks. This presents several problems: First, multi-core compilation involves complex memory allocation and synchronization mechanisms, which are often poorly handled by typical AI compiler frameworks. Second, the software architecture of multi-core compilation is usually more complex, and in cases of poor code framework design, the robustness of the compiler function is not strong. Summary of the Invention

[0003] The purpose of this invention is to provide an AI compiler design method for multi-core edge AI chips to solve the problems of complex storage resource utilization and task dependency management under multi-core architecture.

[0004] To address the aforementioned technical problems, this invention provides a design method for an AI compiler for a multi-core edge AI chip, comprising the following steps: The machine learning model is converted into a first intermediate representation, which uses a first custom dialect and is used to describe the hardware-independent computational logic of the model and its data dependencies. Based on the hardware characteristics of multi-core edge AI chips, at least one optimization traversal is applied to the first intermediate representation to convert the first intermediate representation into a second intermediate representation. The hardware characteristics include the number of processing cores, on-chip storage capacity, and synchronization overhead. In the optimized traversal, a storage allocation and synchronization mechanism for multi-core scenarios is defined, and modularization is achieved through Pass management; The second intermediate indicates the use of a second custom dialect to explicitly describe the mapping relationship between computational tasks and hardware resources in a multi-core processor; and, Executable code adapted to the edge AI chip is generated based on the second intermediate representation.

[0005] In one implementation, the optimized traversal includes: Storage allocation optimization divides the logical operations in the intermediate representation into multiple subtasks according to the on-chip storage capacity and explicitly specifies their storage space allocation locations; The synchronization mechanism is optimized by generating explicit synchronization instructions in the second intermediate representation based on the dependencies between computing tasks, which are used to coordinate the parallel execution of multiple processing cores.

[0006] In one implementation, the synchronization command includes a signal sending command and a signal waiting command; wherein, When a task depends on multiple predecessor tasks, generate a synchronization instruction that can wait for all predecessor tasks to complete. When a task is depended on by multiple subsequent tasks, generate multiple signal sending instructions for that task.

[0007] In one implementation, the optimized traversal is performed during the process of converting the first intermediate representation into the second intermediate representation: Operator fusion combines multiple logical operations into a single hardware-aware operation to reduce data transmission. Operator splitting divides a single logical operation into multiple sub-operations to accommodate on-chip memory limitations and multi-core parallel execution.

[0008] In one implementation, the memory allocation optimization and synchronization mechanism optimization are implemented in a modular manner through a multi-level intermediate representation Pass framework, which enables the compiler to have strong robustness and maintainability when expanding functionality.

[0009] In one implementation, the process of generating executable code adapted to the edge AI chip based on the second intermediate representation includes instruction selection, scheduling, and register allocation steps to ensure the execution efficiency of the generated code on a multi-core edge AI chip.

[0010] This invention provides an AI compiler design method for multi-core edge AI chips, based on the industry-popular MLIR (Multi-Level Intermediate Representation) framework. It is easy to develop corresponding functions, and the storage allocation and synchronization mechanism of multi-cores can be defined in a specific pass to achieve modular management and has very strong software robustness. Attached Figure Description

[0011] Figure 1 This is a flowchart illustrating the design method of the AI ​​compiler for a multi-core edge AI chip provided by the present invention.

[0012] Figure 2 This is a schematic diagram of the intermediate representation conversion process based on MLIR according to the method of the present invention.

[0013] Figure 3 This is a schematic diagram of the multi-core storage allocation pass of the method of the present invention.

[0014] Figure 4 This is a schematic diagram of the synchronization mechanism generated by the method of the present invention. Detailed Implementation

[0015] The following detailed description, in conjunction with the accompanying drawings and specific embodiments, provides a further detailed explanation of the AI ​​compiler design method for a multi-core edge AI chip proposed in this invention. The advantages and features of this invention will become clearer from the following description. It should be noted that the accompanying drawings are all in a very simplified form and use non-precise proportions, and are only used to facilitate and clarify the illustration of the embodiments of this invention.

[0016] This invention provides a design method for an AI compiler for a multi-core edge AI chip, the overall process of which is illustrated below. Figure 1 As shown, it specifically includes: Step 1: Import the external model (e.g., the ONNX exported model) into the compilation front end, convert it into the first intermediate representation IR_1 (the first intermediate representation IR_1 is hardware-independent, for example, Dialect_HIL), and record the operators, shape and dependencies; Step 2: Read the hardware information of the target device's multi-core AI chip (number of cores, on-chip storage per core, bandwidth, etc.). Step 3: Within the MLIR framework, perform "optimized traversal" according to the modular pass, and gradually convert the first intermediate representation IR_1 into the second intermediate representation IR_2 (the second intermediate representation IR_2 is hardware-dependent, for example, Dialect_HWMAP). Step 4: Based on the second intermediate representation IR_2, execute code generation in the backend (including instruction selection, scheduling, register and memory allocation, and synchronization instruction insertion), and output the target executable code.

[0017] Figure 2 This is a detailed explanation of the IR conversion and optimization process in the method of this invention, specifically including: Step 3.1: Operator Fusion / Segmentation Pass: Determine and execute operator fusion or segment large operators by dimension; Step 3.2: Storage Allocation Pass: Perform lifecycle analysis on the tensor and make allocation decisions on on-chip memory; Step 3.3: Synchronize and generate Pass: Generate explicit synchronization primitives (signal / wait / wait-all) based on task dependencies; Step 3.4: After a series of modular passes, the first intermediate representation IR_1 is transformed into a second intermediate representation IR_2 containing specific hardware information in order to generate efficient and correct multi-core parallel code.

[0018] Figure 3 This is the key point of implementing the storage and allocation of Pass in the method of this invention. The specific process includes: Step 3.2.1: Traverse the data dependencies in the first intermediate representation IR_1 and calculate the allocation interval of each tensor to detect possible memory conflicts and reuse. Step 3.2.2: Based on hardware constraints (such as local buffer capacity per core) and access frequency, a greedy algorithm is used to allocate high-priority tensors to on-chip buffers, and place the rest in off-chip memory and generate necessary copy nodes; Step 3.2.3: When a single tensor exceeds the capacity of a single core, it is split along the channel direction to generate multiple sub-tensors and corresponding sub-tasks. In the second intermediate representation, different memory spaces mem_space and hardware core core_affinity are allocated in IR_2. Step 3.2.4: In the second intermediate representation IR_2, mark mem_space for each tensor and copy node, and insert copy operations at the operator boundaries to realize data movement.

[0019] Figure 4 This is the key point of synchronously generating Pass in the method of this invention, and its main contents include: Step 3.3.1: Extract the logical dependencies between tasks from the first intermediate representation IR_1, construct a directed acyclic graph (DAG) of tasks, and label the predecessor and successor of each task; Step 3.3.2: Sync_id (synchronization ID) allocation: Assign a unique identifier Snyc_id to each dependency or event that needs to be synchronized. This identifier is carried in IR_2 and used by the backend. Step 3.3.3: Synchronization primitive generation: Generate the following primitives or mechanisms based on dependencies: signal—issued when the predecessor task is completed to notify the successor; wait—wait for the corresponding signal before the successor task starts; wait-all—when there are multiple predecessors for the successor, generate a compound synchronization that waits for all predecessors to complete. Step 3.3.4: Multiple successors / multiple predecessors: When a task corresponds to multiple successors, generate independent signals for different successors. When a task has multiple predecessors, the successors execute wait-all until all predecessors complete. Step 3.3.5: Synchronization events can be bound to DMA copy completion interrupts or events, so that copy completion can also serve as a signal to trigger subsequent calculations.

[0020] The second middle indicates the insertion position in IR_2: signal is generally inserted after the predecessor operator ends; wait is inserted before the successor operator begins; counter or wait-all creates the corresponding structure where needed.

[0021] The above description is merely a description of preferred embodiments of the present invention and is not intended to limit the scope of the present invention in any way. Any changes or modifications made by those skilled in the art based on the above disclosure shall fall within the protection scope of the claims.

Claims

1. A design method for an AI compiler for a multi-core edge AI chip, characterized in that, Includes the following steps: The machine learning model is converted into a first intermediate representation, which uses a first custom dialect and is used to describe the hardware-independent computational logic of the model and its data dependencies. Based on the hardware characteristics of multi-core edge AI chips, at least one optimization traversal is applied to the first intermediate representation to convert the first intermediate representation into a second intermediate representation. The hardware characteristics include the number of processing cores, on-chip storage capacity, and synchronization overhead. In the optimized traversal, a storage allocation and synchronization mechanism for multi-core scenarios is defined, and modularization is achieved through Pass management; The second intermediate indicates the use of a second custom dialect to explicitly describe the mapping relationship between computational tasks and hardware resources in a multi-core processor; and, Executable code adapted to the edge AI chip is generated based on the second intermediate representation.

2. The AI ​​compiler design method for a multi-core edge AI chip as described in claim 1, characterized in that, The optimized traversal includes: Storage allocation optimization divides the logical operations in the intermediate representation into multiple subtasks according to the on-chip storage capacity and explicitly specifies their storage space allocation locations; The synchronization mechanism is optimized by generating explicit synchronization instructions in the second intermediate representation based on the dependencies between computing tasks, which are used to coordinate the parallel execution of multiple processing cores.

3. The AI ​​compiler design method for a multi-core edge AI chip as described in claim 2, characterized in that, The synchronization instructions include signal sending instructions and signal waiting instructions; wherein... When a task depends on multiple predecessor tasks, generate a synchronization instruction that can wait for all predecessor tasks to complete. When a task is depended on by multiple subsequent tasks, generate multiple signal sending instructions for that task.

4. The AI ​​compiler design method for a multi-core edge AI chip as described in claim 1, characterized in that, The optimized traversal is performed during the process of converting the first intermediate representation into the second intermediate representation: Operator fusion combines multiple logical operations into a single hardware-aware operation to reduce data transmission. Operator splitting divides a single logical operation into multiple sub-operations to accommodate on-chip memory limitations and multi-core parallel execution.

5. The AI ​​compiler design method for a multi-core edge AI chip as described in claim 2, characterized in that, The optimized storage allocation and synchronization mechanism are implemented in a modular manner through the Pass framework with multi-level intermediate representation, which makes the compiler more robust and maintainable when expanding its functions.

6. The AI ​​compiler design method for a multi-core edge AI chip as described in claim 1, characterized in that, The process of generating executable code adapted to the edge AI chip based on the second intermediate representation includes instruction selection, scheduling, and register allocation steps to ensure the execution efficiency of the generated code on the multi-core edge AI chip.

Images