A method for generating a machine code of a multi-task parallel heterogeneous processor

Abstract

The present application belongs to the technical field of heterogeneous processor machine code compilation, and particularly relates to a multi-task parallel heterogeneous processor machine code compilation generation method. The method comprises the following steps: defining a loadable heterogeneous task format based on a protobuf protocol and defining a device task format in a self-defined manner to describe binary machine code for task execution; parsing a model compiled by an MLIR compiler through an MLIR parsing module to extract model structure, weight parameters and task dependency compilation information required for processing unit execution; mapping and generating a binary file containing complete register configuration for describing task execution through processing of an MLIR compiler backend; and constructing a task dependency graph based on the producer-consumer dependency relationship between tasks. The present application realizes multi-task parallel scheduling and resource optimization, thereby reducing redundant occupation of computing resources.

Description

Technical Field

[0001] This invention belongs to the field of heterogeneous processor machine code compilation technology, and specifically relates to a method for compiling and generating machine code for multi-task parallel heterogeneous processors. Background Technology

[0002] Heterogeneous processors are generally divided into a computation control unit (hereinafter referred to as the host program) and an acceleration unit (hereinafter referred to as the heterogeneous processor, i.e., the edge chip). The host program is responsible for task generation and scheduling, dispatching the task code and data to the chip acceleration unit for execution. If multiple tasks need accelerated computation, the host program can only submit these tasks sequentially, wait for their status to return, and then submit the next task, and so on.

[0003] However, the existing technology has the following drawbacks: 1) Heterogeneous computing devices usually rely on the main core to dynamically configure registers: In traditional methods, the configuration value of the register is dynamically calculated and assigned by the main core, which not only requires additional time, but also requires frequent access to hardware registers, increasing scheduling overhead; 2) Serialization of task scheduling: The main core needs to check the task dependencies one by one, which leads to limited parallelism of multi-tasks and low utilization of hardware resources; 3) Uneven computing load: The task allocation lacks a dynamic optimization mechanism, which may cause acceleration units to be idle or overloaded. Summary of the Invention

[0004] The purpose of this invention is to provide a method for compiling and generating machine code for multi-task parallel heterogeneous processors. This invention proposes a method for defining and compiling machine code with pre-configured register bit field information through deep learning model compilation technology based on MLIR (Multi-Level Intermediate Representation). The compiler backend directly generates a binary file containing complete register configuration. Combined with the topological sorting of the task dependency graph, it realizes multi-task parallel scheduling and resource optimization, thereby reducing the redundant occupation of computing resources and improving the utilization efficiency of resources.

[0005] To address the aforementioned technical problems, this invention provides a method for compiling and generating machine code for multi-task parallel heterogeneous processors, comprising:

[0006] The binary machine code for task execution is described by using a loadable heterogeneous task format defined based on the protobuf protocol and a device task format defined in a custom way.

[0007] The MLIR parsing module parses the model compiled by the MLIR compiler and extracts the model structure, weight parameters, and task dependency compilation information required for the execution of the processing unit.

[0008] The MLIR compiler backend processes the parsed compilation information and maps it to generate a binary file containing a complete register configuration that describes the execution of the task. The binary file includes a task.bin file, which is an independent binary file for each task and contains detailed information about each smallest task unit Op.

[0009] A task dependency graph, or computation graph, is constructed based on the producer-consumer dependency relationship between tasks. The execution order of tasks is determined by topological sorting based on MLIR, so as to achieve parallel scheduling and resource optimization of multiple tasks.

[0010] Preferably, the model compiled by the MLIR compiler is a net.mlir file, which includes: tasks, the hierarchical structure of the smallest task unit Op, and the level configuration information of the smallest task unit Op.

[0011] Preferably, the binary file includes: a loadable.bin file, which is a graph-level binary file containing description information of the entire computation graph and metadata and binary data of all tasks; wherein the loadable.bin file contains references to each task.bin file for locating task-level binary data during task scheduling.

[0012] Preferably, the loadable.bin file also generates a JSON file as a descriptive file, which is used to explain the field definitions, data structures and debugging information of the loadable.bin or task.bin file.

[0013] Preferably, the data structure of the loadable.bin file adopts the Protobuf protocol format. The content of the Protobuf protocol format includes: the number of inputs Input_num and its description Input_shapes, the number of outputs Output_num and its description Output_shapes, the number of tasks Task_num, and the binary files of the tasks Tasks Tasks; the data of the binary files of the tasks Tasks Tasks Tasks Tasks Tasks Tasks Tasks Tasks Tasks Tasks Task ID, offset, and producer-consumer dependency information.

[0014] Preferably, the data structure of the task.bin file includes: task description information, instruction and configuration information of each smallest task unit (Op), and constant data information; wherein the task description information includes the task ID and the number of Ops; the instruction and configuration information of each smallest task unit (Op) are placed sequentially according to the Op instruction format and the order of the Op configuration register group; the constant data information includes weight data, bias data, and LUT lookup table data.

[0015] Preferably, the pre-configuration process of the Op configuration register group includes: configuring the bit fields, automatically generating the Op configuration register group bit field configuration table according to the Op type and hardware interface specification, so that the hardware scheduler of the heterogeneous processor can directly read the Op configuration register group bit field configuration table without the main core intervention.

[0016] Preferably, a virtual instruction register is allocated to each minimum task unit Op through the MLIR-based topology sorting; that is, a unique instruction register number is allocated to each minimum task unit Op according to the MLIR topology sorting to record the register mapping table for subsequent bit field configuration.

[0017] Preferably, the process of generating the task.bin file includes: converting the Op instruction into the machine code instruction buildOpInst, integrating the register pre-configuration buildOpConfig and the constant data buildConstant to generate the task.bin file.

[0018] Preferably, the task dependency graph is a directed acyclic graph (DAG), which is serialized using the Protobuf protocol. This serializes the number of tasks, the number of Ops, the Op types and parameters, register configurations, and constant offset information into loadable.bin and loadable.bin.json files. At this point, the entire process of serializing the computation graph into machine code is complete.

[0019] Compared with the prior art, the present invention has the following advantages:

[0020] This invention defines a loadable heterogeneous task format based on the Protobuf protocol and designs a custom format to describe device task tasks. It also describes a Protobuf serialization generator used for compiling and generating loadable heterogeneous tasks, and a machine code compilation and generation method implemented using a custom computation graph data structure (including an MLIR parsing module, register allocator, operation instruction generator, and constant builder). Efficient management of task dependencies is achieved through pre-configured register bit field information. This method directly reads the pre-configured register information through the hardware scheduler, reducing main core intervention, improving multi-task parallel scheduling efficiency, and reducing computational resource waste. Specifically, the hardware scheduler reads the information directly: the hardware scheduler of the heterogeneous processor directly reads the register bit field configuration in task.bin without main core intervention; dynamic resource allocation: computational resources are dynamically allocated based on the core requirements of the task (e.g., 1 core, 4 cores) and the availability of acceleration unit groups; and a register reclamation mechanism: register release nodes are recorded through release_map, and idle registers are dynamically reclaimed during the traversal process. Attached Figure Description

[0021] Figure 1 The flowchart illustrates a method for compiling and generating machine code for a multi-task parallel heterogeneous processor, as provided by this invention.

[0022] Figure 2 This is a diagram showing the definition structure of the machine code in this invention.

[0023] Figure 3 This is a schematic diagram illustrating how the task Bin file of the present invention is stored during operation.

[0024] Figure 4 This is a code structure diagram for generating the multi-task Bin file of the present invention.

[0025] Figure 5 This is a schematic diagram illustrating the pre-configuration process of the config register in this invention. Detailed Implementation

[0026] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. The advantages and features of the present invention will become clearer from the following description. It should be noted that the drawings are all in a very simplified form and use non-precise proportions, and are only used to facilitate and clarify the illustration of the embodiments of the present invention.

[0027] like Figure 1 As shown, the technical solution of the present invention provides a method for defining and compiling machine code for multi-task parallel heterogeneous processors, including the following steps:

[0028] S1 and MLIR file acquisition;

[0029] Input: The net.mlir file generated by a custom compiler built on MLIR, which contains the model's task, Op hierarchy structure and Op-level configuration information.

[0030] S2 and MLIR file parsing;

[0031] Analysis: The MLIR parsing module extracts task dependencies (producer-consumer), Op types, and required computing resources.

[0032] S3, Instruction Register Allocation;

[0033] Instruction register allocation: Based on MLIR topology sorting, a virtual instruction register is allocated to each Op. A unique register number is assigned to each Op according to the MLIR sorting; a register mapping table (reg_map<node ID, instruction register number>) is recorded for subsequent bit field configuration. This avoids register conflicts and supports register reuse during multi-task parallel execution.

[0034] S4. Configure register bit field pre-configuration;

[0035] Bit-segment configuration: Based on Op attributes (such as ConvOp core size and number of channels) and hardware interface specifications (register bit width), generate a configuration register bit-segment configuration table (such as reg_config[Highbit:Lowbit]<register number, bit-segment value>); to reduce main core intervention, enabling the hardware scheduler to directly read pre-configured bit segments and reduce latency.

[0036] S5, task binary generation;

[0037] Task binary construction: Convert Op to machine code instruction buildOpInst, integrate register pre-configuration buildOpConfig and constant data buildConstant, and generate task.bin file.

[0038] S6. Construction of the computational graph;

[0039] Dependency Graph Construction: Based on the producer-consumer relationship of tasks, construct a directed acyclic graph (DAG).

[0040] S7. Computation graph serialization;

[0041] Serialization via the Protobuf protocol: Information such as the number of tasks, the number of Ops, Op types, register configurations, and constant offsets are serialized into loadable.bin and loadable.bin.json files. This completes the entire process of serializing the computation graph into machine code; the specific content of the Protobuf protocol is shown in Table 1 below.

[0042] Table 1

[0043] Protobuf protocol content meaning Input_num The number of inputs contained in the model Input_shapes The shape information of each input is used for the allocation of the input tensor. Output_num The number of outputs contained in the model Output_shapes The shape information of each output is used for the allocation of output tensors. task_num The number of tasks contained in the model tasks The data for tasks is the task data for all tasks, including task ID, offset, producer-consumer dependency relationship, and other information.

[0044] As a further explanation of the embodiments of the present invention, the following terms are explained in the embodiments of the present invention:

[0045] MLIR: Based on a modified Multi-Level Intermediate Representation framework, it transforms a common model into a model file that can run on heterogeneous chips.

[0046] task: The task generated by MLIR contains key information such as the model's weights and register configurations, and includes several Ops;

[0047] loadable: Adopts the Protobuf protocol and contains multiple heterogeneous tasks;

[0048] Op: The smallest unit of task (Operator), including but not limited to deep learning operators such as Conv and ReLU;

[0049] Producer task: This is the parent task of a certain task. The child task can only begin to be calculated after all parent tasks have been completed.

[0050] Consumer task: a subtask of a certain task.

[0051] like Figure 2 The diagram shown is a definition structure diagram of the machine code, that is, a schematic diagram of the task.bin structure, which shows the four main parts of the task: the number of Ops description information, the instruction information of the Ops, the configuration register information of the Ops, and the constant data information.

[0052] The instruction count description section contains Op number information, which indicates the total number of operators to be executed in this task.

[0053] The instruction information description section contains the aforementioned Op instruction information Op_instruction, arranged in sequential order. Each instruction information is 64 bits, and every few bits form a bit field to represent the instruction type Op_type, instruction number Op_index, instruction corresponding configuration information offset address Op_Config_offset, and instruction required constant data offset address Op_constant_offset, etc., which are the information required for execution.

[0054] The configuration register description section contains the aforementioned Op_number Op configuration information Op_configuration, arranged in sequential order. Each configuration information consists of multiple configuration register information, and each configuration register information is 32 bits, which contains one or more bit fields to represent the configuration information required for Op execution, such as kernel_size and slide_stride.

[0055] The constant data description section contains some of the constant data required for Op execution, such as weight and bias.

[0056] like Figure 3 The diagram illustrates the storage of the task bin file during runtime according to the present invention. The task.bin file is loaded into a separate area of ​​the memory pool. The data in the memory pool includes: 1) an Input_Tensor area storing one or more input tensors of the computation graph; 2) a Task_assemble_tensor area storing intermediate results such as tensors generated during the execution of each Task; 3) a Task_data area where each Task.bin file is loaded and stored; and 4) an Output_Tensor area storing one or more output tensors of the computation graph.

[0057] like Figure 4 The diagram shows the code structure for generating multi-task Bin files according to the present invention, including steps such as MLIR parsing, register allocation, task binary construction, and computation graph serialization. The Bin file generation code structure mainly consists of a backend translation module (translator), which takes an mlir file as input and outputs a loadable.bin file and a loadable.bin.json file. The backend graph construction module (MLIR To Graph) is responsible for reading the mlir file and converting it into a backend computation graph, which facilitates more detailed memory allocation, register allocation, and register bit field configuration. The memory allocation module is used for memory pool planning, space allocation, and offset address calculation. Virtual register allocation is used to assign an index (Op_index) to each Op based on topology sorting and allocation algorithms.

[0058] The TaskBin file generation module comprises three sub-modules: 1) The TaskConfig update module, responsible for updating constant information and the offset addresses of each Tensor; 2) The TaskFormat build module, responsible for building... Figure 3 The format of Task.bin mentioned above.

[0059] 3) The TaskBin file export module (dump Task Format to Bin) is responsible for generating Task.bin and Task.bin.json files.

[0060] The Backend Graph Serializer module is responsible for serializing the entire graph information and the information of each task according to the protocol defined by Protobuf, and finally outputting the loadable.bin and loadable.bin.json files.

[0061] like Figure 5 The diagram illustrates the pre-configuration process of the config register, demonstrating how parameter-level data is written to the corresponding bit fields in the bin file according to their respective bit widths. Specifically, the data for each parameter is retrieved from the Op parameter information array, and the corresponding bit fields of the configuration register are assigned values ​​based on the bit width format of the hardware register. For example, if the value of parameter-level OpParam0 is value0, and the corresponding [high_bit:low_bit] bit field in the first configuration register is in the hardware register, then the bit_set method is used to assign the value value0 to the h to l bits of that 32-bit data. When multiple bit fields of a configuration register correspond to multiple parameters, the above method is used multiple times for assignment.

[0062] As a further illustration of the embodiments of the present invention, the embodiments of the present invention are specifically described using the open-source model resnet18.onnx file as an example.

[0063] Specifically, this invention provides a method for defining and compiling machine code for a multi-task parallel heterogeneous processor, comprising the following steps:

[0064] S1 and MLIR file acquisition;

[0065] Taking the open-source model resnet18.onnx file as an example, the resnet18_l.mlir file is first obtained through the ONNX-MLIR process, and then a custom MLIR compiler generates a custom dialect mlir format file, which contains the model's task, Op hierarchy structure and Op-level configuration information.

[0066] S2 and MLIR file parsing;

[0067] Parse the MLIR file and extract task dependencies (producer-consumer), Op types in the task, and required computing resources through the MLIR parsing module.

[0068] S3, BackendGraph computational graph reconstruction;

[0069] The compilation information of the entire model is used to reconstruct all tasks in the backend computation graph. Each task contains several Ops. These objects store key information such as the task and Op number, quantity, offset address in the file, register value, and number of required computation cores.

[0070] S4. Configure register bit field pre-configuration;

[0071] Bit field configuration: Based on the Op type (such as Conv, Relu) and hardware interface specification, generate a configuration register bit field configuration table (reg_config[high_bit : low_bit]]<register number, bit field value>; for example, if bits 0 to 4 of the 31st register of the VPU are configured as 1, then it is vpu_reg_config[4:0]<31, 1>).

[0072] S5, task binary generation;

[0073] The task binary construction process converts the Ops in the task into machine code instructions `buildOpInst`, integrates the pre-configured registers `buildOpConfig` and the constant data `buildConstant`, and generates the `task.bin` file. This process is repeated to encapsulate the `bin` files of each task.

[0074] S6. Construction of computation graph format;

[0075] Dependency graph construction: Based on the producer-consumer relationship of tasks, construct a directed acyclic graph (DAG).

[0076] S7. Computation graph serialization;

[0077] Protobuf serialization is used to serialize information such as the number of tasks, the number of Ops, Op types and parameters, register configurations, and constant offsets into loadable.bin and loadable.bin.json files. This completes the entire process of serializing the computation graph into machine code.

[0078] The above description is merely a description of preferred embodiments of the present invention and is not intended to limit the scope of the present invention in any way. Any changes or modifications made by those skilled in the art based on the above disclosure shall fall within the protection scope of the claims.

Claims

1. A method for compiling and generating machine code for a multi-task parallel heterogeneous processor, characterized in that, The application relates to a method for compiling a model into a binary file for task execution. The method comprises the following steps: Defining a loadable heterogeneous task format based on a protobuf protocol and a device task format in a self-defined manner to describe binary machine code for task execution; Parsing a model compiled by an MLIR compiler through an MLIR parsing module to extract model structure, weight parameters and task dependency compilation information required for processing unit execution; Mapping and generating a binary file containing complete register configuration for describing task execution through processing of an MLIR compiler backend; The binary file comprises a task.bin file as an independent binary file of each task, which contains detailed information of each minimum task unit Op. A task dependency graph, i.e. a calculation graph, is constructed based on the producer-consumer dependency relationship among tasks, and the task execution order is determined based on MLIR-based topological sorting to realize multi-task parallel scheduling and resource optimization. The data structure of the task.bin file comprises description information of the task, instruction and configuration information of each minimum task unit Op and constant data information; the description information of the task comprises the ID and Op quantity information of the task; the instruction and configuration information of each minimum task unit Op are sequentially placed according to the Op instruction format and the order of the Op configuration register group; and the constant data information comprises weight data, bias data and lut lookup table data.

2. A method of generating a machine code for a multi-task parallel heterogeneous processor according to claim 1, wherein, The pre-configuration process of the Op configuration register group comprises the following steps: configuring a bit segment, automatically generating an Op configuration register group bit segment configuration table according to the Op type and hardware interface specification, and realizing direct reading of the Op configuration register group bit segment configuration table by a hardware scheduler of a heterogeneous processor without intervention of a main core.

3. A method of generating a machine code for a multi-task parallel heterogeneous processor according to claim 1, wherein, The model compiled by the MLIR compiler is a net.mlir file, and the net.mlir file comprises the hierarchical structure of the task and the minimum task unit Op and the level configuration information of the minimum task unit Op.

4. A method of generating a machine code for a multi-task parallel heterogeneous processor according to claim 3, wherein, The binary file comprises a loadable.bin file as a binary file at the whole graph level, which contains description information of the whole calculation graph and meta information and binary data of all tasks; the loadable.bin file contains reference relationships of each task.bin file, so as to locate the binary data at the task level during task scheduling. The loadable.bin file simultaneously generates a json file as a descriptive file for explaining the field definition, data structure and debugging information of the loadable.bin or task.bin file.

5. A method of generating a machine code for a multi-task parallel heterogeneous processor according to claim 4, wherein, The data structure of the loadable.bin file adopts a Protobuf protocol format, and the content of the Protobuf protocol format includes an input number Input_num and description Input_shapes, an output number Output_num and description Output_shapes, a task number task_num and a binary file tasks of the task; the data of the binary file tasks of the task is the data of all the tasks, including an ID, an offset and producer-consumer dependency information of the task.

6. A method of generating a machine code for a multi-task parallel heterogeneous processor according to claim 1, wherein, According to the MLIR-based topological sorting, a virtual instruction register is allocated for each minimum task unit Op; that is, a unique instruction register number is allocated for each minimum task unit Op according to the MLIR-based topological sorting, so as to record a register mapping table, which is used for subsequent bit segment configuration.

7. A method of generating a machine code for a multi-task parallel heterogeneous processor according to claim 1, wherein, The generation process of the task.bin file includes converting an Op instruction into a machine code instruction buildOpInst, integrating a register pre-configuration buildOpConfig and constant data buildConstant, and generating the task.bin file.

8. A method of generating a machine code for a multi-task parallel heterogeneous processor according to claim 1, wherein, The task dependency graph is a directed acyclic graph DAG, and is serialized through a Protobuf protocol, that is, a task number, an Op number, an Op type and parameters, a register configuration and constant offset information are serialized into a loadable.bin and a loadable.bin.json file. Thus, the whole computation graph serialization into a machine code flow is completed.

Images