Compilation scheduling method, compiler and electronic device

Abstract

This application provides a compilation scheduling method, compiler, and electronic device. The compilation scheduling method is applied to a multi-core heterogeneous processor, including multiple processor cores, each processor core including tightly coupled memory and various types of hardware execution units. The method includes: obtaining a task graph to be compiled, containing data dependencies; dividing the tasks corresponding to the task graph into multiple subtasks based on a preset scheduling partitioning strategy, and allocating the multiple subtasks to the multiple processor cores; for each processor core with allocated subtasks, constructing an intra-core task pipeline for the subtasks allocated to each processor core based on the capacity constraints of tightly coupled memory and direct memory access; and generating corresponding target operator code based on the allocation result of the processor cores and the construction result of the intra-core task pipeline. This application effectively improves the overall performance of compilation scheduling.

Description

Technical Field

[0001] This application relates to the field of smart chip technology, and in particular to a compilation scheduling method, a compiler, and an electronic device. Background Technology

[0002] With the development of AI (Artificial Intelligence) technology, various AI models are widely used in people's work and life. The operation of these models requires the support of AI chips, therefore, the performance requirements for AI chips are also getting higher and higher.

[0003] In AI chips, compilers play a crucial role, serving not only as a "bridge" between AI chips and AI users but also as a key tool for improving chip performance. However, current compiler development heavily relies on expert experience, requiring manual management of complex pipeline designs and low-level hardware management. This results in deep coupling between code and specific hardware architectures, leading to high development costs and lengthy development cycles. Summary of the Invention

[0004] In view of this, embodiments of this application provide a compilation scheduling scheme to at least partially solve the above-mentioned problems.

[0005] According to a first aspect of the embodiments of this application, a compilation scheduling method is provided, applied to a multi-core heterogeneous processor including multiple processor cores, each processor core including tightly coupled memory and multiple different types of hardware execution units; the method includes: obtaining a task graph to be compiled, containing data dependencies; dividing the tasks corresponding to the task graph into multiple subtasks based on a preset scheduling partitioning strategy, and allocating the multiple subtasks to the multiple processor cores; for each processor core to which subtasks are allocated, constructing an intra-core task pipeline for the subtasks allocated to each processor core based on the capacity constraints of tightly coupled memory and the capacity constraints of direct memory access; and generating corresponding target operator code based on the allocation result of the processor cores and the construction result of the intra-core task pipeline.

[0006] According to a second aspect of the embodiments of this application, a compiler is provided, comprising: a graph capture layer for acquiring a task graph to be compiled, containing data dependencies; a pipelined compilation layer for dividing the tasks corresponding to the task graph into multiple subtasks based on a preset scheduling partitioning strategy, and allocating the multiple subtasks to multiple processor cores; for each processor core to which subtasks are allocated, constructing an intra-core task pipeline for the subtasks allocated to each processor core based on the capacity constraints of tightly coupled memory and direct memory access; a dynamic code generation layer for generating corresponding target operator code based on the allocation result of the processor cores and the construction result of the intra-core task pipeline; and a task scheduling execution layer for scheduling and allocating tasks in the target operator code to matching hardware execution units for execution; wherein the hardware execution unit includes multiple different types of hardware execution units, the hardware execution unit is disposed in a processor core, the processor core is disposed in a multi-core heterogeneous processor, and the processor core further includes tightly coupled memory.

[0007] According to a third aspect of the embodiments of this application, an electronic device is provided, including a compiler and a processor as described in the second aspect, wherein the processor is a multi-core heterogeneous processor including multiple processor cores, each processor core including tightly coupled memory and multiple different types of hardware execution units; the compiler schedules and allocates tasks in the target operator code to the hardware execution units in the multi-core heterogeneous processor for execution through the task scheduling execution layer.

[0008] The solution provided in this application is applied to a multi-core heterogeneous processor, which includes multiple processor cores. Based on the physical structure of the multi-core heterogeneous processor, it obtains a task graph containing data dependencies, enabling the compiler to understand the computation graph topology of the operator to be executed from a global perspective. Then, based on a preset scheduling partitioning strategy, tasks are macroscopically allocated among multiple processor cores, achieving cross-processor core scheduling. In the micro-scheduling stage for a single processor core, the compiler no longer relies on the implicit scheduling mechanism of the underlying hardware, but explicitly uses the capacity constraints of tightly coupled memory and direct memory access as boundary conditions to build intra-core task pipelines for different types of hardware execution units. Furthermore, based on the two-level scheduling decisions (cross-core and intra-core), the compiler directly maps and generates the target operator code. Thus, cross-core scheduling allocation and explicit construction of intra-core task pipelines are achieved through software during the compilation stage, thereby decoupling the deep binding between the upper-level algorithm code and the underlying hardware architecture. Moreover, it eliminates the need for manual management of complex pipeline design and adaptation to the underlying hardware, significantly reducing development costs. Furthermore, the automatic construction of task pipelines based on capacity constraints of tightly coupled memory and direct memory access can effectively utilize hardware resources, improve the utilization rate and overall execution performance of various hardware execution units in multi-core heterogeneous processors, and thus improve the overall performance of compilation and scheduling. Attached Figure Description

[0009] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings.

[0010] Figure 1A This is a flowchart illustrating the steps of a compilation scheduling method according to an embodiment of this application; Figure 1B for Figure 1A A schematic diagram of an example process of inter-core scheduling in the illustrated embodiment; Figure 1C for Figure 1A A schematic diagram of an example process of processor core scheduling in the illustrated embodiment; Figure 1D for Figure 1C A schematic diagram of a specific example; Figure 1E for Figure 1A The illustrated embodiment presents a schematic diagram of a scheduling relationship based on a data structure. Figure 1F for Figure 1A A schematic diagram illustrating the relationship between a hardware execution unit and a task type in the illustrated embodiment; Figure 2A This is a structural block diagram of a compiler according to an embodiment of this application; Figure 2B for Figure 2A The diagram illustrates the compilation scheduling process of a compiler in one of the embodiments shown. Figure 3 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application. Detailed Implementation

[0011] To enable those skilled in the art to better understand the technical solutions in the embodiments of this application, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art should fall within the protection scope of the embodiments of this application.

[0012] The specific implementation of the embodiments of this application will be further described below with reference to the accompanying drawings.

[0013] In practical applications of AI technology, a large amount of inference computation involving AI models is required. Due to the scale of these models and the substantial amount of data involved in the inference computation, AI acceleration has become an increasingly important aspect of AI technology development. In the field of AI acceleration, GPUs have become the primary consideration due to their advantages in architecture design, parallel computing, memory bandwidth, and hierarchical optimization. However, on the one hand, AI tasks are end-to-end processes, still relying on CPUs for data preprocessing, scheduling, and post-processing; on the other hand, in some scenarios, the entire AI task must also be implemented by the CPU. Therefore, CPU-based AI acceleration cannot be ignored.

[0014] Furthermore, GPUs leverage massive concurrent threads to enhance efficiency in their underlying hardware design. They also incorporate sophisticated thread schedulers that quickly switch to another group of ready threads when a thread is blocked waiting for memory access, thus accelerating AI task execution overall. In contrast, CPUs, particularly RISC-V-based CPUs, typically lack thread schedulers capable of managing a large number of concurrent hardware thread contexts. In CPUs, if a hardware execution unit stalls while waiting for data, it creates pipeline bubbles, resulting in wasted computing resources. Moreover, GPUs possess vast register files, allowing compilers to buffer intermediate results extensively, whereas CPU register resources are relatively limited.

[0015] Therefore, many compilation scheduling schemes suitable for GPUs, which rely on hardware multi-threaded dynamic switching and implicit pipelining mechanisms with very large register files, cannot be directly adapted to CPUs, which lack a large number of hardware threads and have fewer registers. This necessitates establishing a dedicated compilation scheduling mechanism for CPU AI acceleration, based on a thorough consideration of the CPU architecture, to efficiently coordinate the execution of AI tasks by scheduling relevant hardware components within the CPU.

[0016] Therefore, embodiments of this application provide a compilation scheduling scheme applicable to multi-core heterogeneous processors including multiple processor cores.

[0017] In several embodiments of this application, the multi-core heterogeneous processor is exemplified by the RISC-V architecture. However, those skilled in the art should understand that other similar multi-core heterogeneous processor architectures, such as ARM architecture and MIPS architecture, are also applicable to the solutions of the embodiments of this application.

[0018] The multi-core heterogeneous processor in this embodiment includes multiple processor cores. Each processor core integrates various hardware execution units (EUs) designed for different computing modes, including but not limited to: scalar or vector computation units for handling general scalar logic and vector operations, matrix computation units for handling high-density tensor multiplication and addition operations, and direct memory access units (DMA) responsible for data movement. Unlike the traditional method of implicitly managing data by relying on multi-level hardware caches, the multi-core heterogeneous processor in this embodiment configures tightly coupled memory (TCM) within each processor core. This tightly coupled memory is an on-chip high-speed static random access memory that can be fully explicitly controlled by software.

[0019] Based on the aforementioned hardware architecture, this application provides a compilation scheduling scheme based on the characteristics of this architecture. The following describes this compilation scheduling scheme through multiple embodiments.

[0020] Reference Figure 1A This document illustrates a flowchart of a compiler scheduling method according to an embodiment of this application. The compiler scheduling method of this embodiment includes the following steps: Step S102: Obtain the task graph to be compiled, which contains data dependencies.

[0021] In this embodiment, the task graph is an abstract model describing the dependencies and data flow between operations in an AI task. It provides the foundation for the compiler to perform task partitioning, parallel scheduling, and resource allocation. In this embodiment, the task graph can be obtained by the compiler by receiving and parsing high-level model code uploaded by the user or input from a higher-level deep learning framework, and converting it into a structured intermediate representation (IR). However, this is not the only method; other methods of obtaining the graph are also applicable to the solutions in this embodiment.

[0022] A task graph can be a directed acyclic graph (DAG) describing the computational logic and data flow of an AI task. In a task graph, each node represents a computation or memory access operation, while the edges between nodes represent the data dependencies between operations. For example, a multiplication operation begins execution after both of its required input tensors have been loaded; this sequence forms the data dependency of the multiplication operation. Through the task graph, the compiler can examine the topology of the entire AI task from a global perspective, laying the foundation for subsequent task decomposition and scheduling.

[0023] Step S104: Based on the preset scheduling and partitioning strategy, the task corresponding to the task graph is divided into multiple subtasks, and the multiple subtasks are assigned to multiple processor cores of the multi-core heterogeneous processor.

[0024] As mentioned earlier, the task graph describes the overall topology of an AI task, but during execution, it needs to be broken down into subtasks that can ultimately be executed by multiple hardware execution units. Therefore, in this step, a preset scheduling partitioning strategy is used to divide the task described in the task graph into multiple subtasks, and these subtasks are compiled and allocated to corresponding processor cores. The scheduling partitioning strategy describes the specific method of dividing the task corresponding to the task graph into subtasks. In one example, based on the spatial dimension of the data corresponding to the task or the logical stage of the computation, the computation process that originally belonged to one task can be cut into multiple subtasks that can run independently or collaboratively through limited communication.

[0025] Therefore, the compiler can perform dimensionality reduction and decoupling operations on the tasks corresponding to the task graph from a global perspective to obtain multiple subtasks.

[0026] In one alternative approach, dividing a task corresponding to a task graph into multiple subtasks based on a preset scheduling partitioning strategy may include: dividing the task corresponding to the task graph into multiple subtasks based on at least one of the following scheduling partitioning strategies. The scheduling partitioning strategies include: a first strategy for instructing parallel partitioning of the task corresponding to the task graph into multiple subtasks matching processor cores for parallel execution across multiple processor cores; a second strategy for instructing the division of the task corresponding to the task graph into multiple subtask pipelines for parallel execution across multiple processor cores; and a third strategy for selecting a strategy from the first and second strategies based on the computation-to-memory access ratio of the task corresponding to the task graph.

[0027] The first strategy employs a data-parallel partitioning mode. In one example, when executing the first strategy, the compiler can equally partition the overall input tensor along a specified dimension based on the data space dimensionality of the task. This allows multiple processor cores to independently process their own data blocks, thus achieving data parallelism among the multiple processor cores. For example, the input tensor can be equally partitioned into 16 data blocks along a specific dimension, such as the batch dimension or the head dimension, allowing each of the 16 processor cores to process 16 independent data blocks. This strategy uses the spatial dimension of the data operated on by the task as the partitioning axis, dividing the task equally or proportionally along a self-decomposable dimension of the data into multiple subtasks matching the number of processor cores. These subtasks do not overlap, and each subtask is assigned to a corresponding processor core for independent execution. Each subtask can complete its computation with little or no inter-core communication, effectively improving task partitioning efficiency and subsequent subtask execution efficiency.

[0028] The second strategy employs a pipelined parallel partitioning model. In one example, if the tightly coupled memory of a single processor core cannot accommodate all the intermediate variables required for the execution of an operator, the compiler can use the second strategy to divide the operator into multiple successive sub-task pipelines according to the computational logic stage. Adjacent stages are then connected through inter-core communication mechanisms, forming a cross-processor core pipelined parallelism. For example, processor core 0 is responsible for loading data and matrix multiplication, while processor core 1 receives data generated by processor core 0 and executes activation function operations, forming a pipelined chain through inter-core communication. This strategy uses the computational logic stage of a task as the partitioning axis, decomposing multiple computational stages that were originally executed sequentially within a single processor core into sub-task pipeline levels that can be carried out by different processor cores. The intermediate results produced by the previous processor core are passed to the next processor core through inter-core communication channels, and the processor cores at each level advance in parallel in an overlapping manner on the time axis, forming a cross-core software pipeline structure. This not only overcomes the capacity limitations of tightly coupled memory on a single processor core but also improves throughput through pipeline overlap.

[0029] The third strategy employs an adaptive hybrid mode. It statically analyzes the ratio between the inherent computational workload and memory access data volume of a task—the computation-to-memory access ratio—and uses this ratio as the criterion to automatically determine whether to adopt the first or second strategy. It does not directly perform task partitioning but rather acts as an adaptive decision-making mechanism based on quantitative indicators. Using the task's computation-to-memory access ratio as the criterion, this ratio reflects the relative weighting of the task's computational and memory bandwidth consumption. Based on this, it dynamically selects the partitioning method more suitable for the current task between the first and second strategies. Thus, it achieves adaptive scheduling based on task characteristics and avoids performance degradation caused by strategy misuse, forming a complementary and complete strategy system with the first and second strategies.

[0030] In practical applications, the above three strategies can be used individually or in combination.

[0031] In one alternative approach, if the scheduling partitioning strategy includes a first strategy, a second strategy, and a third strategy, then the above-mentioned partitioning of the task corresponding to the task graph into multiple subtasks based on the preset scheduling partitioning strategy may include: using the first strategy, the second strategy, and the third strategy respectively to perform multi-task partitioning on the task corresponding to the task graph to obtain a corresponding first subtask set, a second subtask set, and a third subtask set; performing performance evaluation on the subtasks in the first subtask set, the second subtask set, and the third subtask set; and based on the evaluation results, selecting a target subtask set from the first subtask set, the second subtask set, and the third subtask set, and determining multiple subtasks in the target subtask set as the final target subtasks.

[0032] In the above approach, three strategies are executed concurrently, each generating its own set of subtasks. The compiler's built-in evaluation model can evaluate the subtasks within each set from at least one dimension, such as load balancing, communication overhead, theoretical execution cycle, peak memory usage, and hardware utilization, to select the target subtask set. Multiple subtasks from this set are then used as the final target subtasks for further processing. By using these three strategies in parallel, a wider strategy search space can be covered with minimal compilation time overhead, thereby improving the global optima of the finally selected target subtasks.

[0033] In another alternative approach, if the scheduling partitioning strategy includes a first strategy, a second strategy, and a third strategy, then the above-mentioned partitioning of the task graph corresponding to multiple subtasks based on the preset scheduling partitioning strategy may include: determining the execution priority among the first, second, and third strategies; partitioning the task graph corresponding to the task graph sequentially according to the execution priority from high to low until the execution performance of the resulting multiple subtasks meets the preset standard, or all strategies have been executed; and determining the multiple subtasks that meet the preset standard as target subtasks, or determining the multiple subtasks obtained based on the strategy with the lowest execution priority as target subtasks. The priority order among the first, second, and third strategies can be set by those skilled in the art according to actual needs, and the preset standard can also be appropriately set by those skilled in the art according to actual needs, including but not limited to load balancing meeting a preset balance, communication overhead within a preset range, etc.

[0034] In the above approach, the three strategies are assigned different priorities. The compiler applies each strategy sequentially for task partitioning according to their priority from highest to lowest. After each partition generates a subtask, it compares it with a preset standard. Once the standard is met, the compiler immediately stops using the next strategy and outputs the partitioning result, without waiting for all strategies to complete. This transforms the original exhaustive partitioning process, which required using all strategies, into a fast-converging decision-making process, significantly reducing the time overhead of the compilation phase. If all strategies have been executed but none have met the standard, the subtask obtained by the lowest priority strategy is output as a fallback result. This ensures that the compiler can output a usable result under any circumstances, guaranteeing the integrity and robustness of the compilation process.

[0035] In another alternative approach, if the scheduling partitioning strategy includes a first strategy and a second strategy, then the above-mentioned partitioning of the task corresponding to the task graph into multiple subtasks based on the preset scheduling partitioning strategy may include: using the first strategy to perform multi-task partitioning on the task corresponding to the task graph to obtain multiple subtasks; for each subtask, using the second strategy to perform task pipeline partitioning to obtain multiple pipeline subtasks; and obtaining the final target subtask based on the multiple subtasks and the multiple pipeline subtasks corresponding to each subtask.

[0036] In the above approach, the first and second strategies are used in a nested hierarchical structure. At the first level, the compiler first applies the first strategy to horizontally partition the original task graph, distributing the overall computational load along the data dimension across the processor cores, resulting in multiple relatively independent subtasks. At the second level, for each subtask obtained in the first level, the compiler further applies the second strategy to vertically partition its computational stages, breaking down the continuous computation process within each subtask into multiple pipelined subtasks that can overlap and be executed. The horizontal data parallel partitioning and the vertical pipeline stage partitioning are superimposed, allowing inter-core data parallelism and intra-core pipeline parallelism to be planned simultaneously in the same compilation process. Thus, the advantages of the first strategy in eliminating single-processor core serial bottlenecks and balancing the load are fully combined with the advantages of the second strategy in overcoming single-processor core memory capacity limitations and improving throughput through pipeline overlap, allowing the parallel potential of multi-core heterogeneous processors to be fully explored at both the cross-core and intra-core levels. Simultaneously, since the two partitioning levels correspond to different optimization dimensions, there is no mutual interference between them, allowing the compiler to independently optimize in their respective dimensions, reducing the complexity of joint optimization.

[0037] The following, combined with Figure 1B The above-mentioned processor core scheduling method is illustrated by example.

[0038] like Figure 1B As shown, after receiving the global task graph, the compiler initiates scheduling and partitioning. In this process, strategies one, two, and three are activated, partitioning the tasks described by the task graph from different dimensions. As mentioned earlier, strategy one employs a data-parallel partitioning mode, dividing the task according to the batch dimension or head dimension (such as the attention head) of the data operated on by the task, ensuring that each processor core obtains an independent data slice and executes its own non-interfering computational subtasks. Strategy two employs a pipelined parallel partitioning mode, partitioning the task according to the computational logic stage. Figure 1BIn the example of dividing a task into stages according to its sequence length (SeqLen dimension), each processor core takes the adjacent computation stages in a relay manner. The intermediate results produced by the previous processor core are passed to the next processor core for further processing via the inter-core communication channel. The third strategy adopts an adaptive hybrid mode, which comprehensively considers the computation-to-memory ratio and load characteristics of the current task and adaptively combines the first and second strategies to adapt to workload scenarios with more complex computation modes.

[0039] After the three strategies are used in the aforementioned three combinations and each generates its corresponding set of subtasks, the process enters the segmentation combination enumeration stage. In this stage, the compiler systematically enumerates all subtask sets generated by the three strategies, registering each possible segmentation result as a candidate solution to be evaluated. The enumerated objects include not only segmentation results produced by a single strategy acting independently, but also composite segmentation results produced by nested combinations of multiple strategies. Through exhaustive combination enumeration, the compiler can provide the most complete possible input of candidate solutions for subsequent evaluation without overlooking any potential solutions.

[0040] After completing the partitioning and combination enumeration, the process enters the load balancing and communication overhead evaluation stage. In this stage, the compiler quantitatively evaluates each candidate solution (i.e., the partitioning result) from two dimensions: load balancing evaluation and inter-core communication overhead evaluation. Load balancing evaluation measures whether the computational load on each processor core is equal across candidate solutions. If the computational load differs significantly among processor cores, the heavily loaded core will become the performance bottleneck for the overall task execution, while the less loaded cores will have a large amount of idle computing power, resulting in a low load balancing score for the candidate solution. Inter-core communication overhead measures the amount of inter-core data transfer and the number of synchronization waits required during the execution of each candidate solution. Higher communication overhead means longer idle time for each processor core while waiting for data transfer, negatively impacting overall throughput, resulting in a higher communication overhead score for the candidate solution. In one example, the compiler can fuse the evaluation results from these two dimensions, for example, by combining them into a single cost function value, as the overall quantitative basis for evaluating the merits of the candidate solution. By performing the above two-dimensional evaluation on each candidate solution, the compiler can make a horizontal comparison of all candidate solutions on an objective and quantitative basis, without relying on the subjective experience and judgment of experts, making the evaluation results more accurate and objective.

[0041] After completing the quantitative evaluation of all candidate solutions, the process enters the partitioning scheme selection stage. In this stage, the compiler selects the candidate solution with the lowest overall cost from all candidates based on the results of load balancing and communication overhead evaluations, such as the cost function value, as the final task partitioning scheme. This selection process, based on quantitative evaluation, is objective and repeatable, ensuring consistent final results for the same task graph under the same hardware configuration. Furthermore, optionally, in practical applications, if multiple candidate solutions have the same or similar evaluation results, the compiler can further filter them according to preset secondary ranking rules, such as prioritizing solutions with higher load balancing or lower communication overhead. Specific secondary ranking rules can be flexibly set by those skilled in the art based on the actual bandwidth and computing power characteristics of the target hardware platform; this embodiment does not impose such limitations.

[0042] After selecting candidate partitioning schemes, the process enters the core task mapping table output stage, which outputs the final state of the entire cross-core scheduling partitioning process. The core task mapping table is a structured mapping data structure with processor core identifiers as keys and sub-task subgraphs carried by that processor core as values. It completely records the set of all subtasks that each processor core should execute under the finally selected partitioning scheme and their inter-task dependencies. The contents of the core task mapping table will serve as input for the subsequent intra-core task pipeline construction stage. When processing the intra-core scheduling of each processor core, the compiler uses the sub-task subgraphs recorded in the corresponding entries of this mapping table as the operation objects, further refining the intra-core pipeline layout under the dual boundary conditions of tightly coupled memory capacity constraints and direct memory access capacity constraints. By explicitly presenting the results of the above inter-core scheduling as structured data in the form of a core task mapping table, the compiler achieves a clear interface division between the cross-core scheduling stage and the intra-core scheduling stage, enabling the two stages to evolve and iterate independently in a loosely coupled manner. It also provides checkable and debuggable data snapshots of the intermediate states of the compilation process.

[0043] Step S106: For each processor core to which a subtask is assigned, construct an intra-core task pipeline for the subtasks assigned to each processor core based on the capacity constraints of tightly coupled memory and direct memory access.

[0044] After task allocation across processor cores is completed, each processor core knows the subtasks it needs to handle. However, how multiple subtasks are executed in an orderly and efficient manner within a single processor core still requires further planning by the compiler. In this embodiment, the compiler constructs an intra-core task pipeline for the subtasks allocated to each processor core to achieve orderly and efficient execution of subtasks within the processor core.

[0045] However, because the processor core in this embodiment of the application is equipped with processor core-private on-chip high-speed storage, namely tightly coupled memory, the execution of subtasks requires its support but is also limited by its capacity. Therefore, when constructing an in-core task pipeline for subtasks, at least the capacity constraint of tightly coupled memory needs to be considered, that is, at any scheduling time, the total number of bytes of all data blocks residing in tightly coupled memory does not exceed its physical capacity limit.

[0046] In addition, direct memory access (DMA) resources are also a factor affecting the construction of the kernel task pipeline for subtasks. At any given scheduling moment, the number of direct memory access request entries currently queued for execution or initiated but not yet completed does not exceed the maximum number of concurrent slots supported by the direct memory access controller hardware.

[0047] Therefore, this application embodiment avoids hardware anomalies when the core task pipeline runs on a real physical silicon chip by setting dual hardware constraints, namely, the capacity constraint of tightly coupled memory capacity and the capacity constraint of direct memory access, thus ensuring the stability of subtask operation and executing more subtasks in an overlapping manner as much as possible, thereby realizing the pipelined operation of computation and data transfer.

[0048] Based on this, in one alternative approach, for each processor core to which subtasks are assigned, building an intra-core task pipeline for the subtasks assigned to each processor core, based on the capacity constraints of tightly coupled memory and direct memory access, can be implemented as follows: For each processor core, select ready subtasks from the subtasks assigned to it; determine whether the tightly coupled memory has memory space to execute ready subtasks based on the capacity constraints of tightly coupled memory, and determine whether direct memory access has available slots based on the capacity constraints of direct memory access; if both memory space and available slots are available, then generate an intra-core task pipeline for the ready subtasks.

[0049] In this context, a ready subtask can be any subtask whose preceding dependent subtasks on the task graph have been successfully scheduled and confirmed to have been executed, and which currently meets the conditions for execution. For example, the compiler can maintain a ready queue to continuously track the dependency state of each subtask assigned to the current processor core. For a given subtask, once it completes, its successor subtasks are scanned, and all successor subtasks whose dependencies have been satisfied are updated to the ready state and added to the ready queue.

[0050] Because the embodiments of this application set dual capacity constraints of tightly coupled memory and direct memory access, for each ready subtask (such as the ready subtask in the ready queue mentioned above), the compiler will query the remaining available capacity of the current tightly coupled memory and the number of free slots of the current direct memory access controller. When both resources meet the execution requirements of the subtask, an in-kernel task pipeline will be generated for it.

[0051] By employing the above methods, the dynamic resource consumption during the compilation process of subtasks is precisely simulated, avoiding hardware anomalies caused by memory overflow or full direct memory access queues. This ensures the stability and correctness of the generated code when running on real processor cores. At the same time, through dual capacity constraints, the compiler can maximize pipeline fill rate while maintaining hardware safety, thus balancing performance optimization and hardware security.

[0052] In one alternative approach, when both the capacity constraints of tightly coupled memory and direct memory access are met—that is, when both memory space and available slots are available—generating an in-kernel task pipeline for a ready subtask can be implemented as follows: if both memory space and available slots are available, determine whether the ready subtask is a subtask with data dependencies; if so, generate an in-kernel task pipeline for the ready subtask and insert a synchronization barrier in the in-kernel task pipeline.

[0053] A synchronization barrier is a control instruction marker inserted by the compiler into the generated code sequence to forcibly block the advancement of subsequent instructions in the pipeline during the execution of a subtask until it is confirmed that all operations marked before the barrier marker have been completed. Within multi-core heterogeneous processors, there is no automatic implicit synchronization mechanism between different hardware execution units. For example, if the multiplication-addition result of a matrix computation unit has not yet been written back to tightly coupled memory, and the immediately following vector computation unit has already begun reading that address, a data race will occur, leading to incorrect calculation results. To avoid similar problems, in this embodiment, when constructing the kernel task pipeline of a subtask, the compiler traverses the data flow edges between ready subtasks and their predecessor subtasks, identifies all adjacent subtasks with direct data read / write dependencies, and sets synchronization barrier instructions before the instruction sequence of subsequent subtasks to achieve the insertion of the synchronization barrier. This avoids the risk of subtask execution exceptions due to missed dependencies.

[0054] In addition, for multiple ready subtasks, since the execution of most subtasks depends on the data read, in one alternative approach, after selecting ready subtasks, asynchronous direct memory access subtasks can be identified from the ready subtasks and scheduled with priority to prefetch data and speed up task execution time as much as possible.

[0055] However, this is not the only option. In another alternative approach, after selecting ready subtasks, critical path subtasks can be identified from them and prioritized for scheduling. Critical path subtasks refer to tasks that are on the longest dependency chain or whose completion unlocks many subsequent subtasks.

[0056] If a ready subtask contains both an asynchronous direct memory access subtask and a critical path subtask, their scheduling priority can be determined by prioritizing them according to a predefined priority order. The specific priority settings can be configured by those skilled in the art based on actual needs, and this application does not impose any limitations on this.

[0057] Thus, on the one hand, by prioritizing the scheduling of asynchronous direct memory access subtasks, data movement can be started as early as possible. While the processor core is executing the current computation task, the data transmission of the next stage is already proceeding synchronously on the background bus, achieving the overlap of memory access time and computation time, and avoiding pauses caused by hardware execution units waiting for data. On the other hand, by prioritizing the scheduling of critical path subtasks, topological dependency blocking in the task graph can be quickly eliminated, allowing more subsequent subtasks that are in a suspended state to enter the ready state, thereby preventing pipeline bubbles and improving the overall concurrent processing efficiency of heterogeneous processor cores.

[0058] The following, combined with Figure 1C The kernel scheduling process of the above subtasks will be illustrated by example.

[0059] like Figure 1C As shown, for each processor core, its input can be a task subgraph, which represents the multiple subtasks assigned to that processor core and their topological dependencies. Based on this task subgraph, the compiler constructs the in-core task pipeline for each subtask by traversing a loop mechanism of unscheduled subtasks.

[0060] Based on this, firstly, the compiler selects ready subtasks from multiple subtasks. In this example, the compiler further identifies asynchronous direct memory access (DMI) subtasks and / or critical path subtasks among the ready subtasks, prioritizing these for scheduling. Next, hardware constraint checks are performed on the scheduled ready subtasks. Specifically, the compiler sequentially checks whether the tightly coupled memory space is sufficient. If the tightly coupled memory space meets the requirements, it further checks whether the register / direct memory access controller has available slots. If all the above resource capacity constraints are met simultaneously, the ready subtask can be pushed forward; if any resource requirement is not met, the subtask is postponed and kept in the ready subtask queue, awaiting another hardware constraint check after resource release.

[0061] Once a ready subtask passes the hardware constraint check, it will enter the execution plan generation phase. The compiler calculates the earliest start time for the ready subtask and synchronously updates the timeline of the corresponding hardware execution unit, completing the arrangement of the subtask in the time slot. During this arrangement process, the compiler also determines whether the subtask has any data dependencies. If so, it inserts synchronization barrier markers, such as fences or barriers, to ensure the correctness of data read and write operations upstream and downstream of the task pipeline.

[0062] After iterating through all unscheduled subtasks and generating execution plans, the target program timeline will be output. Figure 1C The diagram in the image shows the "sProgram schedule," which represents the construction result of the kernel task pipeline. It records information such as the start and stop timings and synchronization boundaries of each subtask on each hardware execution unit. This target program schedule provides a basis for the generation of subsequent target operator code.

[0063] The following combination Figure 1D From the perspective of hardware execution units, a specific example is used to illustrate the construction of an in-core task pipeline.

[0064] Figure 1D The solid rectangles in the diagram represent the task regions of different hardware execution units within a multi-core heterogeneous processor, including: the direct memory access unit task region, the matrix computation unit task region, and the vector computation unit task region.

[0065] Taking the FlashAttention operator task code included in the target operator code as an example, when the compiler parses the code for loading query matrix data (load_Q), loading key matrix data task (load_K), loading value matrix data task (load_V), and the final result write-back task (store_O), it allocates them to the direct memory access unit (DMU). Figure 1D The compiler allocates the code for tasks involving high-density multiply-accumulate operations to the matrix computation unit (DMA unit). Figure 1D The “Matrix unit” in the text forms a matrix unit task, such as Figure 1D The task involves multiplying the query matrix and key matrix (matmul_QK) and multiplying the attention weights and value matrix (matmul_PV). For computation nodes involving reduction or nonlinear transformations, the compiler assigns them to vector computation units (…). Figure 1D "Vector unit" in the context of vector units, such as Figure 1D The normalized exponential function task (softmax) is shown in the figure.

[0066] Figure 1DThe dashed and solid arrows in the diagram represent the forward and backward data dependencies between subtasks, respectively. They are also crucial for the compiler to determine resource timing and insert synchronization barriers when building the kernel task pipeline. For example, Figure 1D The tasks for loading the query matrix data and the key matrix data are indicated by dashed arrows pointing to the multiplication task of the query matrix and the key matrix. This shows that before the matrix computation unit officially starts the multiplication operation, it relies on the direct memory access unit to completely move the corresponding data block to the on-chip tightly coupled memory. After the multiplication task of the query matrix and the key matrix is ​​completed, its result flows to the normalization exponential function task via solid arrows. Subsequently, the normalization result, together with the output of the load value matrix data task, flows to the next multiplication task. This topology connection clearly guides the compiler to generate synchronization barriers at the corresponding instruction junctions to prevent subsequent hardware units from reading dirty data that has not yet been updated, thus ensuring the accuracy of the calculation results.

[0067] In traditional single-scheduling models, the aforementioned hardware execution units often exhibit a sequential waiting state, resulting in significant idle hardware computing power. However, through the kernel task pipeline construction in this application embodiment, the compiler fully leverages the potential for parallel operation of various hardware execution units. Figure 1D As shown, when the matrix computation unit processes the matrix multiplication task of the current data block (Tile), the compiler, through pre-scheduling, enables the direct memory access unit to utilize its asynchronous bus transmission characteristics to synchronously prefetch the key matrix data or value matrix data required for the next data block in the background. Simultaneously, the vector computation unit can also process the normalization task of the previous data block in parallel. This significantly improves the overall hardware utilization of multi-core heterogeneous processors when facing complex fusion operators, and enhances the execution efficiency of the processor cores.

[0068] Through the above subtask selection, hardware constraint check and execution plan generation, efficient subtask kernel scheduling under hardware resource constraints is achieved.

[0069] Step S108: Based on the processor core allocation results and the construction results of the in-core task pipeline, generate the corresponding target operator code.

[0070] After completing cross-core task allocation (inter-core scheduling) and intra-core task pipeline construction (intra-core scheduling), the compiler has generated relatively detailed subtask allocation and scheduling information. Based on this, the compiler can transform this abstract subtask allocation and scheduling information into target operator code that can be recognized and executed by real physical hardware.

[0071] In one example, if a multi-core heterogeneous processor uses a RISC-V architecture, generating the target operator code may include multiple processes such as instruction selection, register allocation, instruction scheduling, parallel optimization, and code generation to match the task pipeline with the RISC-V instruction set.

[0072] In a specific implementation, one optional approach is to generate the corresponding target operator code based on the processor core allocation result and the construction result of the in-core task pipeline. This may include: generating the corresponding target operator code based on the processor core allocation result and the construction result of the in-core task pipeline, according to the data structure used to describe the task and / or subtask, and the data structure used to describe the hardware execution unit.

[0073] The data structures used to describe tasks and / or subtasks are pre-defined, structured records within the compiler used to carry metadata about the subtasks. These records may contain information such as the computation type of the subtask, the description of the data tensor being operated on, the tile shape parameters, the type of the target hardware execution unit, its dependent preceding subtasks, and subsequent subtasks. The data structures for hardware execution units describe the static capabilities of various physical execution units within the processor core, recording information such as the type of the hardware execution unit, execution latency, throughput, and asynchronous execution characteristics.

[0074] Through the above data structure, the compiler decouples the scheduling logic from the code generation logic, enabling the adaptation and assignment of values ​​to target operator code that matches the new hardware, regardless of the model or configuration of the multi-core heterogeneous processor. This makes it more cross-platform and portable.

[0075] Figure 1E An exemplary task / subtask data structure sTask and an exemplary hardware execution unit data structure sEU are shown.

[0076] Depend on Figure 1E As can be seen, the subtask data structure sTask contains: a subtask identifier ( Figure 1E (The text indicates that the subtask is "stringname"), and the corresponding operation type of the subtask. Figure 1E The field, denoted as "operation_type" in the diagram, is the target unit type field used to indicate which type of hardware execution unit should execute the subtask. Figure 1E The diagram shows "target_sEU" and the Tile shape field used to specify the size of the data chunks. Figure 1E The diagram in the image is "tile_shape", a TCM requirement field. Figure 1E (Illustrated as "tcm_bytes"), register / DMA requirement field ( Figure 1E(Illustrated as "register_count"), information about the preceding subtasks it depends on. Figure 1E The text indicates that the text refers to "predecessors" and information about subsequent subtasks. Figure 1E The text indicates the status of subtasks (represented by "successors"). Figure 1E Information such as "state" (as shown in the diagram) and the information of its corresponding execution function ( Figure 1E The text is a mix of seemingly unrelated phrases and symbols, making it difficult to translate coherently. It appears to be a collection

[0077] like Figure 1E As shown, the operation type "operation_type" corresponding to the subtask can include: DMA_Load / Store (DMA read / store), Matrix_MatMul (matrix multiplication), Vector_Elementwise (vector element-wise operation), Vector_Reduce (vector reduction), and Barrier / Fence (synchronization barrier). The subtask's state "state" can include: Pending, Ready, Running, and Complete. The target unit type "target_sEU" can include: Vector (scalar or vector computation unit), Matrix (matrix computation unit), and DMA (direct memory access unit).

[0078] The hardware execution unit data structure sEU can contain: the type of hardware execution unit ( Figure 1E The text indicates a "type" and execution delay. Figure 1E The diagram shows "latency_cycles" and throughput. Figure 1E (Illustrated as "throughput") and asynchronous execution characteristics ( Figure 1E Information such as "is_async" is shown in the image.

[0079] The connection between sTask and sEU is established through the target unit type field "target_sEU" in sTask. Through the value of this field, the compiler can find a matching hardware execution unit based on the type information of the hardware execution unit in the hardware execution unit data structure.

[0080] Based on the above data structure, the generation of target operator code can be made simpler and more efficient.

[0081] After obtaining the target operator code, in one alternative approach, the heterogeneous task stream contained in the target operator code can be allocated to the matching hardware execution unit for execution based on the type of task that each hardware execution unit can execute.

[0082] The target operator code, oriented towards different types of hardware execution units, forms a collection of multiple parallel or alternating instruction sub-sequences, i.e., a heterogeneous task flow. For each hardware execution unit, there is a specific task type it is suited to handle, such as matrix operations, scalar or vector operations, or data transfer. Based on this, subtasks with different types can be mapped to hardware execution units whose types match theirs.

[0083] like Figure 1F As shown, the matrix computation unit can execute the matrix multiplication subtask "sTask: MatMul"; the scalar or vector computation unit can execute scalar or vector subtasks. Figure 1F The diagram only briefly illustrates the vector computation unit's vector subtask "sTask: Softmax" for handling operations such as normalization; the Direct Memory Access (DMA) unit can execute the data loading and storage subtask "sTask: Load / Store". Furthermore, as... Figure 1F As shown, matrix computation units are constrained by accumulator constraints, scalar or vector computation units are constrained by register constraints, and direct memory access units are constrained by descriptor constraints. Simultaneously, these hardware execution units are also connected to a shared resource, namely a tightly coupled memory (TCM) based on tile-based storage, and are therefore constrained by the TCM capacity. During the subtask allocation phase, the compiler follows... Figure 1F The binding relationship in the code allows matrix multiplication subtasks to be submitted to the matching matrix computation unit, scalar or vector subtasks to be distributed to the scalar or vector computation unit, and data loading and storage subtasks to be executed by the direct memory access unit.

[0084] By matching heterogeneous task flows with hardware execution units, the reduction in execution efficiency or hardware idleness caused by instruction delivery errors is reduced, enabling all heterogeneous computing power within the multi-core heterogeneous processor to perform to its full potential, thereby improving the overall throughput of the target operator code on the physical chip.

[0085] This embodiment leverages the physical structure of a multi-core heterogeneous processor to acquire a task graph containing data dependencies, enabling the compiler to understand the computation graph topology of the operator to be executed from a global perspective. Then, based on a preset scheduling partitioning strategy, tasks are macroscopically allocated across multiple processor cores, achieving cross-core scheduling. In the micro-scheduling stage for a single processor core, the compiler no longer relies on the implicit scheduling mechanism of the underlying hardware. Instead, it explicitly constructs intra-core task pipelines for different types of hardware execution units, using tightly coupled memory capacity constraints and direct memory access capacity constraints as boundary conditions. Furthermore, based on the two-level scheduling decisions (cross-core and intra-core), the compiler directly maps and generates the target operator code. Thus, cross-core scheduling allocation and explicit construction of intra-core task pipelines are achieved through software during the compilation stage, decoupling the deep binding between upper-level algorithm code and the underlying hardware architecture. Moreover, it eliminates the need for manual management of complex pipeline design and adaptation to the underlying hardware, significantly reducing development costs. Furthermore, the automatic construction of task pipelines based on capacity constraints of tightly coupled memory and direct memory access can effectively utilize hardware resources, improve the utilization rate and overall execution performance of various hardware execution units in multi-core heterogeneous processors, and thus improve the overall performance of compilation and scheduling.

[0086] This application also provides a compiler, such as... Figure 2A As shown, it includes: a graph capture layer 202, a pipelined compilation layer 204, a dynamic code generation layer 206, and a task scheduling and execution layer 208.

[0087] in: Graph capture layer 202 is used to capture the task graph to be compiled, which contains data dependencies.

[0088] The pipeline compilation layer 204 is used to divide the tasks corresponding to the task graph into multiple subtasks based on a preset scheduling and partitioning strategy, and to allocate the multiple subtasks to the multiple processor cores; for each processor core to which the subtasks are allocated, an intra-core task pipeline is constructed for the subtasks allocated to each processor core based on the capacity constraints of tightly coupled memory and direct memory access.

[0089] The dynamic code generation layer 206 is used to generate the corresponding target operator code based on the processor core allocation results and the construction results of the in-core task pipeline.

[0090] The task scheduling and execution layer 208 is used to schedule and allocate tasks in the target operator code to matching hardware execution units for execution. The hardware execution units include various types of hardware execution units, which are located in the processor core. The processor core is located in a multi-core heterogeneous processor, and the processor core also includes tightly coupled memory.

[0091] The functional implementation of each layer in the above compiler can be referred to the description of the relevant parts in the aforementioned compiler scheduling method embodiment, and has corresponding beneficial effects, which will not be repeated here.

[0092] Below, in conjunction with Figure 2B The following example illustrates the implementation of each layer of functionality in the compiler.

[0093] The compiler in this example works with multi-core heterogeneous processors, such as RISC-V-based multi-core heterogeneous processors, which include multiple processor cores, each of which includes tightly coupled memory and various types of hardware execution units.

[0094] like Figure 2B As shown, the graph capture layer 202 is the compiler's front-end entry point. It first receives operators or kernel programs written based on a specific IR; then, it calls the corresponding capture interfaces, such as StreamBeginCapture / EndCapture, to intercept the computation data stream; next, it performs semantic analysis on each operation node in the intercepted data stream; finally, it generates a runtime directed acyclic graph (DAG), i.e., the aforementioned task graph to be compiled. By intercepting and parsing high-level operator code, the graph capture layer 202 extracts all computation and transport operations, transforming them into a task graph with a data-dependent edge topology, enabling the compiler to accurately grasp the execution order and inherent dependencies of all operators at runtime.

[0095] The pipelined compilation layer 204 executes dedicated task partitioning, dependency construction, two-level scheduling (inter-core and intra-core), and program generation sequentially based on the task graph. Specifically, dedicated task partitioning breaks down coarse-grained operators or kernel programs into fine-grained sTasks subtasks such as Load, Vector, and Matrix; dependency construction establishes data dependency edges between subtasks; two-level scheduling enables processor core allocation and intra-core task pipeline construction; and pipelined program generation schedules subtasks on a timetable and automatically inserts synchronization barriers. Through the pipelined compilation layer 204, the scheduling logic, originally implicitly managed by hardware, is transformed into explicit software scheduling at compile time. This achieves deep execution overlap between computation and data movement while ensuring the safety of processor core memory resources, thereby improving the overall computation and memory access throughput of multi-core heterogeneous processors.

[0096] In this example, the dynamic code generation layer 206 utilizes the RISC-V code builder, combined with the RISC-V instruction set and its extensions, to perform code transformation based on the kernel task pipeline, generating target operator code. The RISC-V instruction set includes standard RISC-V instructions such as addi / ld / sd; RISC-V extended instructions include: Vector extensions for vector loading, storage, and multiplication-addition operations, such as vle / vse / vfmacc; Matrix extensions for matrix multiplication and addition operations, such as mmul / madd; and DMA custom instructions for controlling the pushing and copying of data in direct memory access, such as dmaPush / dmaCopy. Through the dynamic code generation layer 206, a seamless translation from high-level abstract scheduling intent to low-level concrete hardware behavior is achieved, enabling complex task pipelines to be executed by real hardware execution units, and giving the target operator code the flexibility to adapt to different hardware execution units.

[0097] The task scheduling and execution layer 208 is oriented towards the underlying hardware. Figure 2B The diagram illustrates processor cores arranged sequentially: Core 0, Core 1, Core 2, ... Core 15. Each processor core contains vector computation units, matrix computation units, and direct memory access units. Furthermore, the direct memory access units within each processor core interact with external DDR storage. The task scheduling and execution layer 208, relying on the physical topology of the multi-core heterogeneous processors, dispatches the heterogeneous task flow contained in the target operator code to hardware execution units with corresponding processing capabilities. This allows them to collaborate with external storage to complete the corresponding data processing or transfer work, supported by tightly coupled memory. This effectively releases the physical computing potential of each hardware execution unit, avoiding idle computing power and performance degradation caused by mismatch between instructions and hardware capabilities.

[0098] The compiler in this example achieves efficient, automated, and low-intervention-requirement operator code generation and scheduling execution for multi-core heterogeneous processors through a clear functional hierarchy, providing a systematic toolchain support for operator development from graph capture to code generation.

[0099] Furthermore, embodiments of this application also provide an electronic device. For example... Figure 3 As shown, the electronic device includes the aforementioned compiler and the aforementioned multi-core heterogeneous processor. The multi-core heterogeneous processor includes multiple processor cores, each core comprising tightly coupled memory and various types of hardware execution units (not shown in the figure). The compiler, through its task scheduling execution layer, schedules and allocates tasks in the target operator code to the hardware execution units in the multi-core heterogeneous processor for execution.

[0100] The implementation of the compiler's functions in this embodiment, as well as its scheduling implementation for multi-core heterogeneous processors, can refer to the relevant descriptions in the foregoing embodiments and have corresponding beneficial effects, which will not be repeated here.

[0101] Furthermore, it should be noted that the user-related information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to sample data used for training the model, data used for analysis, stored data, displayed data, etc.) involved in the embodiments of this application are all information and data authorized by the user or fully authorized by all parties. Moreover, the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0102] It should be noted that, depending on the implementation needs, the various components / steps described in the embodiments of this application can be broken down into more components / steps, or two or more components / steps or parts of the operation of components / steps can be combined into new components / steps to achieve the purpose of the embodiments of this application.

[0103] The above embodiments are only used to illustrate the embodiments of this application, and are not intended to limit the embodiments of this application. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the embodiments of this application. Therefore, all equivalent technical solutions also fall within the scope of the embodiments of this application, and the patent protection scope of the embodiments of this application should be defined by the claims.

Claims

1. A compilation scheduling method applied to a multi-core heterogeneous processor comprising multiple processor cores, each processor core including tightly coupled memory and various types of hardware execution units; the method comprising: Obtain the task graph to be compiled, which contains data dependencies; Based on a preset scheduling and partitioning strategy, the tasks corresponding to the task graph are divided into multiple subtasks, and the multiple subtasks are assigned to the multiple processor cores; For each processor core to which a subtask is assigned, an intra-core task pipeline is constructed based on the capacity constraints of tightly coupled memory and direct memory access. Based on the allocation results of the processor cores and the construction results of the task pipeline within the cores, the corresponding target operator code is generated.

2. The method of claim 1, wherein, The preset scheduling and segmentation strategy divides the tasks corresponding to the task graph into multiple sub-tasks, including: The task corresponding to the task graph is divided into multiple subtasks based on at least one of the following scheduling partitioning strategies, wherein the scheduling partitioning strategies include: A first strategy is used to instruct the parallel partitioning of the tasks corresponding to the task graph into multiple subtasks that match the processor cores, for parallel execution in the multiple processor cores. A second strategy is used to instruct the task corresponding to the task graph to be divided into multiple subtask pipelines so that the multiple subtask pipelines can be executed in parallel across the multiple processor cores. A third strategy for selecting a strategy from the first and second strategies based on the memory access ratio of the task corresponding to the task graph.

3. The method of claim 2, wherein, If the scheduling segmentation strategy includes the first strategy, the second strategy, and the third strategy, then the step of dividing the task corresponding to the task graph into multiple sub-tasks based on the preset scheduling segmentation strategy includes: The first strategy, the second strategy, and the third strategy are used respectively to perform multi-task segmentation on the task graph to obtain the corresponding first sub-task set, second sub-task set, and third sub-task set; The execution performance of the subtasks in the first subtask set, the second subtask set, and the third subtask set is evaluated. Based on the evaluation results, a target subtask set is selected from the first subtask set, the second subtask set, and the third subtask set, and multiple subtasks in the target subtask set are determined as the final target subtasks.

4. The method of claim 2, wherein, If the scheduling segmentation strategy includes the first strategy, the second strategy, and the third strategy, then the step of dividing the task corresponding to the task graph into multiple sub-tasks based on the preset scheduling segmentation strategy includes: Determine the execution priority among the first strategy, the second strategy, and the third strategy; According to the execution priority from high to low, the tasks corresponding to the task graph are divided in sequence until the execution performance of the multiple sub-tasks obtained by the division meets the preset standard, or all strategies are executed. Multiple subtasks that meet preset criteria can be identified as target subtasks, or multiple subtasks obtained based on the lowest execution priority strategy can be identified as target subtasks.

5. The method of claim 2, wherein, If the scheduling segmentation strategy includes the first strategy and the second strategy, then the step of dividing the task corresponding to the task graph into multiple sub-tasks based on the preset scheduling segmentation strategy includes: Using the first strategy, the task corresponding to the task graph is divided into multiple sub-tasks. For each subtask, the second strategy is used to perform task pipeline segmentation, resulting in multiple pipeline subtasks; Based on the multiple subtasks and the multiple pipeline subtasks corresponding to each subtask, the final target subtask is obtained.

6. The method of claim 1, wherein, For each processor core assigned a subtask, based on the capacity constraints of tightly coupled memory and direct memory access, an in-core task pipeline is constructed for the subtasks assigned to each processor core, including: For each processor core, select ready subtasks from the subtasks assigned to it; Based on the capacity constraints of the tightly coupled memory, determine whether the tightly coupled memory has memory space to execute the ready subtask, and based on the capacity constraints of the direct memory access, determine whether the direct memory access has available slots; If both the memory space and the available slots are available, then an in-core task pipeline is generated for the ready subtask.

7. The method of claim 6, wherein, If both the memory space and the available slots are available, then generating an in-core task pipeline for the ready subtask includes: If both the memory space and the available slots are available, then determine whether the ready subtask is a subtask with data dependencies. If so, an in-core task pipeline is generated for the ready subtask, and a synchronization barrier is inserted into the in-core task pipeline.

8. The method of claim 6, wherein, After selecting ready subtasks from the assigned subtasks, the method further includes: Identify asynchronous direct memory access subtasks or critical path subtasks from multiple ready subtasks, and prioritize scheduling the asynchronous direct memory access subtasks or critical path subtasks.

9. The method of any one of claims 1-8, wherein, The generation of corresponding target operator code based on the processor core allocation result and the construction result of the in-core task pipeline includes: Based on the processor core allocation results and the construction results of the intra-core task pipeline, corresponding target operator code is generated according to the data structure used to describe the task and / or the subtask, and the data structure used to describe the hardware execution unit.

10. The method of claim 1, wherein, The method further includes: Based on the type of tasks that each hardware execution unit can execute, the heterogeneous task stream contained in the target operator code is allocated to the matching hardware execution unit for execution.

11. The method of claim 1, wherein, The multi-core heterogeneous processor is a RISC-V heterogeneous accelerator, and the hardware execution unit includes at least a scalar or vector computation unit, a matrix computation unit, and a direct memory access unit.

12. A compiler, comprising: The graph capture layer is used to capture the task graph to be compiled, which contains data dependencies. The pipelined compilation layer is used to divide the tasks corresponding to the task graph into multiple subtasks based on a preset scheduling and partitioning strategy, and to allocate the multiple subtasks to the multiple processor cores. For each processor core to which a subtask is assigned, an intra-core task pipeline is constructed based on the capacity constraints of tightly coupled memory and direct memory access. A dynamic code generation layer is used to generate corresponding target operator code based on the allocation results of the processor cores and the construction results of the intra-core task pipeline; The task scheduling and execution layer is used to schedule and allocate the tasks in the target operator code to the matching hardware execution units for execution; The hardware execution unit includes various different types of hardware execution units, which are located in the processor core. The processor core is located in a multi-core heterogeneous processor, and the processor core also includes tightly coupled memory.

13. An electronic device comprising the compiler and processor as described in claim 12, wherein the processor is a multi-core heterogeneous processor comprising multiple processor cores, each processor core comprising tightly coupled memory and multiple different types of hardware execution units; The compiler, through the task scheduling and execution layer, schedules and allocates the tasks in the target operator code to the hardware execution units in the multi-core heterogeneous processor for execution.

Images