Tensor processor compiling method and device, compiler and storage medium

Abstract

The application is suitable for the technical field of computers, and provides a tensor processor compiling method and device, a compiler and a storage medium, the method comprising: obtaining a persistent task primitive, an event primitive and a page allocation primitive, and constructing an extended block-level intermediate representation; performing a block preprocessing operation on the extended block-level intermediate representation, and constructively solving a page-physical page binding result; according to the page-physical page binding result, performing cross-task direct memory access descriptor static lifting iteration to obtain a cross-task software pipeline; uniformly solidifying the page-physical page binding result, the cross-task software pipeline, an event counter register number in an event node and a synchronization threshold into an immediate number, constructing a static dispatch table, and encoding to generate a binary file of a solidified execution kernel. The application can optimize the compiling and executing process of the tensor processor, reduce hardware runtime scheduling overhead, and improve the parallel efficiency of on-chip storage and computing units.

Description

Technical Field

[0001] This invention belongs to the field of computer technology, and particularly relates to a compilation method, apparatus, compiler, and storage medium for a tensor processor. Background Technology

[0002] With the rapid iteration and evolution of large language model inference and training workloads, the industry has successively launched a number of dedicated processor chips for tensor computing. These chips share common characteristics in their architecture, such as high core density, on-chip paging memory, independent and parallel computing units and data transport channels, and synchronization relying on event signal mechanisms.

[0003] Unlike general-purpose graphics processing units (GPUs), these chips often lack mature high-level programming models. Dedicated tensor processors lack mature and complete high-level programming development models. Whether the theoretical computing power of the hardware can be fully released depends on the extent to which the compiler mines and utilizes the static information of the model during the compilation phase. Current mainstream technical solutions generally postpone core decisions such as resource allocation, task scheduling, and resource arbitration to the real-time processing stage of the chip. The resulting control flow overhead will significantly reduce the chip's effective computing throughput and severely restrict the utilization of hardware computing power. Summary of the Invention

[0004] In view of this, embodiments of the present invention provide a compilation method, apparatus, compiler, and storage medium for a tensor processor, so as to optimize the compilation and execution process of the tensor processor, reduce hardware runtime scheduling overhead, and improve the parallel efficiency of on-chip storage and computing units.

[0005] In a first aspect, embodiments of the present invention provide a method for compiling a tensor processor, comprising: Obtain persistent task primitives, event primitives, and page allocation primitives, and construct them into an extended block-level intermediate representation containing task nodes, event nodes, and page allocation nodes; Perform block preprocessing on the extended block-level intermediate representation and constructively solve for the physical page identifiers that match each page allocation node to obtain the page-physical page binding results; Based on the page-physical page binding result, a static lift-up traversal of the cross-task direct memory access descriptor is performed to obtain the cross-task software pipeline. The page-physical page binding result, the cross-task software pipeline, the event counter register number and synchronization threshold in the event node are uniformly solidified into immediate values ​​and a static dispatch table is constructed. The binary file of the solidified execution kernel is then encoded and generated.

[0006] In one possible implementation, the constructive solution of the physical page identifiers matching each page allocation node includes: Based on the on-chip storage capacity constraints and page monotonic usage constraints of each page allocation node, the physical page identifiers matching each page allocation node are structurally solved using interval coloring plus backtracking or integer linear programming as the solution paradigm. If the solution fails, the process returns to performing the block preprocessing operation on the extended block-level intermediate representation to adjust the computation block size corresponding to each task node according to the preset block adjustment parameters, until the physical page identifier matching each page allocation node is obtained.

[0007] In one possible implementation, the step of performing a static lift-up traversal of cross-task direct memory access descriptors based on the page-physical page binding result to obtain a cross-task software pipeline includes: Traverse each task node and rearrange the direct memory access preload instructions and matrix calculation instructions within the task node; By inserting synchronization barrier instructions between the direct memory access channel and the matrix computation unit, a cross-task software pipeline with overlapping timings between the preloading phase and the matrix computation phase of adjacent tasks is constructed.

[0008] In one possible implementation, the encoding generates a binary file containing a fixed execution kernel, including: Based on the immediate value and the static dispatch table, a dispatch instruction segment and a working instruction segment are encoded to obtain the binary file of the solidified execution kernel; The dispatch instruction segment is used to load the dispatch core physical instruction storage unit of the tensor processor, and the working instruction segment is used to load the working core physical instruction storage unit of the tensor processor. The dispatch core and the working core are coupled through an on-chip event counter register group.

[0009] In one possible implementation, the encoding process for generating the dispatch instruction segment and the work instruction segment further includes: Configure different startup modes for each task node and encode them into the working instruction segment; The startup modes include instant startup mode and pre-startup mode.

[0010] In one possible implementation, performing block preprocessing on the extended block-level intermediate representation includes: Calculate the size of the computation block corresponding to each task node; Iterate through all event nodes, merge redundant and duplicate synchronous events, and synchronously update the binding relationship between task nodes and event nodes. For a topology structure where a single task node is associated with multiple events, a dependency normalization operation is performed to limit each task node to triggering at most one event node. For the normalized acyclic task dependency graph, a breadth-first traversal is performed to convert the mesh topology into an ordered linear task sequence, and standardized topology data is output; wherein, the standardized topology data is used to constructively solve the physical page identifiers for matching the allocation nodes of each page.

[0011] In one possible implementation, the dependency normalization operation performed on the topology structure of a single task node associating with multiple events, limiting each task node to triggering at most one event node, includes: If a single task node triggers multiple event nodes simultaneously, then a relay event node and an empty task node corresponding to each of the multiple event nodes are introduced. Bind the task node to the relay event node; Each empty task node is configured to use the aforementioned relay event node as a prerequisite trigger condition, and each node is bound to the corresponding event node for output.

[0012] Secondly, embodiments of the present invention provide a compilation apparatus for a tensor processor, comprising: The acquisition module is used to acquire persistent task primitives, event primitives, and page allocation primitives, and construct them into an extended block-level intermediate representation containing task nodes, event nodes, and page allocation nodes; The processing module is used to perform block preprocessing operations on the extended block-level intermediate representation and constructively solve the physical page identifiers that match each page allocation node to obtain the page-physical page binding results. The lift-up module is used to perform static lift-up traversal of cross-task direct memory access descriptors based on the page-physical page binding results, so as to obtain the cross-task software pipeline. The encoding module is used to uniformly solidify the page-physical page binding result, the cross-task software pipeline, the event counter register number and synchronization threshold in the event node into immediate values, construct a static dispatch table, and encode and generate a binary file for solidifying the execution kernel.

[0013] Thirdly, embodiments of the present invention provide a compiler, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the methods described in the first aspect or any possible implementation thereof.

[0014] Fourthly, embodiments of the present invention provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method described in the first aspect or any possible implementation thereof.

[0015] The beneficial effects of the embodiments of the present invention compared with the prior art are as follows: This invention first constructs a unified extended block-level intermediate representation based on three primitives: task, event, and page allocation. This representation can fully carry all-dimensional information about tensor computation, synchronization interaction, and memory allocation, providing a well-organized data foundation for subsequent unified topology optimization and resource allocation. Then, block preprocessing is performed, and based on this, the physical page identifiers matching each page allocation node are structurally solved to obtain page-physical page binding results. This makes the relationship between block size and on-chip capacity structurally solvable at compile time, fundamentally eliminating the need for dynamic acquisition and release of pages at runtime. Next, a static lift-up traversal of cross-task direct memory access descriptors is performed based on the page binding results, pre-constructing a cross-task software pipeline during the compilation phase. Finally, the memory binding relationships, pipeline arrangement rules, and event synchronization parameters are all solidified into compile-time immediate values ​​and a static dispatch table is generated, outputting a solidified execution kernel binary file. During hardware runtime, only this binary file needs to be processed sequentially, without the need for dynamic resource allocation, instruction reordering, or task scheduling calculations. This significantly reduces the runtime control overhead of the tensor processor and improves overall computational throughput performance. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a schematic diagram illustrating the implementation flow of a tensor processor compilation method provided in an embodiment of the present invention; Figure 2 This is a schematic diagram illustrating the implementation flow of a tensor processor compilation method provided in another embodiment of the present invention; Figure 3 This is a schematic diagram of a dependency normalization operation provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of a binary hardware carrying configuration provided in an embodiment of the present invention; Figure 5 This is a binary event-driven execution timing diagram provided in an embodiment of the present invention; Figure 6 This is a schematic diagram of cross-task software pipeline and immediate addressing paging memory carrying provided by an embodiment of the present invention; Figure 7 This is a schematic diagram of a compilation apparatus for a tensor processor provided in an embodiment of the present invention; Figure 8 This is a schematic diagram of a compiler provided in an embodiment of the present invention. Detailed Implementation

[0018] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of the invention. However, those skilled in the art will understand that the invention can be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods are omitted so as not to obscure the description of the invention with unnecessary detail.

[0019] To illustrate the technical solution described in this invention, specific embodiments are described below.

[0020] In their engineering practice, the inventors of this application discovered that the compilation methods for dedicated tensor processors in related technologies have the following shortcomings: (1) The programming model lacks native primitives for on-chip paging storage and persistent task graphs: resident kernel solutions generally enter the compilation process directly from the computation graph, lacking the front-end entry point of "block-page-event" primitives and page allocation constraints; (2) The division of responsibilities between the compilation phase and the runtime phase is unclear, and the decision-making time is generally too late: a lot of information that could be determined at compile time, such as which physical page a buffer uses, when a direct memory access (DMA) descriptor is elevated, and which event signal triggers which instruction, is postponed to runtime for repeated solutions, thus introducing a significant control path overhead. (3) On-chip paging storage lacks compile-time one-to-one mapping capability: none of the existing solutions have a closed loop where the compiler automatically feeds back a request to adjust the block size when the capacity exceeds the limit; (4) Cross-task DMA and computational overlap are generally achieved by runtime dynamic decomposition of tasks, and the compiler fails to constructively complete cross-operator pipeline orchestration at the static level. (5) The compiled product itself lacks assertable structural features.

[0021] Therefore, embodiments of the present invention provide a compilation method for tensor processors. This method receives the developer's intent through the "block-page-event" three-dimensional abstraction of the front-end compilation primitives, and completes all decisions such as on-chip paging memory binding, cross-task DMA descriptor hoisting, event synchronization solidification, and dispatch item burning in one go during the compilation period, producing a solidified execution kernel binary file with its own assertable structural characteristics. The hardware environment only performs sequential consumption of this binary, and does not hold any queue structure, maintain task state machine, or expose dynamic allocation interface during runtime. This method is particularly suitable for inference and training acceleration scenarios of deep learning models such as large language models and hybrid expert models on dedicated tensor processors (core-intensive, on-chip paging memory, independent and parallelizable computing units and DMA channels, and non-general-purpose GPU accelerators with synchronization relying on event signaling mechanisms).

[0022] Figure 1 This is a schematic diagram illustrating the implementation flow of a tensor processor compilation method provided in an embodiment of the present invention, including: Step S101: Obtain persistent task primitives, event primitives, and page allocation primitives, and construct them into an extended block-level intermediate representation containing task nodes, event nodes, and page allocation nodes.

[0023] This step is used to obtain three types of standardized persistence primitives and complete unified abstract modeling. This embodiment provides three basic primitives (PersistentTask, Event, and PageAlloc) as the unified front-end input for the tensor processor chip-specific compilation process. Each of the three primitives carries independent dimensions of hardware semantics: The persistent task primitive PersistentTask is used to represent an independent computational work unit with computational blocks as the granularity. It corresponds to the block dimension in the "block-page-event" three-dimensional abstract system in this embodiment and serves as the runtime carrier of computational logic. The Event primitive is used to represent a synchronization signal node. It is activated only after all its preceding associated tasks have been completed, thus realizing the synchronization waiting logic between multiple tasks. The PageAlloc primitive corresponds to on-chip memory paging allocation logic whose lifetime can be fully determined during the compile phase. This primitive enforces a monotonic use constraint flag, ensuring that physical pages corresponding to the same page allocation only perform monotonic reads and writes throughout their complete lifetime, disallowing repeated writes across tasks. For example, if a PageAlloc buffer is used as a DMA write target, this buffer can only be read by subsequent computation tasks during its lifetime and cannot be repeatedly overwritten by other DMA tasks, forming a monotonic data stream of single writes, multiple reads, or single writes and single reads. This monotonic read-write characteristic allows the compiler to uniquely determine the data producers and consumers of each physical page during the subsequent page binding solution phase, saving the hardware runtime overhead of maintaining a page allocation state machine.

[0024] After obtaining the above three types of primitives, the compiler performs unified parsing on the computation, synchronization, and memory allocation information carried by the primitives to construct a standardized extended block-level intermediate representation. This intermediate representation contains three types of one-to-one mapping node structures: TaskNode for persistent task primitives, EventNode for event primitives, and PageAllocNode for page allocation primitives. These three types of nodes fully carry all semantic information of block computation, task synchronization, and on-chip memory allocation, providing a unified and standardized data carrier for subsequent processing.

[0025] Step S102: Perform block preprocessing on the extended block-level intermediate representation and constructively solve for the physical page identifiers that match the page allocation nodes to obtain the page-physical page binding results.

[0026] The first part is the block preprocessing operation: The compiler traverses all task nodes, event nodes, and page allocation nodes within the extended block-level intermediate representation, and performs four regularization processes in sequence: First, it performs block size inference, combining the overall size of the model tensor and the on-chip storage hardware limit of the tensor processor to estimate the memory usage scale corresponding to each computation block, and obtain the initial memory requirements of each page allocation node; second, it performs event fusion, removing redundant and repeated synchronous event nodes in the topology, synchronously updating the binding association between tasks and events, and simplifying the overall synchronous topology; third, it performs dependency normalization, for complex topologies where a single task node is associated with multiple input / output events, adding transit event nodes and empty task nodes to complete equivalent transformations, uniformly constraining all task nodes to depend on at most one event and trigger at most one event; finally, it performs breadth-first linearization, performing breadth-first traversal on the normalized acyclic task dependency graph, converting the mesh topology into an ordered linear task sequence, outputting a regularized and standardized intermediate representation topology, eliminating the solution complexity caused by multi-event coupling, and providing regularized input for subsequent page resource allocation.

[0027] The second part involves constructing and solving the physical page identifier: using each page allocation node as the solution variable, and combining two types of core constraints for calculation. The first is the inherent on-chip memory capacity constraint, which limits the total number of physical pages simultaneously occupied by all pages to not exceeding the maximum number of pages in the hardware. The second is the lifecycle interval constraint and monotonic read / write constraint inherent in the page allocation node, which determines whether a page can be reused based on the timing occupancy interval of each buffer. If the lifecycles of multiple page allocation nodes do not overlap, the same physical page identifier can be reused; if the lifecycles overlap, independent physical page identifiers must be allocated.

[0028] In one possible implementation, the solution paradigm could employ interval coloring with backtracking or integer linear programming. Integer linear programming can achieve globally optimal page allocation, but it significantly increases compilation time for medium-sized task graphs; interval coloring with backtracking, on the other hand, offers linear time complexity in engineering terms. Both can complete the one-to-one binding of block buffers to physical pages at compile time.

[0029] In one possible implementation, if the solution fails, the process returns to perform a block preprocessing operation on the extended block-level intermediate representation to adjust the computation block size corresponding to each task node according to the preset block adjustment parameters, until the physical page identifier matching each page allocation node is obtained.

[0030] For example, if the total page occupancy exceeds the on-chip storage capacity during the solution process, the process flows back to the block size inference step to adjust the block size, iterating until all constraints are met, and finally determining the unique physical page identifier for each page allocation node, generating the page-physical page binding result. Here, the size adjustment method can be: iteratively backtracking from large to small according to a preset block size candidate set until convergence or the candidate set is exhausted and a compilation failure is reported. This embodiment does not require dynamic memory allocation or page state machine maintenance at runtime, reducing runtime resource scheduling overhead from the underlying layer.

[0031] Step S103: Based on the page-physical page binding result, perform a static lift-up traversal of the cross-task direct memory access descriptor to obtain the cross-task software pipeline.

[0032] In one possible implementation, this embodiment uses the page-physical page binding result obtained from the solution as the basis for determining the legality of the instruction arrangement. It performs a cross-task DMA descriptor static lift-up traversal on all task nodes in the normalized extended block-level intermediate representation, and constructs a cross-task software pipeline that can be executed in parallel through instruction rearrangement and synchronization barrier insertion operations.

[0033] First, the compiler iterates through all task nodes in the intermediate representation one by one, and rearranges the DMA preloading instructions and matrix calculation instructions that were originally arranged serially within a single task node. Based on the physical page identifier and buffer lifecycle timing information bound to each page allocation node, the DMA preloading instructions used for tensor data transfer by subsequent adjacent tasks are arranged upwards in advance, so that the data preloading operation of downstream tasks can be carried out synchronously with the matrix calculation operation of upstream tasks.

[0034] Secondly, a synchronization barrier instruction is automatically inserted between each DMA data transfer channel and the matrix calculation unit. The synchronization barrier instruction is used to control the hardware execution timing: the matrix calculation unit is only allowed to read the corresponding page data and perform calculations after the DMA channel has completed writing all tensor data and the buffer data is ready. This avoids abnormal problems such as data read / write competition and premature calculation of unread data, and ensures data consistency.

[0035] After the instruction rearrangement and synchronization control instruction insertion processes described above, a cross-task software pipeline structure is formed, achieving overlapping execution times for adjacent tasks: the preceding task performs the matrix calculation stage, while the following task synchronously performs the DMA preloading stage, transforming the originally serial data transfer and matrix calculation into a hardware parallel execution mode. This pipeline structure is entirely statically generated at compile time, without runtime dynamic instruction scheduling logic. It fully reuses the parallel hardware resources of the tensor processor's DMA channel and matrix calculation unit, eliminating computational idle time caused by data waiting and effectively improving the overall hardware computing throughput efficiency.

[0036] Step S104: Unify the page-physical page binding result, cross-task software pipeline, event counter register number and synchronization threshold in the event node into immediate values, construct a static dispatch table, and encode and generate a binary file for the solidified execution kernel.

[0037] In this embodiment, the compiler converts all four types of information into compile-time immediate values ​​for permanent storage. Specifically, the page-physical page binding result represents the on-chip physical page number corresponding to each logical buffer, and is entirely fixed as a constant usable for hardware addressing, eliminating the need for runtime lookup and resolution of memory mapping relationships. The rearranged DMA preloading instructions, matrix calculation instructions, and synchronization barrier instructions in the cross-task software pipeline, along with instruction offsets and scheduling timings, are all fixed as constants, allowing hardware to execute directly according to the fixed sequence without runtime instruction adjustment logic. The event counter register number and synchronization threshold are the hardware parameters required for multi-task synchronization, and all synchronization trigger conditions are fixed as numerical constants, saving the overhead of runtime synchronization parameter reading and conversion.

[0038] Secondly, a global static dispatch table is constructed based on the solidified event synchronization parameters. This dispatch table is stored in the instruction storage area corresponding to the dispatch core, recording the mapping relationship between each task and the event counter and trigger threshold. It is used by the hardware dispatch core to directly trigger the corresponding task according to the event completion status. All mapping relationships are determined during the compilation stage, and there is no need to dynamically establish the scheduling mapping at runtime.

[0039] After constant solidification and static dispatch table construction are completed, the compiler performs the overall coding operation: the static dispatch table is encapsulated into a dispatch instruction segment, and the DMA preloading instructions, matrix calculation instructions, synchronization barrier instructions, task startup mode identifiers, along with the solidified pages and synchronization constants corresponding to each task, are uniformly encapsulated into a working instruction segment. Additionally, different startup modes can be configured for each task node and encoded into the working instruction segment; these startup modes include immediate startup mode and pre-startup mode. Finally, the dispatch instruction segment and the working instruction segment are integrated and packaged to output the binary file containing the solidified execution kernel.

[0040] The compilation output of this embodiment is a binary file containing a fixed execution kernel, which has the following three structural features: Structural Feature A: The binary instruction segment contains two types of instruction segments: the dispatch segment and the working segment. These segments are statically loaded into the physical instruction storage of the dispatch core and the working core, respectively. There is no queue structure, message channel, or runtime shared data structure between the two segments. The only coupling medium is the on-chip event counter register group. Structural Feature B: The binary contains two types of synchronous microinstructions: atomic increment instructions for the event counter register and wait-for-threshold instructions. Both the register number and the threshold are immediate values ​​at compile time. The dispatch segment is a static table of "event counter number → working core number + entry offset" that is fixed at compile time. The dispatch behavior is semantically equivalent to a cross-core instruction pointer register write operation.

[0041] Structural characteristic C: All page addressing operands in the binary are compile-time immediate values, and the hardware environment does not expose any dynamic allocation interfaces.

[0042] Since all resource mappings, pipeline timings, and synchronization control parameters in this binary file are fixed constants at compile time and there are no runtime configurations that can be dynamically modified, the dispatch segment and working segment can be statically loaded into the dispatch core and working core physical instruction storage respectively during subsequent hardware loading. The tensor processor no longer executes scheduling logic such as memory allocation, instruction reordering, and synchronization parameter calculation during operation, which greatly reduces the chip runtime control overhead and fully releases the parallel capabilities of the computing hardware.

[0043] This invention first constructs a unified extended block-level intermediate representation based on three primitives: task, event, and page allocation. This representation can fully carry all-dimensional information about tensor computation, synchronization interaction, and memory allocation, providing a well-organized data foundation for subsequent unified topology optimization and resource allocation. Then, block preprocessing is performed, and based on this, the physical page identifiers matching each page allocation node are structurally solved to obtain page-physical page binding results. This makes the relationship between block size and on-chip capacity structurally solvable at compile time, fundamentally eliminating the need for dynamic acquisition and release of pages at runtime. Next, a static lift-up traversal of cross-task direct memory access descriptors is performed based on the page binding results, pre-constructing a cross-task software pipeline during the compilation phase. Finally, the memory binding relationships, pipeline arrangement rules, and event synchronization parameters are all solidified into compile-time immediate values ​​and a static dispatch table is generated, outputting a solidified execution kernel binary file. During hardware runtime, only this binary file needs to be processed sequentially, without the need for dynamic resource allocation, instruction reordering, or task scheduling calculations. This significantly reduces the runtime control overhead of the tensor processor and improves overall computational throughput performance.

[0044] Figure 2 This is a schematic diagram illustrating the implementation flow of a tensor processor compilation method provided in another embodiment of the present invention. The overall compilation process includes: The front-end compilation primitive layer inputs block-level DSL source code (PersistentTask, Event, PageAlloc) → P1 block size inference → P2 event fusion → P3 dependency normalization → P4 breadth-first linearization → P5 block-page compile-time one-to-one binding (if capacity exceeds the limit, return to P1) → P6 cross-task DMA descriptor static lift-up → task implementation code generation and startup mode marking → P7 compile-time hardening (immediate values ​​and static dispatch table burning) → outputs the hardened execution kernel binary file.

[0045] Figure 3 This is a schematic diagram of a dependency normalization operation provided in an embodiment of the present invention. In this embodiment, if a single task node T0 simultaneously triggers multiple event nodes e1, e2, and e3, a relay event node e′ is introduced, along with empty task nodes T1, T2, and T3 corresponding one-to-one with the multiple event nodes. Task node T0 is bound to the relay event node e′; each empty task node T1, T2, and T3 uses the relay event node e′ as a prerequisite trigger condition and is bound to the corresponding event nodes e1, e2, and e3 respectively. Without changing the task synchronization logic, all task nodes satisfy the single-event output constraint, the intermediate representation topology is standardized, and the subsequent on-chip memory resource allocation and task static scheduling process is simplified.

[0046] Figure 4 This is a schematic diagram of the hardware carrying structure of a fixed execution kernel binary provided in an embodiment of the present invention. For example... Figure 4 As shown, D0 and D1 are two independent scheduling processing units within the dispatch core, each with its own physical instruction storage and loaded static dispatch tables generated during compilation; [A] is the on-chip event counter register group, which is the only coupling medium between the dispatch core and the worker core array, without message queues or dedicated transmission channels; [B] is the hardware address corresponding to the cross-core IP write operation, which is fixed as an immediate value during the compilation phase and stored in the dispatch segment instructions. After the worker cores W0~W7 complete their tasks, they update the corresponding counts in their registers using the atomic++ atomic increment operation; dispatch cores D0 and D1 poll and read the register count values, compare them with the synchronization threshold fixed during compilation, and determine whether the event is ready. Worker Core Array W0, W1...W7: Multiple parallel computing worker cores within the array, each equipped with its own physical instruction storage and loaded with worker segment instructions containing DMA, computation, and startup mode identifiers.

[0047] Figure 5 This is an event-driven execution timing diagram of a fixed execution kernel binary provided in an embodiment of the present invention. For example... Figure 5 As shown, following the timeline as the execution order, D0 initiates dispatch actions multiple times, dispatching TaskA and TaskD to W0, TaskB and TaskE to W1, and TaskC and TaskF to W2 via cross-core IP writes. After each task is completed, the corresponding working core atomically increments the event counter. When the accumulated value of a certain event counter reaches the synchronization threshold fixed at compile time, the threshold is marked as reached, triggering D0 to perform the next round of dispatch actions, thus completing the scheduling loop of multi-task serial synchronization and multi-core parallel computing. The entire scheduling logic relies on the task mapping, synchronization threshold, and cross-core address immediate values ​​fixed at compile time, without any runtime dynamic scheduling calculations.

[0048] Figure 6 This is a schematic diagram of a cross-task software pipeline and immediate addressing paging memory carrying provided by an embodiment of the present invention. The cross-task software pipeline section shows the execution timing of two adjacent tasks T1 and T2 within a single working core: after the preloading phase of T1 is completed, it enters the computation phase. During the execution of the computation phase of T1, the preloading phase of T2 starts simultaneously. The two phases overlap in time, and this overlap is generated by the static DMA descriptor construct produced by P6 at compile time. The immediate addressing paging memory carrying section shows the immediate addressing carrying form of on-chip paging memory: P0~P15 are 16 independent physical paging units on the chip; all page addressing operands are compile-time immediate values, and the page occupancy relationship between T1 and T2 is pre-bound.

[0049] According to the above embodiments, the present invention moves all decision-making logic of memory allocation, task scheduling, resource arbitration, and event synchronization triggering to the compilation stage, and no dynamic decision-making calculations are performed during the hardware operation stage; the compilation output has differentiated fixed structure features, including three unique structures: dispatch segment, separate storage of working segment instructions, fixed immediate values ​​of event synchronization parameters, and physical page immediate value addressing; at the same time, three types of dedicated front-end compilation primitives are customized as the basis for compilation modeling, and the page allocation primitives with statically determined lifecycles and monotonic read-write constraints fully carry the semantics of on-chip storage allocation, providing front-end support for static resource binding throughout the entire process.

[0050] The key technical points of this invention include: (1) Designing PersistentTask, Event, and PageAlloc as chip-specific unified front-end inputs; among which, the life cycle of the PageAlloc forced constraint buffer can be completely determined at compile time, and it is accompanied by a monotonic use flag, which limits the unidirectional data flow of only one write and multiple reads within the life cycle of the buffer. This standardized front-end input system is a fundamental foundation that the existing resident execution kernel scheme does not have. (2) Innovatively introducing PageAllocNode as a new intermediate representation node type, which maps and resolves the three types of front-end primitives one by one with task nodes and event nodes; the node fully carries the closed interval of the page life cycle and the monotonic use constraint flag, providing a standardized data carrier for the subsequent physical page binding solution at compile time. (3) Using the monotonic read and write constraint of PageAlloc as the pre-construction constraint for solving, the page-physical page matching is completed by using interval coloring combined with backtracking algorithm; when the total occupied pages exceed the hardware storage limit, it flows back to the block preprocessing step to adjust the block size, forming a closed-loop iterative solution logic, realizing the multi-task on-chip page resource reuse. (4) Using the page-physical page binding result as the basis for judging the legality of instruction reordering, the DMA preloading instructions and matrix calculation instructions are rearranged in order within the task node, and synchronization barrier instructions are inserted to build a parallel pipeline. (5) The page binding mapping relationship, pipeline layout information, event counter register number, synchronization threshold, and static dispatch table are all converted into compile-time immediate values ​​and fixed entries, and encoded into dispatch segment and working segment instructions respectively to generate a solidified execution kernel binary file.

[0051] The beneficial effects achieved by the embodiments of the present invention include: (1) The end-to-end inference throughput performance is greatly improved. The complete model inference can be completed by loading the solidified execution kernel binary only once, saving the overhead of repeated kernel startup; relying on the cross-task static pipeline, the timing overlap of DMA preloading and matrix calculation is realized, eliminating hardware idle bubbles; the data transfer communication logic is synchronously incorporated into the compilation and solidification, achieving fine-grained parallelism of calculation and communication. In the actual large language model decoding scenario, compared with the traditional operator-by-operator compilation scheme, the overall inference throughput is increased by 1.0~1.7 times, and the single token decoding latency is close to the theoretical minimum latency of the hardware. (2) The hardware runtime control overhead is almost zero. The tensor processor hardware has no task scheduling queue, no page free bitmap, and no page reference count maintenance logic. All allocation, synchronization, and scheduling logic is statically determined in advance. There is no additional control calculation during the running phase, which is particularly significant for the performance gain of dedicated tensor processors in multi-core array architectures. (3) On-chip storage resource utilization is optimized by relying on static page binding during compilation and realizing cross-task reuse of physical paging based on buffer lifecycle timing. Compared with the global static pre-allocation scheme, idle storage space is greatly reduced. Compared with the runtime dynamic paging scheme, memory management hardware and state maintenance overhead are completely eliminated. (4) Three types of DSL front-end primitives are unified as standardized input entry points. The compiler automatically completes the entire chain processing of source code parsing, topology optimization, resource binding, pipeline generation, and binary kernel output. There is no need for manual tuning of block, memory, and scheduling logic. It can automatically adapt to various network model structures. (5) The solidified execution kernel binary file output by compilation has three unique fixed structural features: dispatch / working segment separation, immediate value synchronization, and immediate value addressing. It can directly establish protection boundaries for the hardware-delivered binary product, breaking through the limitation of existing schemes that can only claim rights from the level of runtime scheduling methods.

[0052] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0053] Figure 7 This is a schematic diagram of the structure of a compilation device for a tensor processor provided in an embodiment of the present invention. Figure 7 As shown, the tensor processor's compilation device 7 includes: The acquisition module 71 is used to acquire persistent task primitives, event primitives and page allocation primitives, and construct them into an extended block-level intermediate representation containing task nodes, event nodes and page allocation nodes; Processing module 72 is used to perform block preprocessing operations on the extended block-level intermediate representation and constructively solve the physical page identifiers that match the page allocation nodes to obtain the page-physical page binding results. The lift module 73 is used to perform static lift traversal of cross-task direct memory access descriptors based on the page-physical page binding results, and obtain the cross-task software pipeline. Encoding module 74 is used to unify and solidify the page-physical page binding results, cross-task software pipeline, event counter register numbers and synchronization thresholds in event nodes into immediate values, construct a static dispatch table, and encode and generate a binary file for solidifying the execution kernel.

[0054] In one possible implementation, processing module 72 is used for: Based on the on-chip storage capacity constraints and page monotonic usage constraints of each page allocation node, the physical page identifiers matching each page allocation node are structurally solved using interval coloring plus backtracking or integer linear programming as the solution paradigm. If the solution fails, return to the previous step and perform block preprocessing on the extended block-level intermediate representation to adjust the computation block size corresponding to each task node according to the preset block adjustment parameters until the physical page identifier matching each page allocation node is obtained.

[0055] In one possible implementation, the lift-up module 73 is used for: Traverse each task node and rearrange the direct memory access preload instructions and matrix calculation instructions within the task node; By inserting synchronization barrier instructions between the direct memory access channel and the matrix computation unit, a cross-task software pipeline with overlapping timings between the preloading phase and the matrix computation phase of adjacent tasks is constructed.

[0056] In one possible implementation, the encoding module 74 is used for: Based on immediate values ​​and static dispatch tables, dispatch instruction segments and working instruction segments are encoded to obtain a binary file of the solidified execution kernel; The dispatch instruction segment is used to load the dispatch core physical instruction storage unit of the tensor processor, and the working instruction segment is used to load the working core physical instruction storage unit of the tensor processor. The dispatch core and the working core are coupled through the on-chip event counter register group.

[0057] In one possible implementation, when encoding the dispatch instruction segment and the work instruction segment, the encoding module 74 is also used for: Configure different startup modes for each task node and encode them into the working instruction segment; The startup modes include instant startup mode and pre-startup mode.

[0058] In one possible implementation, processing module 72 is used for: Calculate the size of the computation block corresponding to each task node; Iterate through all event nodes, merge redundant and duplicate synchronous events, and synchronously update the binding relationship between task nodes and event nodes. For a topology structure where a single task node is associated with multiple events, a dependency normalization operation is performed to limit each task node to triggering at most one event node. For the normalized acyclic task dependency graph, a breadth-first traversal is performed to transform the mesh topology into an ordered linear task sequence, outputting standardized topology data; the standardized topology data is used to constructively solve for the physical page identifiers matching the allocation nodes of each page.

[0059] In one possible implementation, processing module 72 is used for: If a single task node triggers multiple event nodes simultaneously, then a relay event node and an empty task node corresponding to each of the multiple event nodes are introduced. Bind the task node to the relay event node; Each empty task node is configured to use the relay event node as a prerequisite for triggering, and each node is bound to the corresponding event node for output.

[0060] This invention first constructs a unified extended block-level intermediate representation based on three primitives: task, event, and page allocation. This representation can fully carry all-dimensional information about tensor computation, synchronization interaction, and memory allocation, providing a well-organized data foundation for subsequent unified topology optimization and resource allocation. Then, block preprocessing is performed, and based on this, the physical page identifiers matching each page allocation node are structurally solved to obtain page-physical page binding results. This makes the relationship between block size and on-chip capacity structurally solvable at compile time, fundamentally eliminating the need for dynamic acquisition and release of pages at runtime. Next, a static lift-up traversal of cross-task direct memory access descriptors is performed based on the page binding results, pre-constructing a cross-task software pipeline during the compilation phase. Finally, the memory binding relationships, pipeline arrangement rules, and event synchronization parameters are all solidified into compile-time immediate values ​​and a static dispatch table is generated, outputting a solidified execution kernel binary file. During hardware runtime, only this binary file needs to be processed sequentially, without the need for dynamic resource allocation, instruction reordering, or task scheduling calculations. This significantly reduces the runtime control overhead of the tensor processor and improves overall computational throughput performance.

[0061] Figure 8 This is a schematic diagram of the compiler provided in an embodiment of the present invention. Figure 8 As shown, the compiler 8 in this embodiment includes a processor 80 and a memory 81. The memory 81 stores a computer program 82. When the processor 80 executes the computer program 82, it implements the steps in the various method embodiments described above. Alternatively, when the processor 80 executes the computer program 82, it implements the functions of each module / unit in the various device embodiments described above.

[0062] For example, computer program 82 can be divided into one or more modules / units, which are stored in memory 81 and executed by processor 80 to complete the present invention. The one or more modules / units can be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of computer program 82 in compiler 8.

[0063] The compiler 8 may include, but is not limited to, a processor 80 and a memory 81. Those skilled in the art will understand that... Figure 8 This is merely an example of Compiler 8 and does not constitute a limitation on Compiler 8. It may include more or fewer components than illustrated, or combine certain components, or different components. For example, Compiler 8 may also include input / output devices, network access devices, buses, etc.

[0064] For the sake of simplicity and clarity, only the above-described functional modules / units are used as examples. In practical applications, the functions described above can be assigned to different functional modules / units as needed. These modules / units can be implemented in hardware, software, or a combination of both.

[0065] In the above embodiments, the descriptions of each embodiment have their own emphasis. Parts not detailed or described in a particular embodiment can be referred to in the relevant descriptions of other embodiments. Unless otherwise specified or in conflict with logic, the terminology and / or descriptions between different embodiments are consistent and can be referenced interchangeably. Technical features in different embodiments can be combined to form new embodiments based on their inherent logical relationships.

[0066] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A compilation method for a tensor processor, characterized in that, include: Obtain persistent task primitives, event primitives, and page allocation primitives, and construct them into an extended block-level intermediate representation containing task nodes, event nodes, and page allocation nodes; Perform block preprocessing on the extended block-level intermediate representation and constructively solve for the physical page identifiers that match each page allocation node to obtain the page-physical page binding results; Based on the page-physical page binding result, a static lift-up traversal of the cross-task direct memory access descriptor is performed to obtain the cross-task software pipeline. The page-physical page binding result, the cross-task software pipeline, the event counter register number and synchronization threshold in the event node are uniformly solidified into immediate values ​​and a static dispatch table is constructed. The binary file of the solidified execution kernel is then encoded and generated. The constructive solution for matching the physical page identifiers of each page allocation node includes: Based on the on-chip storage capacity constraints and page monotonic usage constraints of each page allocation node, the physical page identifiers matching each page allocation node are structurally solved using interval coloring plus backtracking or integer linear programming as the solution paradigm. If the solution fails, the process returns to performing the block preprocessing operation on the extended block-level intermediate representation to adjust the computation block size corresponding to each task node according to the preset block adjustment parameters, until the physical page identifier matching each page allocation node is obtained.

2. The compilation method for a tensor processor as described in claim 1, characterized in that, The process of performing a static lift-up traversal of cross-task direct memory access descriptors based on the page-physical page binding result to obtain a cross-task software pipeline includes: Traverse each task node and rearrange the direct memory access preload instructions and matrix calculation instructions within the task node; By inserting synchronization barrier instructions between the direct memory access channel and the matrix computation unit, a cross-task software pipeline with overlapping timings between the preloading phase and the matrix computation phase of adjacent tasks is constructed.

3. The compilation method for a tensor processor as described in claim 1, characterized in that, The encoding process generates a binary file that permanently executes the kernel, including: Based on the immediate value and the static dispatch table, a dispatch instruction segment and a working instruction segment are encoded to obtain the binary file of the solidified execution kernel; The dispatch instruction segment is used to load the dispatch core physical instruction storage unit of the tensor processor, and the working instruction segment is used to load the working core physical instruction storage unit of the tensor processor. The dispatch core and the working core are coupled through an on-chip event counter register group.

4. The compilation method for a tensor processor as described in claim 3, characterized in that, The process of encoding and generating the dispatch instruction segment and the work instruction segment also includes: Configure different startup modes for each task node and encode them into the working instruction segment; The startup modes include instant startup mode and pre-startup mode.

5. The compilation method for the tensor processor as described in any one of claims 1-3, characterized in that, The step of performing block preprocessing on the extended block-level intermediate representation includes: Calculate the size of the computation block corresponding to each task node; Iterate through all event nodes, merge redundant and duplicate synchronous events, and synchronously update the binding relationship between task nodes and event nodes. For a topology structure where a single task node is associated with multiple events, a dependency normalization operation is performed to limit each task node to triggering at most one event node. For the normalized acyclic task dependency graph, a breadth-first traversal is performed to convert the mesh topology into an ordered linear task sequence, and standardized topology data is output; wherein, the standardized topology data is used to constructively solve the physical page identifiers for matching the allocation nodes of each page.

6. The compilation method for a tensor processor as described in claim 5, characterized in that, The topology structure for a single task node associating with multiple events performs dependency normalization to limit each task node to triggering at most one event node, including: If a single task node triggers multiple event nodes simultaneously, then a relay event node and an empty task node corresponding to each of the multiple event nodes are introduced. Bind the task node to the relay event node; Each empty task node is configured to use the aforementioned relay event node as a prerequisite trigger condition, and each node is bound to the corresponding event node for output.

7. A compilation apparatus for a tensor processor, characterized in that, include: The acquisition module is used to acquire persistent task primitives, event primitives, and page allocation primitives, and construct them into an extended block-level intermediate representation containing task nodes, event nodes, and page allocation nodes; The processing module is used to perform block preprocessing operations on the extended block-level intermediate representation and constructively solve the physical page identifiers that match each page allocation node to obtain the page-physical page binding results. The lift-up module is used to perform static lift-up traversal of cross-task direct memory access descriptors based on the page-physical page binding results, so as to obtain the cross-task software pipeline. The encoding module is used to solidify the page-physical page binding result, the cross-task software pipeline, the event counter register number and synchronization threshold in the event node into immediate values, construct a static dispatch table, and encode and generate a binary file for solidifying the execution kernel. The constructive solution for matching the physical page identifiers of each page allocation node includes: Based on the on-chip storage capacity constraints and page monotonic usage constraints of each page allocation node, the physical page identifiers matching each page allocation node are structurally solved using interval coloring plus backtracking or integer linear programming as the solution paradigm. If the solution fails, the process returns to performing the block preprocessing operation on the extended block-level intermediate representation to adjust the computation block size corresponding to each task node according to the preset block adjustment parameters, until the physical page identifier matching each page allocation node is obtained.

8. A compiler, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method as described in any one of claims 1 to 6.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 6.

Images