A data processing method, device, medium and program product of a heterogeneous accelerator

Abstract

This invention discloses a data processing method, device, medium, and program product for a heterogeneous accelerator. The method includes: acquiring data processing instructions and determining whether the operand types of the source and destination ends are tensor queues based on the data processing instructions; determining a target branch in preset compile-time branches based on whether the operand types of the source and destination ends are tensor queues; determining synchronization primitives for data processing between the source and destination ends based on the processing flow of the target branch; determining the data processing engine type based on the data processing instructions and selecting a target engine based on the engine type; and using the target engine to perform data processing between the source and destination ends based on the synchronization primitives. By pre-setting the processing flow of multiple branches during compile-time, automatic insertion of synchronization primitives can be achieved, reducing code redundancy, logical nesting complexity, and engine switching complexity in data processing.

Description

Technical Field

[0001] This invention relates to the field of chip development technology, and in particular to a data processing method, device, medium and program product for a heterogeneous accelerator. Background Technology

[0002] In the development of operators for heterogeneous accelerators, multi-level storage hierarchies are typically used for asynchronous data processing to achieve high-performance tensor computation.

[0003] To hide data processing latency and improve the overlap between data computation and data movement, a "multi-buffered pipeline" technique, such as double-buffered or triple-buffered pipelines, is typically employed. This means that while data is being moved to buffer A, the computation of ready-to-read data in buffer B is simultaneously performed. In multi-buffered pipelines, developers need to manage three core functional objects simultaneously: the allocation and indexing of multi-buffered storage areas, pipeline synchronization between producers and consumers, and the configuration of various data movement operations within the data processing engine. In existing heterogeneous data processing, these three core functional objects are managed independently. When writing pipelined data processing tasks, developers need to manually declare multiple buffers matching the pipeline depth and calculate their sizes; each time a data processing operation involves a pipeline buffer, they need to manually insert the correct producer and consumer synchronization primitives; and each data processing operation type also requires independent configuration of engine parameters and virtual channels.

[0004] This three-way separation programming model has the following problems: First, code redundancy. For the same data processing operation type, there are four different synchronization scenarios depending on whether the source and destination ends are associated with pipeline buffers. Each additional operation type requires writing four sets of synchronization code. Second, pipeline nesting is difficult to combine correctly. When a data processing operation involves two pipeline buffers simultaneously, the synchronization logic at both ends must be correctly nested; otherwise, deadlocks or data races are very likely to occur. Third, the operation interfaces of various data processing engines are completely different, involving a large amount of business code modification when switching engines. Summary of the Invention

[0005] This invention provides a data processing method, device, medium, and program product for heterogeneous accelerators to reduce code redundancy, logical nesting complexity, and engine switching complexity in data processing.

[0006] According to one aspect of the present invention, a data processing method for a heterogeneous accelerator is provided, the method comprising: Obtain data processing instructions, and determine whether the operand types of the source and destination ends are tensor queues based on the data processing instructions; Based on whether the operand types of the source and destination are tensor queues, the target branch is determined in the preset compile-time branches; and the synchronization primitives for data processing at the source and destination are determined according to the processing flow of the target branch. The data processing engine type is determined according to the data processing instructions, and the target engine is selected according to the engine type. The target engine is used to perform data processing between the source and destination ends based on the synchronization primitives.

[0007] According to another aspect of the present invention, a data processing apparatus for a heterogeneous accelerator is provided, the apparatus comprising: The operand type determination module is used to acquire data processing instructions and determine whether the operand types of the source and destination ends are tensor queues based on the data processing instructions. The synchronization primitive determination module is used to determine the target branch in the preset compile-time branches based on whether the operand types of the source and destination are tensor queues; and to determine the synchronization primitives for data processing at the source and destination based on the processing flow of the target branch. The target engine determination module is used to determine the data processing engine type according to the data processing instructions, and select the target engine according to the engine type; The data processing module is used to perform data processing between the source and destination ends using the target engine based on the synchronization primitives.

[0008] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising: At least one processor; and a memory communicatively connected to said at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, which enables the at least one processor to perform the data processing method for heterogeneous accelerators according to any embodiment of the present invention.

[0009] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to execute and implement the data processing method of the heterogeneous accelerator according to any embodiment of the present invention.

[0010] According to another aspect of the present invention, a computer program product is provided, comprising a computer program that, when executed by a processor, implements the data processing method for heterogeneous accelerators according to any embodiment of the present invention.

[0011] The technical solution of this invention obtains data processing instructions and determines whether the operand types of the source and destination ends are tensor queues based on the data processing instructions; determines the target branch in the preset compile-time branches based on whether the operand types of the source and destination ends are tensor queues; determines the synchronization primitives for data processing between the source and destination ends based on the processing flow of the target branch; determines the data processing engine type based on the data processing instructions, and selects the target engine based on the engine type; and uses the target engine to perform data processing between the source and destination ends based on the synchronization primitives. This solves the problem of complex data processing programming in heterogeneous accelerators. By presetting the processing flow of multiple branches during compile-time, automatic insertion of synchronization primitives can be achieved, reducing code redundancy, logical nesting complexity, and engine switching complexity in data processing.

[0012] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0013] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0014] Figure 1 This is a flowchart of a data processing method for a heterogeneous accelerator according to Embodiment 1 of the present invention; Figure 2 This is a schematic diagram of a data processing structure provided in Embodiment 1 of the present invention; Figure 3 This is a schematic diagram of a process for automatically establishing dependencies in a preset compile-time branch according to Embodiment 1 of the present invention; Figure 4 This is a schematic diagram of a dual-engine adaptation process provided in Embodiment 1 of the present invention; Figure 5 This is a schematic diagram of a tensor queue structure provided in Embodiment 1 of the present invention; Figure 6 This is a schematic diagram of a pipeline synchronization based on dual barriers according to Embodiment 1 of the present invention; Figure 7 This is a schematic diagram of pipeline management of a tensor queue under double buffering according to Embodiment 1 of the present invention; Figure 8 This is a flowchart of a data processing method for a heterogeneous accelerator according to Embodiment 2 of the present invention; Figure 9 This is a flowchart of a coordinate caching optimization method provided in Embodiment 2 of the present invention; Figure 10 This is a flowchart illustrating an application example of multi-level tensor queue cascading according to Embodiment 2 of the present invention; Figure 11 This is a schematic diagram of the structure of a data processing device for a heterogeneous accelerator according to Embodiment 3 of the present invention; Figure 12 This is a schematic diagram of the structure of an electronic device that implements the data processing method of the heterogeneous accelerator according to an embodiment of the present invention. Detailed Implementation

[0015] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0016] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0017] Example 1 Figure 1 This is a flowchart of a data processing method for a heterogeneous accelerator according to Embodiment 1 of the present invention. This embodiment is applicable to situations where data processing is performed through multi-level storage hierarchies in the operator development of heterogeneous accelerators such as Graphics Processing Units (GPUs) or dedicated Artificial Intelligence (AI) chips. This method can be executed by a data processing device for the heterogeneous accelerator, which can be implemented in hardware and / or software and can be configured in electronic devices such as computers, mobile phones, smart terminals, GPUs, or chips. Figure 1 As shown, the method includes: Step 110: Obtain the data processing instructions and determine whether the operand types of the source and destination ends are tensor queues based on the data processing instructions.

[0018] Optionally, the data processing instructions include at least one of the following: asynchronous tile cutting (tile_async), asynchronous untile (untile_async), asynchronous copy (copy_async), asynchronous transpose (transpose_async), asynchronous emission (emit_async), and asynchronous pruning (tile_cropping_async). The data processing instructions can be input into the data processing structure of the heterogeneous accelerator through the user interface layer.

[0019] Figure 2 This is a schematic diagram of a data processing structure provided according to Embodiment 1 of the present invention. Figure 2 As shown, the target branch and target engine determination, as well as the data processing between the source and destination, can be applied to each data processing operation. In other words, the only change for a new data processing operation type is the operation itself, while the pipeline synchronization capability of data processing can be determined through the preset compile-time branch dependency management layer and the dual-engine adaptation layer in the data processing structure, thereby enabling data processing.

[0020] The source and destination can be specified through data processing instructions, representing the form of the original and destination data for a single data processing operation. Operand types can also be determined through data processing instructions. For example, the data processing instruction is `tile_async(tq, src, coord)`, where "tq" indicates that the destination operand type is a tensor queue, "src" indicates that the source operand type is a tensor, "coord" indicates the data processing coordinates, and "tile_async" indicates that the data processing operation type is asynchronous chunking.

[0021] A tensor is a basic data unit in network model inference, i.e., a multidimensional array. A tensor queue (tq) is a queue formed by multiple tensors. In this embodiment of the invention, the tensor queue can be used as a programming object, and technical encapsulation can be performed within it. For example, multi-level circular buffers, pipeline synchronizers, data processing execution engines, storage space ownership management, and coordinate cache optimization can be encapsulated within the tensor queue.

[0022] Step 120: Determine the target branch in the preset compile-time branches based on whether the operand types of the source and destination are tensor queues; and determine the synchronization primitives for data processing between the source and destination based on the processing flow of the target branch.

[0023] In this embodiment of the invention, when the operand types of the source and destination ends are different, different preset compile-time branches can be entered. Each preset compile-time branch can pre-encapsulate different processing flows based on the tensors or tensor queues to which the operand types of the source and destination ends belong. During data processing, the synchronization primitives for data processing between the source and destination ends can be determined based on the dependencies in the processing flows of the preset compile-time branches. By identifying the operand types of the source and destination ends, the target branch can be directly generated in the preset compile-time branches, without requiring the developer to manually determine and insert the synchronization primitives. This ensures the correctness of the dependencies and avoids redundant code coding. For developers, when developing operators, they only need to determine the data processing instructions, without having to manually determine the dependencies between the source and destination ends in the data processing. Developers can form call operations with the same user interface, and the electronic device can automatically complete the correct synchronization logic, fundamentally eliminating the omissions, out-of-order issues, or deadlocks that may occur when manually inserting synchronization primitives. Furthermore, the processing of the preset compile-time branches is completed entirely at compile time, without introducing runtime overhead.

[0024] Synchronization primitives refer to the synchronization control statements between producers and consumers during data processing at the source and destination ends. For example, synchronization primitives may include `push_queue_get_async`, `push_queue_put_async`, `pop_queue_get`, and `pop_queue_put`. In these synchronization primitives, "push_queue" represents enqueueing, "pop_queue" represents dequeueing, "get" represents requesting, "put" represents committing, and "async" represents asynchronous operation. The specific synchronization primitives used differ depending on the operand types at the source and destination ends.

[0025] Optionally, based on whether the operand types of the source and destination ends are tensor queues, a target branch is determined in the preset compile-time branches; and the synchronization primitives for data processing at the source and destination ends are determined according to the processing flow of the target branch, including: when the operand types of both the source and destination ends are tensors, the target branch is determined as the first branch; when the operand types of both the source and destination ends are tensor queues, the target branch is determined as the second branch; when the operand types of both the source and destination ends are tensors, the target branch is determined as the third branch; when the operand types of both the source and destination ends are tensor queues, the target branch is determined as the fourth branch; wherein, the processing flow of the first branch does not have synchronization primitives; the synchronization primitives corresponding to the processing flow of the second branch are control statements for consumer dependencies; the synchronization primitives corresponding to the processing flow of the third branch are control statements for producer dependencies; and the synchronization primitives corresponding to the processing flow of the fourth branch are control statements for consumer and producer dependencies.

[0026] Figure 3 This is a schematic diagram illustrating a process for automatically establishing dependencies using preset compile-time branches, as provided in Embodiment 1 of the present invention. Figure 3 As shown, for data processing instructions at the user interface layer, it can be directly determined whether the operand types at the source and destination ends are tensor queues. When both the source and destination operand types are tensors, the process enters the first branch. The processing flow of the first branch does not have synchronization primitives and can directly issue data processing operations through the data processing engine without establishing pipeline dependencies. For example, the data processing engine can issue the data processing operation engine_.tile_(dst,src), indicating that the data processing engine performs a data slicing operation where both the source and destination (dst) are tensors.

[0027] like Figure 3 As shown, when both the source and destination operand types are tensor queues, the process enters the second branch. The synchronization primitives in the second branch can be control statements for consumer dependencies. For example, before data processing, `pop_queue_get_async()` is called to wait for data to be ready in the current stage buffer of the source tensor queue, obtaining a readable tensor view; after data processing, `pop_queue_put_async()` is called to release the stage, notifying the producer that it is ready to be overwritten. The processing flow of the second branch can be: calling `pop_queue_get_async()`, the data processing engine executing `engine_.tile_(dst, src_tq)` (i.e., performing data slicing operations where the source is a tensor queue and the destination is a tensor), and then calling `pop_queue_put_async()`. In the second branch, the source automatically manages consumer dependencies without requiring the user to manually insert synchronization primitives.

[0028] like Figure 3As shown, when the source operand type is tensor and the destination operand type is tensor queue, the process enters the third branch. The synchronization primitive corresponding to the processing flow of the third branch is the producer dependency control statement. For example, before the data processing operation, push_queue_get_async() is called to obtain an idle stage in the destination tensor queue; after the data processing operation is completed, push_queue_put_async() is called to submit the producer's data ready signal to notify the consumer that the data is readable. The processing flow of the third branch can be: calling push_queue_get_async(), the data processing engine executing engine_.tile_(dst_tq,src), which is to perform data slicing operation with tensors at the source and tensor queues at the destination, and calling push_queue_put_async(). In the third branch, the destination automatically manages the producer dependency without requiring the user to manually insert synchronization primitives.

[0029] like Figure 3 As shown, when both the source and destination operand types are tensor queues, the process enters the fourth branch. The synchronization primitives corresponding to the processing flow of the fourth branch are control statements for consumer and producer dependencies, that is, performing consumer dependency management on the source and producer dependency management on the destination. For example, before the data processing operation, `pop_queue_get_async()` is called on the source to wait for the data in the current buffer (stage) of the source tensor queue to be ready, obtaining a readable tensor view; `push_queue_get_async()` is called on the destination to obtain a free stage in the destination tensor queue; after the data processing operation is completed, `push_queue_put_async()` is called on the destination to submit the producer's data ready signal to notify the consumer that it is readable; `pop_queue_put_async()` is called on the source to release the stage, notifying the producer that it can be overwritten. The fourth branch's processing flow can involve calling `pop_queue_get_async()`, calling `push_queue_get_async()`, the data processing engine executing `engine_.tile_(dst_tq, src_tq)` (which performs data chunking operations on both the source and destination sides being tensor queues), calling `push_queue_put_async()`, and calling `pop_queue_put_async()`. In this fourth branch, by managing both ends simultaneously, the correct nesting of producer and consumer synchronization logic is ensured, avoiding deadlock.

[0030] By automatically expanding data manipulation instructions according to preset compile-time branches, user operations are reduced, and the problems of code redundancy and difficulty in correctly combining nested pipelines are solved.

[0031] Step 130: Determine the data processing engine type according to the data processing instructions, and select the target engine according to the engine type.

[0032] Data processing is executed through a data processing engine, such as an asynchronous data transfer engine (DTE). A DTE is a hardware transfer engine that can independently perform data transfer between different storage tiers, supporting various data processing operation types. Figure 2 As shown, after the preset compile-time branch dependency management layer determines the synchronization primitives, it can enter the dual-engine adaptation layer. The dual-engine adaptation layer can set various types of data processing engines. For example, a general engine and a dedicated engine can be set in the data processing engine type. The data processing engine type is determined according to the data processing instructions. For example, the engine type can be bound to the operands at the source and destination ends of the data processing instructions. That is, tensors or tensor queues can be bound to engine types. By selecting the target engine according to the engine type, the tensors or tensor queues are actually moved. In data processing, through methods such as... Figure 2 The process shown allows users to complete all data processing operations without needing to understand the underlying layering details, simply by inputting data processing commands through a unified user interface.

[0033] Optionally, the data processing engine type is determined based on the data processing instruction, and a target engine is selected based on the engine type, including: when the parameter of the data processing engine type in the data processing instruction is a general engine, the general engine is determined as the target engine; wherein, the general engine encapsulates the full configuration of data processing parameters, data processing pipeline barrier dependencies, data processing instruction issuance and execution, and scheduling state advancement; when the parameter of the data processing engine type in the data processing instruction is a dedicated engine, the dedicated engine is determined as the target engine; wherein, the dedicated engine encapsulates incremental data processing updates, scheduling authority management, and atomic instruction issuance and execution.

[0034] Figure 4 This is a schematic diagram of a dual-engine adaptation process provided according to Embodiment 1 of the present invention. Figure 4As shown, the configuration and instruction issuance methods of the general-purpose engine and the dedicated engine are completely different. In this embodiment of the invention, the differences between the two engines are transparently encapsulated at the underlying level. Users determine the engine type at compile time by calling the same interface, and are automatically assigned to different execution flows according to the engine type. The general-purpose engine (engine_t) provides full configuration capability for DTE, resetting all parameters for each operation. The general-purpose engine supports scheduling management of multiple virtual channels (VCs), establishing dependencies with pipeline barriers through the cur_schedule_().succ_() and cur_schedule_().pred_() methods. Among them, the cur_schedule_().succ_() method is used to notify the end of data processing and notify the next piece of data processing. The cur_schedule_().pred_() method is used to request data processing, i.e., to configure and trigger data processing. After issuing the DTE operation instruction, the scheduling state is advanced through next_schedule_(). When the engine manages multiple VCs, the tensor queue rotates among multiple VCs through an internal locking and unlocking mechanism to ensure that the DTE configuration and triggering of the same task are completed on the same VC. The general-purpose engine supports all operation types and is suitable for scenarios with diverse operation types and where flexibility is paramount.

[0035] Dedicated engines, such as the dedicated copy atom engine (copy_atom), provide an incremental configuration interface. During the initial construction, all DTE parameters are fully initialized. Subsequent operations only update the changed source or destination operands using replace_src_tensor() or replace_dst_tensor(), and then execute the DTE transfer via the copy() atomic instruction, avoiding the full configuration overhead of general-purpose engines for each operation. During data processing, scheduling is managed using lock_schedule() and unlock_schedule(). lock_schedule() is used to acquire scheduling rights, and unlock_schedule() is used to release scheduling rights. Dedicated engines are suitable for scenarios with fixed operation types and a pursuit of ultimate performance.

[0036] Both engines are automatically assigned through compile-time engine type constraints under the same set of operation interfaces. When users replace the general engine with the dedicated engine, they only need to modify the tensor queue's constructor parameters (such as changing the engine type from engine_t to copy_atom), and all business layer operation call code remains unchanged, solving the problem of difficult engine switching.

[0037] Step 140: Use the target engine to process data between the source and destination based on the synchronization primitives.

[0038] The target engine can process data between the source and destination based on synchronization primitives and execution statements, and manage the producer and consumer dependencies in data processing.

[0039] To further manage the pipelined processing order of tensor queues in data processing and avoid data races, in an optional embodiment of this invention, the tensor queue encapsulates multiple levels of circular buffers. Each level of buffer is equipped with a double barrier, including an empty barrier and a full barrier. An empty barrier indicates that the corresponding level buffer has been released by the consumer, granting write access to the producer. A full barrier indicates that the corresponding level buffer has been filled by the producer, granting read access to the consumer. When the target engine processes data between the source and destination based on synchronization primitives, the producer waits for the empty barrier corresponding to the current level buffer accessed in the current round to be ready before writing data and triggers the full barrier after data processing. The consumer waits for the full barrier corresponding to the current level buffer accessed in the current round to be ready before reading data and triggers the empty barrier after data processing. This alternating triggering of the double barriers forms a closed-loop data processing pipeline.

[0040] Figure 5 This is a schematic diagram of a tensor queue structure according to Embodiment 1 of the present invention. During construction, the tensor queue manages a contiguous storage space (size is "single tensor size × stages"), i.e., a multi-level circular buffer, based on layout and pipeline depth. For example... Figure 5 As shown, the tensor queue encapsulates multiple levels of circular buffers, such as stage 0, stage 1, ..., stage N-1. Circular access is performed using the modulo operation (i mod stages), where i represents the current tensor fragment number and stages represents the pipeline depth of the circular memory. Each level of buffer can be accessed using... Figure 5 The pipeline synchronizer shown is used for control. Each level of buffer is equipped with a double barrier: an empty barrier and a full barrier. An empty barrier indicates that the corresponding level buffer has been released by the consumer, granting the producer write access; a full barrier indicates that the corresponding level buffer has been filled by the producer, granting the consumer read access. During data processing, the producer first waits for the empty barrier to confirm writability, then triggers a full barrier to notify the consumer after performing DTE data transfer. The consumer first waits for the full barrier to confirm readable access, then triggers an empty barrier to notify the producer after completing the computation. The two types of barriers trigger alternately to form a closed loop, ensuring that no data races occur at any stage of the pipeline.

[0041] Specifically, Figure 6 This is a schematic diagram of a pipeline synchronization based on dual barriers according to Embodiment 1 of the present invention. Figure 6 As shown, each stage is bound to a storage space, an empty barrier, and a full barrier, forming a fixed triplet of "buffer[i] + empty_barrier[i] + full_barrier[i]". This ensures that producer writes, consumer reads, and barrier signals always operate on the same stage, preventing state corruption. The internal function `tensor(i)` maps the i-th stage to the corresponding tensor view using pointer offset `ptr + i × size(layout)`, and uses the modulo operation `i mod stages` to ensure circular reuse.

[0042] like Figure 6 As shown, the pipeline state is managed through the producer state `prod_state_` and the consumer state `cons_state_`. Both the producer and consumer state pointers can include two fields: the buffer index of the current operation (`stage_`) and the current entry round (`phase_`). When the producer calls `push_queue_get_async()`, it waits for the corresponding level buffer's empty barrier `empty_barrier[stage_]` using `prod_state_.stage_` as the index. Upon successful completion, it returns the corresponding tensor view `tensor(stage_)`. When the producer commits, it calls `push_queue_put_async()`, triggering the corresponding level buffer's full barrier to reach `full_barrier[stage_].arrive()`. It then executes the scheduling state advance(), advancing `stage_` to the next level buffer. Simultaneously, the round number `phase_` is automatically updated when crossing stage boundaries. The consumer's `pop_queue_get / put` functions symmetrically with the producer, but use the consumer state `cons_state_` and the empty / full barriers in the opposite direction. Its pipeline processing will not be elaborated upon here.

[0043] like Figure 5As shown, in the pipeline synchronizer, each level of buffer is equipped with a double barrier indicating that the producer can write to the buffer of that level and the consumer can read it. The current level buffer accessed in the current round is indicated by the producer and consumer states, thus achieving automatic generation of a closed-loop data processing pipeline in the tensor queue. Updating the round number `phase_` solves the ABA problem in multi-round multiplexing of multi-level circular buffers: when a buffer is used multiple times, the stage index alone cannot distinguish which round of waiting it is, while `phase_` records the current round entered. During barrier waiting, `wait(phase_)` ensures that only the access signal of the current round can pass through.

[0044] like Figure 5 and Figure 6 The tensor queue sets empty / full barriers for each level of buffer, as well as producer and consumer states for each buffer. This allows the buffer index, barrier index, tensor view, and state progression to be automatically associated through the same stage_ variable, eliminating the need for users to manually maintain any index correspondences.

[0045] For example, Figure 7 This is a schematic diagram of pipelined management of a tensor queue under double buffering according to Embodiment 1 of the present invention. Figure 7 As shown, the user only needs to write one line of code on the producer side (e.g., `tile_async(tq,src,coord)`), corresponding to the third branch. According to the method provided in this embodiment, the execution flow can be automatically expanded into a complete process: `push_queue_get_async()` automatically waits for an empty barrier and obtains the tensor view of the current stage → automatically adapts to the engine to perform DTE operations → `push_queue_put_async()` automatically triggers a full barrier and advances the producer state. The consumer side only needs two lines of code: `pop_queue_get()` and `pop_queue_put()`, automatically completing the full barrier wait, tensor view return, empty barrier release, and state advancement. Through automatic flow expansion, the user does not need to manually perform the six steps of buffer indexing, buffer state determination, engine configuration, instruction issuance, buffer state update, and state advancement on the producer side; nor does the user need to manually perform the five steps of buffer indexing, buffer state determination, instruction execution, buffer state update, and state advancement on the consumer side. Thus, the at least 11 steps that the user needs to manually maintain in the prior art are reduced to 3 lines of code. Figure 7 The operations within the dashed box shown are automatically maintained operations based on the method provided in this embodiment of the invention, omitting user processing. Because this embodiment of the invention embeds a pipelined index relationship, it ensures that data processing is complete without omissions or out-of-order issues, avoiding manual maintenance, especially deadlocks or data races caused by errors in manually maintaining index relationships during large-scale tensor processing.

[0046] The technical solution of this embodiment obtains data processing instructions and determines whether the operand types of the source and destination ends are tensor queues based on the data processing instructions; determines the target branch in the preset compile-time branches based on whether the operand types of the source and destination ends are tensor queues; determines the synchronization primitives for data processing between the source and destination ends based on the processing flow of the target branch; determines the data processing engine type based on the data processing instructions, and selects the target engine based on the engine type; and uses the target engine to perform data processing between the source and destination ends based on the synchronization primitives. This solves the problem of complex data processing programming in heterogeneous accelerators. By presetting the processing flow of multiple branches during compile-time, automatic insertion of synchronization primitives can be achieved, reducing code redundancy, logical nesting complexity, and engine switching complexity in data processing.

[0047] Example 2 Figure 8 This is a flowchart of a data processing method for a heterogeneous accelerator according to Embodiment 2 of the present invention. This embodiment is a further refinement of the above technical solution, and the technical solution in this embodiment can be combined with various optional solutions in one or more of the above embodiments. Figure 8 As shown, the method includes: Step 810: Obtain the data processing instruction and determine whether the operand types of the source and destination ends are tensor queues based on the data processing instruction.

[0048] Optionally, the data processing instructions include at least one of the following: asynchronous slicing, asynchronous restore, asynchronous copy, asynchronous transpose, asynchronous instruction issuance, and asynchronous trimming.

[0049] Step 820: Determine the target branch in the preset compile-time branches based on whether the operand types of the source and destination are tensor queues; and determine the synchronization primitives for data processing between the source and destination based on the processing flow of the target branch.

[0050] Optionally, based on whether the operand types of the source and destination ends are tensor queues, a target branch is determined in the preset compile-time branches; and the synchronization primitives for data processing at the source and destination ends are determined according to the processing flow of the target branch, including: when the operand types of both the source and destination ends are tensors, the target branch is determined as the first branch; when the operand types of both the source and destination ends are tensor queues, the target branch is determined as the second branch; when the operand types of both the source and destination ends are tensors, the target branch is determined as the third branch; when the operand types of both the source and destination ends are tensor queues, the target branch is determined as the fourth branch; wherein, the processing flow of the first branch does not have synchronization primitives; the synchronization primitives corresponding to the processing flow of the second branch are control statements for consumer dependencies; the synchronization primitives corresponding to the processing flow of the third branch are control statements for producer dependencies; and the synchronization primitives corresponding to the processing flow of the fourth branch are control statements for consumer and producer dependencies.

[0051] Step 830: When the data processing engine type parameter in the data processing instruction is a general engine, determine the general engine as the target engine; when the data processing engine type parameter in the data processing instruction is a special engine, determine the special engine as the target engine.

[0052] The general engine encapsulates full configuration of data processing parameters, data processing pipeline barrier dependencies, data processing instruction issuance and execution, and scheduling status advancement; the dedicated engine encapsulates incremental data processing updates, scheduling authority management, and atomic instruction issuance and execution.

[0053] Step 840: When the target engine performs data processing between the source and destination based on the synchronization primitives, the producer waits for the empty barrier corresponding to the current level buffer accessed in the current round to be ready before writing data, and triggers a full barrier after data processing.

[0054] Step 850: When the consumer waits for the current level buffer corresponding to the current round of access to be full and the barrier is ready, it reads the data and triggers an empty barrier after data processing; a closed-loop data processing pipeline is formed by alternating triggering of dual barriers.

[0055] The tensor queue encapsulates multiple levels of circular buffers. Each level of buffer is equipped with a double barrier, which includes an empty barrier and a full barrier. An empty barrier indicates that the corresponding level of buffer has been released by the consumer and that the producer has write access. A full barrier indicates that the corresponding level of buffer has been filled by the producer and that the consumer has read access.

[0056] Based on the above implementation, optionally, a source data index cache and a buffer coordinate mapping cache are set in the tensor queue; when the target engine performs data processing between the source and destination based on the synchronization primitive, if the source data index cache and the buffer coordinate mapping cache of the current data processing operation are the same as those of the previous data processing operation, the target engine performs pipeline advancement between the source and destination based on the synchronization primitive, skipping the actual data processing execution action.

[0057] Figure 9 This is a flowchart of a coordinate caching optimization method provided according to Embodiment 2 of the present invention. Figure 9 As shown, the tensor queue maintains a coordinate index cache (coord_maps_) and a source data pointer cache (raw_ptr_maps_) at each stage. coord_maps_ indicates the data offset, and raw_ptr_maps_ indicates the data start position; together, they determine the specific data. Before performing a data processing operation, the coordinate index and source data pointer of the current operation are compared with the cached values ​​of the previous operation at that stage. If they match, it means the data for this operation is exactly the same as the previous one; only the engine state is refreshed and the pipeline is advanced, skipping the actual DTE transport. This can be ensured by using flush() to guarantee the previous operation's completion. In scenarios with overlapping sliding windows in convolution operations, this coordinate cache optimization can reduce the actual DTE transport count by 10%-40%, solving the redundant transport problem in existing solutions that cannot automatically identify data reuse.

[0058] Optionally, storage ownership can be set in the tensor queue; storage ownership includes self-ownership mode and reference mode.

[0059] The ownership of storage space in a tensor queue is determined by its construction method, which can include self-ownership and reference mode. Self-ownership mode means that only the layout is passed in during construction, without requiring external storage or an engine. Internally, the tensor queue automatically allocates storage space based on the layout and pipeline depth (size = cosize(layout) × stages), and selects the memory region based on the scope type: SharedScope corresponds to shared storage, and LocalScope corresponds to local storage. Simultaneously, it creates independent engine instances internally (engine type is held using value semantics). Self-ownership mode eliminates common errors associated with manually calculating buffer sizes and selecting memory regions.

[0060] Non-Owning reference mode refers to passing an externally allocated memory pointer and engine reference during construction. Tensor queues do not own the lifecycle of the memory and engine; instead, they manage external resources by reference (the engine type is held using reference semantics). Reference mode is suitable for scenarios where multiple tensor queues share the same engine or memory pool, such as when different tensor queues in a multi-level cascade reuse the same DTE engine, avoiding redundant allocation of engine resources. All operation interfaces are completely consistent between the two modes, and users can switch transparently.

[0061] Figure 10 This is a flowchart illustrating an application example of multi-level tensor queue cascading according to Embodiment 2 of the present invention. Figure 10 As shown, taking the three-level data flow of "off-chip storage → shared storage → local storage → computing core" as an example, the application process of tensor queue is demonstrated. First, taking double buffer stages=2 as an example, the pipeline timing is as follows: at time i=0, data tile[0] is read from off-chip storage to shared storage (first round); at i=1, data tile[1] is read from off-chip storage to shared storage (first round), and data is copied from shared storage to local storage copy[0]; at i=2, data tile[0] is read from off-chip storage to shared storage (second round), and data is copied from shared storage to local storage copy[1], and the computing core performs computation[0] based on the data in local storage; at i=3, data tile[1] is read from off-chip storage to shared storage (second round), and data is copied from shared storage to local storage copy[0], and the computing core performs computation[1] based on the data in local storage; at i=4, data tile[0] is read from off-chip storage. The data is copied from the shared storage (third round) to the local storage copy[1], and the computing core performs the calculation compute[0] based on the data in the local storage; and so on, by performing the calculation of the previous data in the current data transfer process, the data transfer time is hidden.

[0062] In the timing sequence of storing data from the current-level buffer in the current round to shared memory and then to local memory for computation, the following pipeline control process can be implemented: Step 1: Construct two tensor queues, Tensor Queue A (tq_a) and Tensor Queue B (tq_b). Tensor Queue A allocates buffers in shared memory using a self-sustaining mode and is bound to a general-purpose engine. Tensor Queue B allocates buffers in local memory using a self-sustaining mode and is bound to a dedicated engine. Step 2: The data chunks from the off-chip storage are moved to Tensor Queue A using the data processing instruction `tile_async(tq_a, src_tensor, coord)`. At this point, the source is a tensor, and the destination is a tensor queue. Corresponding to the third branch, the following calls are made: `tq_a.push_queue_get_async()` (general-purpose engine automatically obtains an idle stage), `engine_.tile_(dst_tq, src)` (issues DTE tile instructions with full configuration), and `tq_a.push_queue_put_async()` (submits data ready). Next, the execution data is moved from shared storage to local storage. The third step involves cross-queue dependency transfer. Consumers in tensor queue A wait to bind to the engine of tensor queue B via `tq_a.pop_queue_get_async(tq_b)`, creating a pipeline between shared storage → local storage and off-chip storage → shared storage. The fourth step involves copying data from shared storage to local storage using the data processing instruction `copy_async(tq_b, tq_a)`. At this point, both the source and destination are the fourth branch corresponding to the tensor queue. This involves sequentially calling `tq_a.pop_queue_get_async()`, `tq_b.push_queue_get_async()`, the data processing engine executing `engine_.copy_(tq_b, tq_a)` to issue copy commands in incremental configuration, and calling `tq_b.push_queue_put_async()` and `tq_a.pop_queue_put_async()`. Step 5: The computation core synchronously waits for the locally stored data to be ready via `tq_b.pop_queue_get()` before executing `compute(tensor)`. After completion, it calls `tq_b.pop_queue_put()` to release the data. By combining the above process with the double-barrier design of the tensor queue's multi-level circular buffer, producer and consumer states, pipelined processing at each time step can be achieved, ensuring the correctness of automatic nesting. During this process, if the coordinates of an adjacent iteration are the same as the previous one, data transfer can be skipped due to a coordinate cache hit.

[0063] The technical solution of this invention involves acquiring data processing instructions and determining whether the operand types of the source and destination ends are tensor queues based on these instructions. A target branch is determined within a preset compile-time branch based on the operand types of the source and destination ends. Synchronization primitives for data processing between the source and destination ends are determined based on the processing flow of the target branch. When the data processing engine type parameter in the data processing instructions is a general engine, the general engine is determined as the target engine. When the data processing engine type parameter in the data processing instructions is a dedicated engine, the dedicated engine is determined as the target engine. When the target engine performs data processing between the source and destination ends according to the synchronization primitives, the producer waits for the empty barrier corresponding to the current level buffer accessed in the current round to be ready before writing data and triggers a full barrier after data processing. The consumer waits for the full barrier corresponding to the current level buffer accessed in the current round to be ready before reading data and triggers an empty barrier after data processing. By alternately triggering dual barriers to form a closed-loop data processing pipeline, the problem of heterogeneous data processing is solved. To address the complexity of data processing programming in accelerators, the integration of storage, synchronization, and engines reduces code redundancy, lowering overall data processing complexity by 70%-80%, with a complete pipeline requiring only 3-5 lines of code. Pre-defined compile-time four-branch dependency handling resolves issues of omissions, out-of-order delivery, and deadlocks associated with manually inserted synchronization primitives, all at compile time without runtime overhead. A unified multi-operation interface eliminates the need for rewriting synchronization code for each operation, with new operation types automatically inheriting full pipeline synchronization capabilities. Transparent dual-engine adaptation eliminates the need to rewrite business code when switching engines, requiring only modifications to constructor parameters; dedicated engines reduce DTE configuration time by approximately 30%-50%. Storage ownership management resolves errors in manual buffer size calculations, automatically allocating and eliminating common defects. Coordinate caching optimization addresses redundant data transfer in reuse scenarios, reducing DTE transfers by 10%-40%, lowering latency and power consumption.

[0064] Example 3 Figure 11 This is a schematic diagram of the structure of a data processing device for a heterogeneous accelerator according to Embodiment 3 of the present invention. Figure 11 As shown, the device includes: an operand type determination module 1101, a synchronization primitive determination module 1102, a target engine determination module 1103, and a data processing module 1104. Wherein: The operand type determination module 1101 is used to obtain data processing instructions and determine whether the operand types of the source and destination ends are tensor queues based on the data processing instructions. The synchronization primitive determination module 1102 is used to determine the target branch in the preset compile-time branches based on whether the operand types of the source and destination are tensor queues; and to determine the synchronization primitives for data processing at the source and destination based on the processing flow of the target branch. The target engine determination module 1103 is used to determine the data processing engine type according to the data processing instructions, and select the target engine according to the engine type; The data processing module 1104 is used to perform data processing between the source and destination ends using the target engine based on synchronization primitives.

[0065] Optionally, the synchronization primitive determination module 1102 is specifically used for: When both the source and destination operands are of tensor type, the target branch is determined as the first branch; When the operand type at the source end is a tensor queue and the operand type at the destination end is a tensor, the target branch is determined to be the second branch. When the operand type at the source end is tensor and the operand type at the destination end is tensor queue, the target branch is determined to be the third branch. When both the source and destination operand types are tensor queues, the target branch is determined to be the fourth branch; The processing flow of the first branch does not have a synchronization primitive; the synchronization primitive corresponding to the processing flow of the second branch is the control statement of the consumer dependency relationship; the synchronization primitive corresponding to the processing flow of the third branch is the control statement of the producer dependency relationship; and the synchronization primitive corresponding to the processing flow of the fourth branch is the control statement of the consumer and producer dependency relationships.

[0066] Optionally, the target engine determination module 1103 is specifically used for: When the data processing engine type parameter in the data processing instruction is a general engine, the general engine is determined as the target engine; The general engine encapsulates the full configuration of data processing parameters, the data processing pipeline barrier dependencies, the issuance and execution of data processing instructions, and the advancement of scheduling status. When the data processing engine type parameter in the data processing instruction is a dedicated engine, the dedicated engine is determined as the target engine; The dedicated engine encapsulates data processing, incremental updates, scheduling authority management, and atomic instruction issuance and execution.

[0067] Optionally, the tensor queue encapsulates multiple levels of circular buffers; each level of buffer is equipped with a double barrier, which includes an empty barrier and a full barrier; the empty barrier is used to indicate that the corresponding level of buffer has been released by the consumer and the producer has the right to write; the full barrier is used to indicate that the corresponding level of buffer has been filled by the producer and the consumer has the right to read. Data processing module 1104 includes: The dual-barrier triggering unit is used when the target engine processes data between the source and destination ends according to the synchronization primitives. The producer waits for the empty barrier corresponding to the current level buffer accessed in the current round to be ready before writing data, and triggers a full barrier after data processing; the consumer waits for the full barrier corresponding to the current level buffer accessed in the current round to be ready before reading data, and triggers an empty barrier after data processing; a closed-loop data processing pipeline is formed by the alternating triggering of dual barriers.

[0068] Optionally, the source data index cache and the coordinate mapping cache of the buffer can be set in the tensor queue; Data processing module 1104 includes: The coordinate cache optimization unit is used to skip the actual data processing execution action when the target engine performs data processing between the source and destination based on the synchronization primitives. If the coordinate mapping cache of the source data index cache and the buffer of the current data processing operation are the same as those of the previous data processing operation, the target engine will use the synchronization primitives to advance the pipeline between the source and destination.

[0069] Optionally, storage ownership can be set in the tensor queue; storage ownership includes self-ownership mode and reference mode.

[0070] Optionally, the data processing instructions include at least one of the following: asynchronous slicing, asynchronous restore, asynchronous copy, asynchronous transpose, asynchronous instruction issuance, and asynchronous trimming.

[0071] The data processing device for heterogeneous accelerators provided in the embodiments of the present invention can execute the data processing method for heterogeneous accelerators provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of executing the method.

[0072] Example 4 Figure 12 This is a schematic diagram of the structure of an electronic device implementing the data processing method of a heterogeneous accelerator according to embodiments of the present invention. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (such as helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0073] like Figure 12As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) or random access memory (RAM), communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded into the RAM 13 from the storage unit 18. The RAM 13 can also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. Input / output (I / O) interfaces are also connected to the bus 14.

[0074] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0075] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as data processing methods of heterogeneous accelerators.

[0076] In some embodiments, the data processing method for the heterogeneous accelerator can be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the data processing method for the heterogeneous accelerator described above can be performed. Alternatively, in other embodiments, processor 11 can be configured to perform the data processing method for the heterogeneous accelerator by any other suitable means (e.g., by means of firmware).

[0077] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0078] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0079] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, RAM, ROM, erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0080] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0081] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0082] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0083] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0084] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A data processing method for a heterogeneous accelerator, characterized in that, include: Obtain data processing instructions, and determine whether the operand types of the source and destination ends are tensor queues based on the data processing instructions; The target branch is determined from the preset compile-time branches based on whether the operand types of the source and destination ends are tensor queues. And determine the synchronization primitives for data processing at the source and destination ends based on the processing flow of the target branch; The data processing engine type is determined according to the data processing instructions, and the target engine is selected according to the engine type. The target engine is used to perform data processing between the source and destination ends based on the synchronization primitives.

2. The method according to claim 1, characterized in that, The target branch is determined from the preset compile-time branches based on whether the operand types of the source and destination ends are tensor queues. And based on the processing flow of the target branch, determine the synchronization primitives for data processing at the source and destination ends, including: When both the source and destination operand types are tensors, the target branch is determined to be the first branch; When the operand type at the source end is a tensor queue and the operand type at the destination end is a tensor, the target branch is determined to be the second branch; When the operand type at the source end is tensor and the operand type at the destination end is tensor queue, the target branch is determined to be the third branch; When both the source and destination operand types are tensor queues, the target branch is determined to be the fourth branch; Specifically, the processing flow of the first branch does not have a synchronization primitive; the synchronization primitive corresponding to the processing flow of the second branch is a control statement for consumer dependency; the synchronization primitive corresponding to the processing flow of the third branch is a control statement for producer dependency; and the synchronization primitive corresponding to the processing flow of the fourth branch is a control statement for both consumer and producer dependency.

3. The method according to claim 1, characterized in that, Determining the data processing engine type based on the data processing instructions, and selecting the target engine based on the engine type, includes: When the data processing engine type parameter in the data processing instruction is a general engine, the general engine is determined as the target engine; The general engine encapsulates full configuration of data processing parameters, data processing pipeline barrier dependencies, data processing instruction issuance and execution, and scheduling state advancement. When the data processing engine type parameter in the data processing instruction is a dedicated engine, the dedicated engine is determined as the target engine; The dedicated engine encapsulates data processing incremental updates, scheduling authority management, and atomic instruction issuance and execution.

4. The method according to claim 1, characterized in that, The tensor queue encapsulates multiple levels of circular buffers; each level of buffer is equipped with a double barrier, which includes an empty barrier and a full barrier; the empty barrier is used to indicate that the corresponding level of buffer has been released by the consumer and the producer has the right to write; the full barrier is used to indicate that the corresponding level of buffer has been filled by the producer and the consumer has the right to read. When the target engine performs data processing between the source and destination based on the synchronization primitive, the producer waits for the empty barrier corresponding to the current level buffer accessed in the current round to be ready before writing data, and triggers a full barrier after data processing; the consumer waits for the full barrier corresponding to the current level buffer accessed in the current round to be ready before reading data, and triggers an empty barrier after data processing; a closed-loop data processing pipeline is formed by alternating triggering of dual barriers.

5. The method according to claim 1, characterized in that, The tensor queue is configured with a source data index cache and a coordinate mapping cache for the buffer. When the target engine performs data processing between the source and destination based on the synchronization primitive, if the source data index cache and the coordinate mapping cache of the buffer of the current data processing operation are the same as those of the previous data processing operation, then the target engine performs pipeline advancement between the source and destination based on the synchronization primitive, skipping the actual data processing execution action.

6. The method according to claim 1, characterized in that, The tensor queue establishes storage space ownership; the storage space ownership includes self-ownership mode and reference mode.

7. The method according to claim 1, characterized in that, Data processing instructions include at least one of the following: asynchronous slicing, asynchronous restore, asynchronous copy, asynchronous transpose, asynchronous instruction issuance, and asynchronous trimming.

8. An electronic device, characterized in that, The electronic device includes: At least one processor; and a memory communicatively connected to said at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the data processing method for the heterogeneous accelerator according to any one of claims 1-7.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the data processing method of the heterogeneous accelerator according to any one of claims 1-7.

10. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the data processing method for a heterogeneous accelerator according to any one of claims 1-7.

Images