A computing system, method, apparatus, medium, and program for a fusion operator

Abstract

The embodiment of the application discloses a kind of fusion operator computing system, method, equipment, medium and program, comprising: host side control plane, GPU, FPGA, shared storage area and hardware dependent fence;Host side control plane is used to identify the rule operator and irregular operator in the model to be executed to generate fusion operator graph, according to fusion operator graph and task issue condition detection result determines current to be distributed operator task and issue it to target operator execution equipment;GPU is used to execute the operator task corresponding to regular operator;FPGA is used to execute the operator task corresponding to irregular operator;Shared storage area is commonly accessed by GPU and FPGA, and is used to temporarily store, exchange and reuse intermediate dependent tensor generated and / or used in the process of executing operator task by GPU and FPGA;Hardware dependent fence is used to detect the storage data in shared storage area, to determine the state detection result of target intermediate dependent tensor.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of artificial intelligence technology, and in particular to a computing system, method, device, medium and program for a fusion operator. Background Technology

[0002] With the widespread application of various AI (Artificial Intelligence) model workloads, such as large language models, hybrid expert models, and multimodal models, in training and inference scenarios, single computing devices struggle to simultaneously handle both high-throughput dense computation and high-efficiency irregular computation. In current practice, GPUs (Graphics Processing Units) are typically well-suited for handling dense operators with strong regularity and high parallelism, such as General Matrix Multiply (GEMM), convolution, and attention matrix operations. However, irregular operators, such as routing, Top-K selection, Gather / Scatter, dynamic mask generation, index rearrangement, and conditional branch control, often exhibit characteristics like discrete memory access patterns, complex control logic, and significant thread dispersion. When executed on GPUs, these operations are prone to reduced resource utilization and higher latency.

[0003] To address the aforementioned contradictions, existing technologies typically employ the following solutions: The first solution involves the CPU (Central Processing Unit) handling irregular operators, while the GPU handles regular, dense operators. This solution is simple to implement, but intermediate results often need to be moved between host-side memory and GPU-side storage, resulting in long software synchronization links and potentially significant latency. The second solution uses an FPGA (Field Programmable Gate Array) for some preprocessing or post-processing, while the GPU handles the backbone operators. While this solution introduces heterogeneous devices, data exchange between devices usually still relies on Direct Memory Access (DMA) block transfers or explicit software coordination, resulting in insufficient fine-grained cross-device dependency management capabilities. The third solution expands accessible memory capacity through CXL (Compute Express Link). This type of solution primarily improves capacity but does not solve the problem of insufficient cross-device dependency management capabilities between the GPU and FPGA. The fourth solution employs a pure GPU or pure FPGA approach. Pure GPUs have limited efficiency in adapting to irregular operators, while pure FPGAs struggle to replace the throughput advantage of GPUs in large-scale dense parallel computing.

[0004] Therefore, in scenarios where dense and irregular operators coexist in the same or adjacent model layers, and these operators have close intermediate data dependencies, existing technologies typically have at least the following shortcomings: long cross-device intermediate result transmission paths and numerous data moves; dependencies are mostly controlled by coarse-grained synchronization on the software side, which can easily lead to unnecessary waiting; GPUs and FPGAs often cooperate in a staged serial manner, making it difficult to form operator-level collaborative execution; and there is a lack of unified addressing, unified state management, and unified scheduling mechanisms for shared intermediate tensors. Summary of the Invention

[0005] This invention provides a computing system, method, device, medium, and program for fusion operators, which can improve the computing efficiency of fusion operators and thus improve the inference throughput and training efficiency in single-node and multi-node scenarios in AI clusters.

[0006] According to one aspect of the present invention, a computing system with fusion operators is provided, configured on heterogeneous computing nodes, including a host-side control plane, a GPU, an FPGA, a shared memory region, and a hardware dependency barrier; wherein: The host-side control plane is communicatively connected to the GPU, the FPGA, the shared storage area, and the hardware dependency barrier. It is used to identify regular and irregular operators in the model to be executed, generate a fusion operator graph corresponding to the regular and irregular operators, determine the current operator task to be assigned based on the fusion operator graph and the task assignment condition detection result, and assign the current operator task to the corresponding target operator execution device according to the operator type of the current operator task. The target operator execution device includes the GPU and / or the FPGA; the task assignment condition detection result includes the state detection result of the target device and the target intermediate dependency tensor. The GPU is used to execute the operator tasks corresponding to the rule operators; The FPGA is used to execute the operator tasks corresponding to the irregular operator; The shared storage area is a shared intermediate data work area in the CXL consistency domain, configured for joint access by the GPU and the FPGA, used to temporarily store, exchange and reuse intermediate dependency tensors generated and / or used by the GPU and the FPGA during the execution of operator tasks; The hardware dependency barrier is used to detect the stored data in the shared storage area to determine the state detection result of the target intermediate dependency tensor.

[0007] According to another aspect of the present invention, a method for calculating a fusion operator is provided, applied to a calculation system for the fusion operator described in the first aspect, comprising: The GPU executes the first rule operator task to obtain the first intermediate dependency tensor, writes the first intermediate dependency tensor into the first logical address segment of the shared storage area, and sends the first shared segment commit record of the first intermediate dependency tensor to the hardware dependency fence. The hardware dependency barrier updates the state of the first logical address segment based on the first shared segment commit record; When the host-side control plane determines that the state of the first logical address segment and the FPGA meets the scheduling conditions, it issues the first irregular operator task to the FPGA. The FPGA reads the first target dependency data from the first logical address segment, executes the first irregular operator task according to the first target dependency data, obtains the second intermediate dependency tensor, writes the second intermediate dependency tensor into the second logical address segment of the shared storage area, and sends the second shared segment commit record of the second intermediate dependency tensor to the hardware dependency barrier. The hardware dependency barrier updates the state of the second logical address segment based on the second shared segment commit record; When the host-side control plane determines that the second logical address segment and the state of the GPU meet the scheduling conditions, it issues a second rule operator task to the GPU. The GPU reads the second target dependency data from the second logical address segment and executes the second rule operator task based on the second target dependency data.

[0008] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising a computing system for the fusion operator described in the first aspect: the computing system for the fusion operator is capable of executing the computing method for the fusion operator described in the second aspect.

[0009] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to execute and implement the calculation method of the fusion operator according to any embodiment of the present invention.

[0010] According to another aspect of the present invention, a computer program product is also provided, comprising a computer program that, when executed by a processor, implements the calculation method of the fusion operator described in any embodiment of the present invention.

[0011] This invention provides a computing system for fused operators, comprising a host-side control plane, GPU, FPGA, shared memory region, and hardware dependency fence. This system can be configured on heterogeneous computing nodes to perform fused computations on operators with dependencies. Specifically, the host-side control plane is communicatively connected to the GPU, FPGA, shared memory region, and hardware dependency fence. It identifies regular and irregular operators in the model to be executed, generates a fused operator graph corresponding to the regular and irregular operators, and determines the currently assigned operator task based on the fused operator graph and the task assignment condition detection results, including the state detection results of the target device and the target intermediate dependency tensor. Then, based on the operator type of the currently assigned operator task, it assigns the task to the corresponding target operator execution device, such as the GPU and / or FPGA. The GPU can execute the operator task corresponding to the regular operator, and the FPGA can execute the operator task corresponding to the irregular operator. The shared storage area, a shared intermediate data workspace within the CXL consistency domain, is configured for joint access by the GPU and FPGA. It allows for the temporary storage, exchange, and reuse of intermediate dependency tensors generated and / or used during the execution of operator tasks by the GPU and FPGA. Hardware dependency barriers can inspect the stored data in the shared storage area to determine the state detection results of the target intermediate dependency tensor. The aforementioned fusion operator computation system addresses the problems of high intermediate data exchange overhead, coarse synchronization granularity, and low operator computation efficiency caused by insufficient operator-level collaboration in existing GPU-FPGA collaborative schemes. It improves the computational efficiency of the fusion operator, thereby enhancing inference throughput and training efficiency in single-node and multi-node scenarios within AI clusters.

[0012] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0013] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0014] Figure 1 This is a schematic diagram of the structure of a computing system for a fusion operator provided in Embodiment 1 of the present invention; Figure 2 This is a schematic diagram of the internal structure of a computing system for another fusion operator provided in Embodiment 1 of the present invention; Figure 3This is a flowchart of a calculation method for a fusion operator provided in Embodiment 2 of the present invention; Figure 4 This is a schematic diagram of a GPU and FPGA fusion execution link provided in Embodiment 2 of the present invention; Figure 5 This is a flowchart illustrating an FPGA-GPU near-memory operator fusion computing method based on the CXL consistency domain, provided in Embodiment 2 of the present invention. Figure 6 This is a flowchart illustrating the process of interleaved scheduling of multiple batches of operator tasks in a computing system for fusion operators provided in Embodiment 2 of the present invention. Figure 7 This is a schematic diagram of a logical address segment state management and SCR-HDF dependency release mechanism provided by the second invention; Figure 8 This is a schematic diagram of the structure of an electronic device provided in Embodiment 3 of the present invention. Detailed Implementation

[0015] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0016] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0017] Example 1 Figure 1 This is a schematic diagram of the structure of a computing system with a fusion operator according to Embodiment 1 of the present invention. This computing system with a fusion operator can be configured on heterogeneous computing nodes. For example... Figure 1As shown, the computing system architecture of this fusion operator includes: a host-side control plane 110, a GPU 120, an FPGA 130, a shared memory region 140, and a hardware dependency barrier 150; wherein: The host-side control plane 110 is communicatively connected to the GPU 120, FPGA 130, shared memory area 140, and hardware dependency barrier 150. It is used to identify regular and irregular operators in the model to be executed, generate a fused operator graph corresponding to the regular and irregular operators, determine the current operator task to be assigned based on the fused operator graph and the task assignment condition detection result, and assign the current operator task to the corresponding target operator execution device according to the operator type of the current operator task. The target operator execution device includes the GPU and / or the FPGA; the task assignment condition detection result includes the state detection result of the target device and the target intermediate dependency tensor.

[0018] The host-side control plane 110 can be a control logic platform deployed on the host's processor, used to interact with peripheral devices and complete tasks such as device initialization, capability negotiation, parameter configuration, control command issuance, and status acquisition. The host-side control plane can generate control commands and issue them to the devices through hardware interfaces, and the device's data plane will then perform data processing based on these commands.

[0019] The model to be executed can be an AI model that needs to run through heterogeneous computing nodes, such as, but not limited to, expert network models and various large models. This embodiment of the invention does not limit the type of model to be executed. The large model can be any available Large Language Model (LLM). A large language model, also called a large-scale language model, refers to a deep learning model trained with a large amount of relevant data (such as text data, speech data, or combined text and image data) to process a sequence (such as text, speech, image, audio / video, or combined text and image sequences). It can generate natural language text or understand the meaning of language text. These models typically have billions of parameters. Large language models can handle various natural language tasks, such as text classification, question answering, and dialogue, and have wide applications. The input to a large language model is data, such as text data, speech data, or combined text and image data. The large language model encodes the input data to obtain corresponding word vector representations, and then decodes the encoded word vectors to automatically complete the input data processing and obtain the corresponding output data. For example, text can be input into a large language model, which then processes and predicts the input text to output the corresponding response text.

[0020] The fused operator graph is the computation graph obtained after operator fusion optimization of the original operator computation graph. Fusion optimization can be understood as identifying the dependencies between operators in the operator graph. That is, the fused operator graph can clearly and intuitively display the dependencies between operators. These dependencies can include the dependencies between input and output data. The task assignment condition detection result is the result of checking whether the operator meets the assignment conditions before assigning the task. The currently assigned operator task can be the operator task that can be assigned to the corresponding device. An operator task is a schedulable execution instance generated by the operator at runtime; it carries the input / output tensors, data dependencies, execution device information, and runtime context corresponding to this computation. The target operator execution device can be a device capable of executing the currently assigned operator task. The target device can be the device whose execution status is checked before the currently assigned operator task is assigned. The target intermediate dependency tensor can be the tensor that the currently assigned operator task needs to depend on during the computation process.

[0021] Figure 2 This is a schematic diagram of the internal structure of a computing system for another fusion operator provided in Embodiment 1 of the present invention. In a specific example, such as Figure 2 As shown in this embodiment of the invention, the internal structure of the host-side control plane may include, but is not limited to, a host CPU, a fusion scheduler, an address management module, and a CXL consistency interconnect module. The host CPU can perform system initialization, device management, and runtime control. The host CPU can also identify regular and irregular operators in the model to be executed, and configure the dependencies between operators based on the original operator graph corresponding to the model, thereby generating a fusion operator graph corresponding to the regular and irregular operators, thus achieving finer-grained heterogeneous collaboration during the execution of the model. The fusion scheduler can be deployed on the host-side control plane, or it can be deployed in a runtime module that collaborates with the host-side control plane; this embodiment of the invention does not limit the deployment method of the fusion scheduler. Accordingly, during the execution of the model, the host-side control plane can use the fusion scheduler to uniformly schedule and distribute various operator tasks, such as different GPU tasks and FPGA tasks. Specifically, the fusion scheduler can determine the operator tasks that need to be executed according to the fusion operator graph, and perform pre-task distribution checks on the operator tasks that need to be executed. Based on the task distribution condition detection results of the operator tasks that need to be executed, it determines the currently available operator tasks to be distributed and assigned, and then distributes the currently available operator tasks to the corresponding target operator execution devices such as GPUs or FPGAs according to the operator type of the currently available operator tasks.

[0022] Specifically, the fusion scheduler can detect the execution status of the target device corresponding to each currently executing operator task to determine the device's availability. If the target device is confirmed to be available, it passes the status check. Simultaneously, the fusion scheduler can also detect the status of the target intermediate dependency tensors that each currently executing operator task depends on, determining whether the status of the target intermediate dependency tensors meets the execution conditions of the operator task. For example, when the status of the target intermediate dependency tensor is "ready," it indicates that the target intermediate dependency tensor has been processed and meets the execution conditions of the operator task. Only when the status of both the target device and the target intermediate dependency tensor corresponding to the currently executing operator task passes the checks can the fusion scheduler determine that the task assignment condition check result for the currently executing operator task is passed. Correspondingly, the currently executing operator task with a passed task assignment condition check result can be identified as the currently pending operator task. Therefore, the fusion scheduler can uniformly schedule GPU and FPGA tasks based on the fusion operator graph, the status of the intermediate tensors that the operators depend on, and device availability. The scheduling method of the fusion scheduler can be either round-robin or event-driven, and the embodiments of the present invention do not limit it in this regard.

[0023] It should be noted that there can be multiple currently unassigned operator tasks, and the target operator execution device corresponding to each currently unassigned operator task can be the same or different. That is, the target operator execution device corresponding to each target operator execution device may be a GPU 120 or an FPGA 130.

[0024] like Figure 2 As shown, the host-side control plane 110 may further include an address management module and a CXL coherence interconnect. The address management module can be used for address mapping and segment management of logical address segments in the shared memory region; the CXL coherence interconnect can be used to provide shared coherence access capabilities within the coherence domain, enabling the host CPU, GPU, and FPGA to exchange data and perform collaborative processing around the shared memory region.

[0025] GPU 120 is used to execute operator tasks corresponding to regular operators, and FPGA 130 is used to execute operator tasks corresponding to irregular operators.

[0026] Regular operators can be operators with regular data, predictable memory access patterns, fixed computational graph structures, and high parallelism; they can also be called dense operators. Irregular operators can be operators with dynamically variable shapes, sparseness, random memory access, complex control flow, and difficulty in hardware acceleration. For example, regular operators can include, but are not limited to, generalized matrix multiplication, convolution, linear projection, normalization, and attention backbone matrix operations; irregular operators can include, but are not limited to, routing, top K-item filtering, aggregation / scattering, dynamic mask generation, index rearrangement, and conditional branch control.

[0027] Specifically, the GPU 120 and FPGA 130 can directly access intermediate dependency tensors stored in the shared memory area, or write the output of the currently executed operator task to the shared memory area. The FPGA 130 can be configured with access agents, status reporting logic, and irregular operator execution cores to complete data processing around the shared memory area.

[0028] Shared storage area 140 can be a shared intermediate data workspace in the CXL consistency domain, configured for joint access by GPU 120 and FPGA 130, for temporary storage, exchange and reuse of intermediate dependency tensors generated and / or used during the execution of operator tasks by GPU 120 and FPGA 130.

[0029] Among them, intermediate dependency tensors can be data stored in a shared memory area, or tensors generated by a certain operator that need to be used by other operators for computation.

[0030] like Figure 2As shown, the shared storage area can be divided into multiple logical address segments, such as S1, S2, and S3, based on the dependencies between the operators included in the fusion operator graph. Each logical address segment can be used to store intermediate dependency tensors generated by a certain device (FPGA or GPU). For example, S1 can store intermediate dependency tensors generated when the GPU executes the operator task of operator 1, and S2 can store intermediate dependency tensors generated when the FPGA executes the operator task of operator 2, etc. Optionally, the shared storage area can be a software-based cache area or a hardware-based storage area. This embodiment of the invention does not limit the specific storage method that the shared storage area can use. The shared storage area can be physically implemented as a storage area with shared access capabilities, or it can be formed by combining multiple work area levels. For example, the shared storage area can be a single physical shared storage area, or it can be a combination of one or more levels of FPGA on-chip low-latency storage, FPGA external high-bandwidth storage, and CXL extended memory resources. However, the address management module or the corresponding memory management logic will make the shared storage area appear to the outside world as a logically continuous near-memory shared intermediate data space under unified address management, so as to take into account the requirements of low-latency access, capacity expansion, and hierarchical placement of intermediate results.

[0031] Optionally, the shared memory region can be mapped to a shared address space that can be directly accessed by the GPU and FPGA via CXL.mem, enabling both types of devices to perform read, write, and multiplexing operations on the corresponding shared logical segments around a unified address view. CXL.mem is a dedicated protocol for memory access in the CXL standard. It is built on the existing PCIe physical layer and electrical interface, based on the PCIe 5.0 physical layer.

[0032] Therefore, the shared storage region, as a near-memory shared workspace for heterogeneous device collaborative execution links, is mainly used to carry short-lifecycle intermediate results transferred between GPUs and FPGAs, enabling relevant intermediate tensors to be temporarily stored, exchanged, and reused closer to the heterogeneous device collaborative processing path. Compared with the method of moving intermediate results back and forth through host memory, this shared storage region not only provides shared access capabilities but also serves subsequent dependency release and fusion scheduling based on the shared logical segment state.

[0033] The Hardware Dependency Fence 150, also known as HDF, is used to inspect the stored data in the shared memory region 140 to determine the state detection results of the target intermediate dependency tensor. Specifically, the Hardware Dependency Fence 150 can be a module used for state management of the stored data in the shared memory region 140 and to provide task scheduling assistance information to the fusion scheduler. The GPU 120 and FPGA 130 can access the shared memory region 140 via the CXL coherent interconnect and complete the collaborative processing of the fusion operator under the dependency control of the Hardware Dependency Fence 150.

[0034] In this embodiment of the invention, the hardware dependency barrier 150 can track the status of each logical address segment in the shared memory region 140 used for storing intermediate dependency data. Optionally, the status information maintained by the hardware dependency barrier 150 for the shared memory region 140 may include, but is not limited to, at least one or more of the following: segment identifier, version number, producer identifier, consumer identifier, and completion status. The segment identifier can be used to uniquely identify a logical address segment. The version number can be used to identify the current version of the logical address segment. The producer identifier can be used to identify the device that generates the intermediate dependency data in the logical address segment. The consumer identifier can be used to identify the device that consumes the intermediate dependency data in the logical address segment. The completion status can identify the current processing status of the intermediate dependency data in the logical address segment through various different status types.

[0035] For intermediate dependent data written to a logical address segment in shared memory region 140, the device that generated the intermediate dependent data simultaneously generates a corresponding SCR (Shared Segment Commit Record) after the write is completed. The SCR is used to indicate that the logical address segment has met the predetermined readable conditions. Hardware dependency barrier 150 and / or fusion scheduler can perform status checks on the corresponding logical address segment before subsequent consumer tasks start; if the status of the logical address segment meets the requirements, the corresponding consumer task is released; if the status of the logical address segment does not meet the requirements, the delivery is postponed. With this mechanism, the additional waiting caused by using the completion of the entire batch or the entire task as the synchronization condition can be avoided.

[0036] Optionally, the hardware-dependent barrier 150 can be a software-based logic control unit or a hardware-based logic control unit, such as a programmable logic controller or a microprogrammable controller. This embodiment of the invention does not limit the implementation form of the hardware-dependent barrier 150.

[0037] The entire computational system of the fusion operator can adopt a unified scheduling mechanism. During the operation of the model to be executed, the host-side control plane 110 can scan the operator tasks corresponding to the operators to be executed in the fusion operator graph according to the topological order through the fusion scheduler, and check whether the operator tasks corresponding to the operators to be executed meet the following task delivery conditions: (1) whether the intermediate dependency tensors of the predecessor have been generated; (2) whether there are read / write conflicts or overwrite conflicts in the logical address segment storing the intermediate dependency tensors; (3) whether the execution resources of the corresponding device corresponding to the operator to be executed are available; (4) whether the current version of the logical address segment storing the intermediate dependency tensors matches the current task. When all the above task delivery conditions are met, the fusion scheduler can currently deliver the operator tasks corresponding to the operators to be executed according to the type of the operators to be executed. If the operator to be executed is a regular operator, the operator tasks corresponding to the current operator to be executed can be sent to the GPU side for execution; if the operator to be executed is an irregular operator, the operator tasks corresponding to the current operator to be executed can be sent to the FPGA side for execution. If at least one of the above-mentioned task issuance conditions is not met, the operator task corresponding to the currently pending operator is retained and the process waits for the next round of detection or event notification.

[0038] Therefore, the computing system for fusion operators provided in this embodiment of the invention does not simply place the GPU and FPGA in a heterogeneous computing node, but rather allows them to collaboratively process fusion operators around a shared memory region within the CXL consistency domain. After the model to be executed is parsed, a fusion operator graph is generated. By uniformly orchestrating regular operators suitable for GPUs and irregular operators suitable for FPGAs, and writing the intermediate results that each operator depends on into the logical address segment of the shared memory region, it is submitted to the hardware dependency barrier via SCR. The hardware dependency barrier then releases subsequent tasks with fine-grained precision based on the shared segment status, batch number, version number, and dependency relationship. This reduces explicit intermediate data movement, reduces software coarse synchronization, supports multi-batch interleaved scheduling, and improves overall throughput.

[0039] In an optional embodiment of the present invention, the host-side control plane 110 may also be used to: detect the host, the GPU, the FPGA and interconnect resources; and determine the storage address range, device access relationship and device dependency relationship of the shared storage area based on the detection results.

[0040] Before computing the fusion operator, the host-side control plane 110 can also probe the host, GPU 120, FPGA 130, and interconnect resources. On one hand, by probing the hardware computing capabilities of the acceleration devices, the host-side control plane 110 can obtain hardware constraints such as the operators supported by the devices, fusion operators, and tensor specifications. This ensures that when the host side constructs the fusion operator graph and generates operator tasks, it only generates tasks that the device hardware supports, leaving unsupported operator tasks to be executed on the host side, thus avoiding the issuance of invalid tasks. On the other hand, probing the queue and bandwidth capabilities of interconnect resources can be used to rationally allocate transmission resources, constrain data transfer between host devices, and prevent transmission anomalies caused by exceeding interconnect resource limits. Simultaneously, the probe results can be used to complete capability negotiation, task scheduling, and load adaptation between the host CPU and devices, ensuring the correct operation of the heterogeneous system. Correspondingly, the host-side control plane 110 can determine the storage address range, device access relationships, and device dependencies of the shared storage area based on the probe results. Among these, the device access relationships characterize the hardware-level access path capabilities between physical devices, recording whether the access path between devices is a direct or relay access path. Device dependencies can characterize the pre-initial constraints between hardware devices, representing the other underlying hardware devices that a device depends on for it to function properly, and are used to control the order of device initialization and reset.

[0041] In an optional embodiment of the present invention, the host-side control plane 110 may also be used to: perform address planning for the shared memory region within the storage address range of the shared memory region, generate multiple logical address segments of the shared memory region; and establish a mapping relationship between each of the logical address segments and the GPU and the FPGA through the CXL memory protocol.

[0042] During system initialization, the host-side control plane 110 can perform address planning for the shared memory region 140 through the address management module, establishing the mapping relationship required for the GPU 120 and FPGA 130 to access the shared memory region 140. Specifically, the address management module can determine the optional range of storage addresses that can be used to generate the shared memory region 140 based on the resource probing results of the host-side control plane 110, and then perform address planning for the shared memory region 140 within the determined storage address range, generating multiple logical address segments for the shared memory region, and establishing the mapping relationship between each logical address segment and the GPU 120 and FPGA 130 through the CXL memory protocol (such as CXL.mem). Each logical address segment can be identified and associated using a unique segment identifier. Optionally, the address management module can allocate different logical address segments for different operator outputs to facilitate dependency state management and conflict detection. Optionally, the address management module can be implemented collaboratively by one or more parts of the host driver, mapping table management logic, and device-side access agent.

[0043] In an optional embodiment of the present invention, after completing the operator task, the producer device writes task result data to the target logical address segment of the shared storage region; wherein, the producer device may include GPU 120 and / or FPGA 130; after determining that the task result data has been written to the target logical address segment, the producer device submits a shared segment commit record to the hardware dependency barrier; the hardware dependency barrier is further used to update the state of the target logical address segment and the consistency maintenance action of the CXL core transaction layer protocol of the target logical address segment according to the shared segment commit record; the hardware dependency barrier is further used to trigger the consistency maintenance action of the CXL core transaction layer protocol for consumer devices when it is determined that the state detection result of the target intermediate dependency tensor meets the task delivery conditions.

[0044] In this context, the producer device can be a device used to generate intermediate dependency tensors; the producer device can be a GPU or an FPGA. The target logical address segment can be the logical address segment where the intermediate dependency tensor needs to be written. The shared segment commit record can be used to indicate that the producer device has written the intermediate dependency tensor to the corresponding logical address segment. The consumer device can be a device that uses the intermediate dependency tensor to perform operator tasks.

[0045] Specifically, after the producer device (which can be a GPU or FPGA) completes the operator task, it can use some or all of the data from the operator task processing result as intermediate dependency tensors and store the determined intermediate dependency tensors in the target logical address segment corresponding to the shared memory area 140. For example, the GPU, as a producer device, can write the intermediate dependency tensor corresponding to operator 1 to the logical address segment S1 and the intermediate dependency tensor corresponding to operator 3 to the logical address segment S3. Simultaneously, the producer device can submit a shared segment commit record for the corresponding target logical address segment to the hardware dependency barrier 150, indicating that the intermediate dependency tensor of the target logical address segment has been written. The hardware dependency barrier 150 can update the status of the target logical address segment based on the shared segment commit record. For example, the status of each logical address segment can include, but is not limited to, idle, writing, ready, released, consumed, and reusable. Optionally, the hardware dependency barrier 150 and / or the fusion scheduler can detect the status of the logical address segments that the consumer device depends on before startup, and release the corresponding consumer task execution when the dependency is satisfied.

[0046] For example, after the producer device writes intermediate dependency data into the target logical address segment, it submits the shared segment commit record corresponding to the target logical address segment. In response to the shared segment commit record, the hardware dependency barrier 150 updates the current state of the target logical address segment to the ready state. The hardware dependency barrier 150 can perform dependency determination by combining segment identifier, batch number, version number, and consumer dependency relationship. When the determination meets the release conditions, the hardware dependency barrier 150 releases the corresponding consumer task for the fusion scheduler to continue executing. After the consumer device (which can be a GPU or FPGA) completes the reading or processing of the intermediate dependency tensor, the current state of the target logical address segment corresponding to the intermediate dependency tensor enters the consumed state or reusable state, for subsequent task reallocation or overwriting.

[0047] Optionally, after receiving the shared segment commit record submitted by the producer device, the hardware dependency barrier 150 can, in addition to updating the state of the target logical address segment, also trigger consistency maintenance actions of the CXL core transaction layer protocol for the target logical address segment and the logical address segments corresponding to the subsequent consumer tasks (i.e., the operator tasks that the consumer devices need to execute), such as triggering consistency maintenance actions of CXL.cache (cache). For example, the consistency maintenance actions of the CXL core transaction layer protocol triggered for the target logical address segment and the logical address segments corresponding to the subsequent consumer tasks may include, but are not limited to, cache line invalidation, update notification, or visibility confirmation, to ensure that the released operator tasks can read the latest data that meets consistency requirements when subsequently accessing the relevant logical address segments through CXL.mem. The hardware dependency barrier 150 can also trigger consistency maintenance actions of the CXL core transaction layer protocol for consumer devices if the state detection result of the target intermediate dependency tensor meets the task delivery conditions, to ensure that the operator tasks of the subsequent consumer devices can read the latest data that meets consistency requirements when subsequently accessing the logical address segment through CXL.mem. For example, consistency maintenance actions of the CXL core transaction layer protocol for consumer devices may include, but are not limited to, invalidating, updating notifications, or confirming visibility of relevant cache lines on the consumer device side.

[0048] In an optional embodiment of the present invention, the host-side control plane 110 may also be used to: write the current operator task to be assigned into the target queue of the target device according to the operator type of the current operator task when it is determined that the current operator task to be assigned is a dependent operator task; and send the independent operator task to the target device in parallel according to the operator type of the independent operator task when it is determined that the current operator task to be assigned is an independent operator task.

[0049] Among them, a dependent operator task can be an operator task that needs to rely on the output data of other operators. An independent operator task can be an operator task that can run independently and does not need to rely on the output data of other operators.

[0050] In this embodiment of the invention, the host-side control plane 110 can maintain ready queues or unified ready tables for GPU 120 and FPGA 130 through a fusion scheduler, and continuously scan candidate nodes according to the topological order of the fusion operator graph to determine the currently assigned operator task from the scanned subsequent nodes based on the task assignment condition detection results. For currently assigned operator tasks whose predecessor dependencies have been confirmed to be satisfied by the hardware dependency barrier 150, whose logical address segments have no access conflicts, and whose target device resources are available, the fusion scheduler can write them into the target queue of the target device, such as writing GPU operator tasks into the GPU's command queue, and FPGA operator tasks into the FPGA's task queue, etc. The fusion scheduler performs parallel assignment for mutually independent operator tasks with logical address segment dependencies, thereby enabling GPU and FPGA to form operator-level collaborative execution around the shared memory area, and reducing explicit copying and invalid waiting while ensuring the correctness of data dependencies. For independent operator tasks, the fusion scheduler can also assign them in parallel to the corresponding target device, such as GPU or FPGA, according to the operator type of the independent operator task. Parallel assignment of operator tasks can significantly improve the execution efficiency of operators.

[0051] It should be noted that even existing technologies, despite possessing unified addressing, shared memory access, or high-speed interconnection capabilities between devices, mostly remain at the data reachability or block-level transmission level. They typically employ synchronization control at the whole task, whole operator, or whole batch granularity, making it difficult to form a fine-grained dependency release mechanism around cross-device shared intermediate results. The key to the computing system of the fusion operator provided in this embodiment of the invention does not lie in using any single element from CXL, GPU, or FPGA, but rather in using a shared storage area as the cross-device intermediate result working area within the CXL consistency domain, and using each logical address segment of the shared storage area as the dependency management granularity. Through the shared segment commit record (SCR) and hardware dependency barriers, the state management of logical address segments achieves segment-level dependency satisfaction determination and subsequent operator task release. The fusion scheduler then performs unified cross-device operator-level scheduling based on this, supporting the fusion execution of dense and irregular operators. Thus, during the computation of the fusion operator, explicit copying and invalid waiting are reduced while ensuring data dependency correctness, thereby improving the computational efficiency of the fusion operator and ultimately improving inference throughput and training efficiency in single-node and multi-node scenarios in AI clusters.

[0052] For example, such as Figure 2 As shown, Figure 2The arrows in the diagram represent control, mapping, shared access, and task dependency release relationships. Straight arrows indicate control paths, shared access paths, or read / write paths; dashed arrows indicate status feedback, SCR submission, and task dependency release relationships. The host-side control plane provides task scheduling and address mapping support to the shared memory area and related execution units. The GPU and FPGA perform read / write operations on the shared memory area respectively. The hardware dependency barrier releases subsequent operator tasks based on SCR and dependency detection results, thereby reducing cross-device data transfer overhead and improving the execution efficiency of fused operators.

[0053] In summary, this invention provides a computing system for fused computing of FPGA-GPU near-memory operators based on CXL consistency domains. This system can be deployed within a single heterogeneous computing node and includes at least a host-side control plane, a GPU accelerator, an FPGA programmable logic device, a shared memory region, and a hardware dependency barrier. The shared memory region is a CXL interconnect infrastructure that supports shared consistency access capabilities. The host-side control plane, configured with a host CPU, a fused scheduler, an address management module, and a CXL consistency interconnect, is used to complete device initialization, shared address planning, fused operator graph generation, and runtime task distribution. The shared memory region stores intermediate dependency tensors flowing across devices. The hardware dependency barrier records the state of shared logic segments and controls the release of subsequent tasks. In this fused computing system, the GPU preferably executes dense operators with strong regularity, while the FPGA preferably executes irregular operators with strong memory access or control characteristics. Both perform intermediate result reading, writing, and reuse around the same shared memory region, without repeatedly moving data through independent buffers at each operator boundary. For cross-device dependencies, the producer device submits the shared segment commit record corresponding to the logical address segment after the write is completed, and the hardware dependency barrier determines whether the dependency is satisfied, so as to release the subsequent consumer task when the condition is met.

[0054] This invention provides a computing system for fused operators, comprising a host-side control plane, GPU, FPGA, shared memory region, and hardware dependency fence. This system can be configured on heterogeneous computing nodes to perform fused computations on operators with dependencies. Specifically, the host-side control plane is communicatively connected to the GPU, FPGA, shared memory region, and hardware dependency fence. It identifies regular and irregular operators in the model to be executed, generates a fused operator graph corresponding to the regular and irregular operators, and determines the currently assigned operator task based on the fused operator graph and the task assignment condition detection results, including the state detection results of the target device and the target intermediate dependency tensor. Then, based on the operator type of the currently assigned operator task, it assigns the task to the corresponding target operator execution device, such as the GPU and / or FPGA. The GPU can execute the operator task corresponding to the regular operator, and the FPGA can execute the operator task corresponding to the irregular operator. The shared storage area, a shared intermediate data workspace within the CXL consistency domain, is configured for joint access by the GPU and FPGA. It allows for the temporary storage, exchange, and reuse of intermediate dependency tensors generated and / or used during the execution of operator tasks by the GPU and FPGA. Hardware dependency barriers can inspect the stored data in the shared storage area to determine the state detection results of the target intermediate dependency tensor. The aforementioned fusion operator computation system addresses the problems of high intermediate data exchange overhead, coarse synchronization granularity, and low operator computation efficiency caused by insufficient operator-level collaboration in existing GPU-FPGA collaborative schemes. It improves the computational efficiency of the fusion operator, thereby enhancing inference throughput and training efficiency in single-node and multi-node scenarios within AI clusters.

[0055] Example 2 Figure 3 This is a flowchart of a calculation method for a fusion operator provided in Embodiment 2 of the present invention. This embodiment is applicable to situations where a fusion operator-based computing system executes the fusion operator. The method can be executed by the fusion operator computing system, which can be configured in a heterogeneous computing node. This heterogeneous computing node can be a terminal device or a server device. The present invention does not limit the specific device type of the heterogeneous computing node. Correspondingly, as... Figure 3 As shown, the method includes the following operations: S110, the GPU executes the first rule operator task to obtain the first intermediate dependency tensor, writes the first intermediate dependency tensor into the first logical address segment of the shared storage area, and sends the first shared segment commit record of the first intermediate dependency tensor to the hardware dependency barrier.

[0056] Here, the first rule operator task can be an operator task of a rule operator currently being executed by the GPU. The first intermediate dependency tensor can be an intermediate dependency tensor corresponding to the first rule operator task, generated after the GPU executes the first rule operator task. The first logical address segment can be a logical address segment used to store the first intermediate dependency tensor of the first rule operator task. The first shared segment commit record can be a shared segment commit record of the first intermediate dependency tensor.

[0057] Figure 4 This is a schematic diagram of a GPU and FPGA fusion execution link provided in Embodiment 2 of the present invention. In a specific example, such as Figure 4 As shown, taking a model layer including Mixture of Experts (MoE) routing and attention-related computations as an example, the GPU receives a first rule operator task from the fusion scheduler of the host-side control plane. This first rule operator task can be a query / key / value (Q / K / V) operator task, a linear projection task, a matrix computation task, or an Expert (an independent sub-network in the Mixture of Experts model architecture responsible for processing specific types of data) input generation task. After the GPU processes the first rule operator task, it generates a first intermediate dependency tensor based on the output of the first rule operator task. The first intermediate dependency tensor can be part or all of the output of the first rule operator task. Simultaneously, the GPU writes the first intermediate dependency tensor into the first logical address segment S1 of the shared memory region and sends the first shared segment commit record SCR1 of the first intermediate dependency tensor to the hardware dependency fence. Figure 4 As shown, the shared storage area can be used to store intermediate results that are reused across devices, and it can be divided into a first logical address segment S1, a second logical address segment S2, and subsequent logical address segments.

[0058] S120. The hardware dependency barrier updates the status of the first logical address segment based on the first shared segment commit record.

[0059] Correspondingly, such as Figure 4 As shown, when the hardware dependency barrier receives the first shared segment commit record SCR1 of the first logical address segment S1, it updates the state of the first logical address segment S1 to be ready and notifies the fusion scheduler of the host-side control plane that it can continue to issue operator tasks that depend on the first logical address segment S1.

[0060] S130. When the host-side control plane determines that the state of the first logical address segment and the FPGA meets the scheduling conditions, it issues the first irregular operator task to the FPGA.

[0061] The first irregular operator task can be an operator task of an irregular operator executed by the FPGA. The first irregular operator task needs to rely on the first intermediate dependency tensor of the first regular operator task for execution.

[0062] Furthermore, such as Figure 4 As shown, the host-side control plane can detect whether the FPGA's status is available. If the host-side control plane determines that the status of the first logical address segment S1 and the FPGA simultaneously meet the scheduling conditions—that is, the first logical address segment S1 is in a ready state and the FPGA is currently in an executable state—then the first irregular operator task can be issued to the FPGA through the fusion scheduler. For example, the first irregular operator task can include, but is not limited to, operator tasks involving irregular operators such as route determination, Top-K filtering, or mask generation.

[0063] S140, the FPGA reads the first target dependency data from the first logical address segment, executes the first irregular operator task according to the first target dependency data, obtains the second intermediate dependency tensor, writes the second intermediate dependency tensor into the second logical address segment of the shared storage area, and sends the second shared segment commit record of the second intermediate dependency tensor to the hardware dependency barrier.

[0064] The first target dependency data can be the intermediate tensor data required by the FPGA to execute the first irregular operator task. It can be understood that the first target dependency data can be part or all of the data in the first intermediate dependency tensor. The second intermediate dependency tensor can be the intermediate dependency tensor corresponding to the first irregular operator task, generated after the FPGA executes the first irregular operator task. The second logical address segment can be the logical address segment used to store the second intermediate dependency tensor of the first irregular operator task. The second shared segment commit record can be the shared segment commit record of the second intermediate dependency tensor.

[0065] like Figure 4 As shown, after receiving the first irregular operator task from the fusion scheduler, the FPGA reads part or all of the data of the first intermediate dependency tensor from the first logical address segment S1 to obtain the first target dependency data, and then executes the first irregular operator task based on the first target dependency data. After the FPGA completes the processing of the first irregular operator task, it generates a second intermediate dependency tensor based on the output result of the first irregular operator task. The second intermediate dependency tensor can be part or all of the output result of the first irregular operator task. At the same time, the FPGA writes the second intermediate dependency tensor into the second logical address segment S2 of the shared memory region and sends the second shared segment commit record SCR2 of the second intermediate dependency tensor to the hardware dependency barrier.

[0066] Therefore, the hardware dependency barrier can receive the Shared Segment Commit Record (SCR) submitted by the producer device when executing operator tasks, and release the corresponding consumer device's operator tasks according to the state of the logical address segment.

[0067] S150, The hardware-dependent barrier updates the state of the second logical address segment based on the second shared segment commit record.

[0068] Correspondingly, such as Figure 4 As shown, when the hardware-dependent barrier receives the second shared segment commit record SCR2 of the second logical address segment S2, it updates the state of the second logical address segment S2 to be ready and notifies the fusion scheduler of the host-side control plane that it can continue to issue subsequent operator tasks.

[0069] S160. When the host-side control plane determines that the second logical address segment and the state of the GPU meet the scheduling conditions, it issues a second rule operator task to the GPU.

[0070] The second rule operator task can be an operator task of another rule operator that is executed sequentially by the GPU.

[0071] Furthermore, such as Figure 4 As shown, the host-side control plane can detect whether the GPU is available. If the host-side control plane determines that the second logical address segment S2 and the GPU's state simultaneously meet the scheduling conditions—that is, the second logical address segment S2 is ready and the GPU is currently in an executable state—then the second rule operator task can be issued to the GPU through the fusion scheduler. For example, the second rule operator task can include, but is not limited to, Expert Gather, Attention-related calculations, Softmax normalization, weighted aggregation, or output transformation. Among these, Expert Gather is a key step in the forward propagation of the hybrid expert model, referring to the process of re-aggregating (summarizing) the output results scattered across different expert sub-networks into a unified tensor according to routing weights to generate the final output of that layer. S170, the GPU reads the second target dependency data from the second logical address segment and executes the second rule operator task based on the second target dependency data.

[0072] The second target dependency data can be the intermediate tensor data required by the GPU to execute the second rule operator task. It can be understood that the second target dependency data can be part or all of the data from the second intermediate dependency tensor.

[0073] like Figure 4As shown, after the GPU receives the second rule operator task issued by the fusion scheduler, it reads part or all of the data of the second intermediate dependency tensor from the second logical address segment S2 to obtain the second target dependency data, and then executes the second rule operator task according to the second target dependency data.

[0074] In summary, such as Figure 4 As shown, in the model layer including MoE routing and Attention-related calculations, the GPU first completes the first regular operator task, such as Q / K / V linear projection or Expert input generation, and obtains the first intermediate dependency tensor, which is written to the first logical address segment S1. Then, it submits the first shared segment submission record SCR1. The hardware dependency barrier releases the FPGA's first irregular operator task, reads some or all of the data from the first logical address segment S1, performs irregular processing such as MoE routing judgment and Top-K Expert filtering, and writes it back to the second logical address segment S2. Subsequently, the FPGA submits the second shared segment submission record SCR2 again, the hardware dependency barrier releases the GPU's second regular operator task, and the GPU reads the second logical address segment S2 to continue executing second regular operator tasks such as ExpertGather, Attention calculation, Softmax, or output transformation. If there are still irregular operator tasks suitable for FPGA processing, the corresponding intermediate results are written to a new logical address segment, and the above collaborative process of "producer device submitting intermediate dependency data—HDF detection—consumer device releasing operator tasks" is repeated. It should be noted that... Figure 4 The CXL coherent interconnect in the context of GPUs, shared memory regions, and FPGAs is placed in the same coherent access domain to support consistent access to intermediate results shared across devices.

[0075] In the aforementioned GPU and FPGA integrated execution chain, the GPU and FPGA do not require strict full-layer serial switching, but can instead collaborate more finely based on logical segment dependencies. Shared memory areas are used to store critical intermediate results reused across devices, reducing explicit data movement. Hardware dependency barriers track status by logical address segment, rather than synchronizing only by the overall task completion status.

[0076] Figure 5 This is a flowchart illustrating a CXL consistency domain-based FPGA-GPU near-memory operator fusion computing method provided in Embodiment 2 of the present invention. The flowchart describes the overall process of the fusion operator computing system during actual operation, from initialization configuration, shared address establishment, and fusion operator graph generation, to task scheduling, logical address segment state submission, dependency detection, and loop execution completion. In a specific example, such as... Figure 5 As shown, the FPGA-GPU near-memory operator fusion computation method based on the CXL consistency domain can include the following operations: Step 1: The host-side control plane performs node capability detection and initialization configuration.

[0077] Specifically, the host-side control plane can detect the host, GPU, FPGA and CXL interconnect resources, establish the basic configuration of the availability range of the shared storage area, device access capabilities and dependency management capabilities, and initialize the fusion scheduler, address management module and hardware dependency barrier.

[0078] Step 2: The host-side control plane can establish a mapping relationship for the shared storage area.

[0079] Specifically, the host-side control plane can perform address planning for the shared memory area, forming a shared memory area that is accessed by both the GPU and FPGA. The shared memory area is divided into multiple logical address segments, and a mapping relationship between each logical address segment and the address interval is established.

[0080] Step 3: Analyze the model graph and generate the fusion operator graph.

[0081] In the process of generating the fusion operator graph, regular and irregular operators in the model to be executed can be identified, operator dependency edges, logical address segment allocation information and task execution description can be established, and then the fusion operator graph can be generated.

[0082] Step 4: Load the execution kernel and form the runtime context.

[0083] Specifically, regular operator executors can be loaded onto the GPU side, irregular operator executors or corresponding configurations can be loaded onto the FPGA side, and a list of logical address segments, a list of dependencies, and scheduling context information can be prepared.

[0084] Step 5: Scan for ready operator tasks and check execution conditions.

[0085] The host-side control plane scans the operator tasks to be executed according to the fusion operator graph through the fusion scheduler, and checks whether their dependencies are satisfied, whether the corresponding logical address ranges conflict, whether they are accessible and whether the versions match, and whether the corresponding devices are executable, etc.

[0086] Optionally, the fusion scheduler can maintain a ready queue or ready table and continuously list operator tasks to be executed according to the topological order of the fusion operator graph. When the predecessor dependency of an operator task to be executed is satisfied, there are no conflicts in the logical address range, and the target device resources are available, the operator task to be executed can be marked as a task that can be deployed.

[0087] Step 6: Issue the operator tasks for the GPU and / or the operator tasks for the FPGA.

[0088] The fusion scheduler distributes tasks that meet the conditions, enabling the GPU and FPGA to execute operator tasks in parallel or interleaved around the shared memory area.

[0089] Optionally, the fusion scheduler can write GPU operator tasks into the GPU command queue and FPGA operator tasks into the FPGA task queue respectively, and execute parallel delivery of GPU operator tasks and FPGA operator tasks that are independent of each other and whose logical address segments do not conflict.

[0090] Step 7: Write intermediate dependency data into the logical address segment and commit the shared segment commit record.

[0091] After the operator task of the producer device completes the writing of the intermediate dependency data of the corresponding target logical address segment, it submits the corresponding shared segment commit record to the hardware dependency barrier and updates the status of the corresponding target logical address segment.

[0092] Step 8: Perform dependency checks and release subsequent consumer device operator tasks.

[0093] The hardware dependency barrier releases subsequent operator tasks that have met the conditions based on the logical address segment status and dependency relationships, allowing the fusion scheduler to continue issuing them.

[0094] Optionally, when the hardware-dependent barrier confirms that the target logical address range meets the release conditions, it can also trigger a CXL.cache consistency maintenance action for the corresponding consumer device to complete the invalidation, update notification, or visibility confirmation of the relevant cache lines before the release of subsequent tasks.

[0095] Step 9: Repeat the process until the fusion operator graph is complete.

[0096] Repeat steps five through eight until all operator tasks in the current fusion operator graph have been completed.

[0097] It should be noted that, Figure 5 The straight arrows in the diagram represent the main process flow path; the dashed arrows in the diagram represent loop return paths or processing paths that are checked again after waiting.

[0098] In an optional embodiment of the present invention, the first regular operator task, the first irregular operator task, and the second regular operator task are batch operator tasks in the first batch. While the FPGA writes the second intermediate dependency tensor into the second logical address segment of the shared memory region, the method may further include: the host-side control plane issuing a third regular operator task to the GPU; wherein the third regular operator task is a batch operator task in the second batch; the GPU executes the third regular operator task to obtain a third intermediate dependency tensor, writes the third intermediate dependency tensor into the third logical address segment of the shared memory region, and sends a third shared segment commit record of the third intermediate dependency tensor to the hardware dependency barrier. The hardware dependency barrier updates the state of the third logical address segment based on the third shared segment commit record. When the host-side control plane determines that the states of the third logical address segment and the FPGA meet the scheduling conditions, it issues a second irregular operator task to the FPGA. The second irregular operator task is a second batch of batch operator tasks. The FPGA reads the third target dependency data from the third logical address segment, executes the second irregular operator task based on the third target dependency data, obtains a fourth intermediate dependency tensor, writes the fourth intermediate dependency tensor into the fourth logical address segment of the shared memory region, and sends the fourth shared segment commit record of the fourth intermediate dependency tensor to the hardware dependency barrier.

[0099] In this context, a batch operator task refers to a task in batch computing scenarios where a series of data processing logics are encapsulated into operators and combined into jobs according to sequence or dependencies for centralized execution of the data at once. A third regular operator task can be a batch operator task that requires GPU execution; this batch operator task can be a regular operator task. A third intermediate dependency tensor can be an intermediate dependency tensor corresponding to the third regular operator task, generated after the GPU executes the third regular operator task. A third logical address segment can be a logical address segment used to store the third intermediate dependency tensor of the third regular operator task. A third shared segment commit record can be a shared segment commit record of the third intermediate dependency tensor. A second irregular operator task can be a batch operator task of irregular operators executed by the FPGA. The second irregular operator task requires the third intermediate dependency tensor of the third regular operator task for execution. The third target dependency data can be part or all of the data in the third intermediate dependency tensor. A fourth intermediate dependency tensor can be an intermediate dependency tensor corresponding to the second irregular operator task, generated after the FPGA executes the second irregular operator task. The fourth logical address segment can be a logical address segment used to store the fourth intermediate dependency tensor of the second irregular operator task. The fourth shared segment commit record can be a shared segment commit record of the fourth intermediate dependency tensor.

[0100] To improve equipment utilization, the computing system for fusion operators provided in this embodiment of the invention can also perform interleaved scheduling of operator tasks from different batches. While one batch of GPU operator tasks is being executed, another batch of FPGA operator tasks that depend on already satisfied conditions can be executed in parallel; or, when one batch of FPGA operator tasks is waiting for logical address segments to become ready, the fusion scheduler can prioritize distributing operator tasks from other batches of GPUs that have already met the conditions. This batch scheduling method can further improve the overall workload of both GPUs and FPGAs.

[0101] Figure 6 This is a flowchart illustrating the interleaved scheduling of multiple batches of operator tasks in a computational system for fusion operators provided in Embodiment 2 of the present invention. In a specific example, such as... Figure 6 As shown, Figure 6 The upper middle section illustrates the shared access and dependency release relationships between the GPU, shared memory region, FPGA, and hardware dependency fence (HDF) / scheduler within the same CXL consistency domain; Figure 6 The lower half shows the staggered scheduling timing of batches B1 and B2 on the GPU and FPGA channels. Batch B1 can be understood as the first batch of batch operator tasks, and batch B2 can be understood as the second batch of batch operator tasks.

[0102] Combined with appendix Figure 6 The specific process steps may include: Step 1: Batch B1 first executes the first rule operator task of Batch B1 on the GPU channel, completes the front-end dense computation of Batch B1, and writes the first intermediate dependency tensor of the first rule operator task into the first logical address segment S1 in the shared memory area.

[0103] Step 2: After completing the write operation of the first intermediate dependency tensor in the first logical address segment S1, the first batch operator task of the producer equipment in batch B1 submits the first shared segment commit record. The hardware dependency barrier updates the state of the first logical address segment S1 according to the first shared segment commit record.

[0104] Step 3: After the hardware dependency barrier detects that the corresponding state of the first logical address segment S1 and the FPGA meets the release condition, it releases the first irregular operator task of the FPGA in batch B1 according to the state of the first logical address segment S1, the FPGA state and the task dependency relationship, and the fusion scheduler sends the first irregular operator task to the FPGA channel for execution.

[0105] Step 4: When the FPGA executes the first irregular operator task of batch B1, it reads the first target dependency data in the first logical address segment S1, performs routing judgment, filtering, mask generation or other irregular processing, and generates a second intermediate dependency tensor based on the processing result. The second intermediate dependency tensor is then written back to the second logical address segment S2 in the shared memory area.

[0106] Step 5: While batch B1 is waiting for the completion of the second intermediate dependency tensor in the second logical address segment S2, a reusable execution window appears in the GPU channel. The fusion scheduler selects ready tasks that have met the conditions from other batches and continues to issue them, so that the third rule operator task of the GPU in batch B2 is inserted into the waiting interval for execution, thereby filling the GPU idle time.

[0107] Step Six: After the third rule operator task of the GPU in batch B2 is executed, the GPU can write the third intermediate dependency tensor of the third rule operator task into the shared memory area corresponding to the third logical address segment S1' of batch B2, and send the third shared segment commit record of the third intermediate dependency tensor to the hardware dependency barrier, so that the hardware dependency barrier updates the state of the third logical address segment according to the third shared segment commit record. After determining that the subsequent operator task meets the conditions, the hardware dependency barrier enters the subsequent dependency release process. Accordingly, the second irregular operator task of the FPGA in batch B2 can be released for execution after its dependencies are satisfied.

[0108] Step 7: After the first irregular operator task of the FPGA in batch B1 is completed and the second intermediate dependency tensor of the first irregular operator task is written to the second logical address segment S2, the FPGA resubmits the second shared segment commit record. The hardware dependency barrier updates the state of the second logical address segment according to the second shared segment commit record. After the hardware dependency barrier detects that the state of the second logical address segment S2 and the GPU meets the release conditions, it releases the second regular operator task of the GPU in batch B1, allowing the GPU channel to resume execution of subsequent calculations in batch B1.

[0109] Step 8: When the GPU executes the second rule operator task of batch B1, it reads the second target dependency data in the second logical address segment S2 and continues to execute subsequent Attention-related calculations, Gather, output transformation or other subsequent stage calculations to complete the subsequent processing chain of batch B1.

[0110] Step Nine: Following the same mechanism, the hardware dependency barrier and the fusion scheduler can continue to dynamically select ready tasks from different batch operator tasks based on the status of each logical address segment, and distribute them to the GPU channel or FPGA channel for execution, thus forming a continuous cross-batch interleaved scheduling. For example, after the second irregular operator task of the FPGA in batch B2 is released for execution after its dependency is satisfied, the FPGA reads the third target dependency data from the third logical address segment S1', executes the second irregular operator task according to the third target dependency data, obtains the fourth intermediate dependency tensor, and writes the fourth intermediate dependency tensor into the shared memory area corresponding to the fourth logical address segment S2' of batch B2, etc. At the same time, it sends the fourth shared segment commit record of the fourth intermediate dependency tensor to the hardware dependency barrier. This process continues, forming a continuous cross-batch interleaved scheduling.

[0111] It should be noted that, Figure 6 Solid lines in the code represent the main execution path or data path; dashed lines represent the commit path, status feedback path, or task release path for shared segment commit records. For example... Figure 6 As shown, under the dependency release mechanism of the shared storage area based on the CXL consistency domain and the hardware dependency barrier, the fusion scheduler can dynamically switch ready tasks between different batch operator tasks according to the logical address segment status, which can reduce device idle time caused by waiting and improve overall throughput.

[0112] Figure 7 This is a schematic diagram of a logical address segment state management and SCR-HDF dependency release mechanism provided by the present invention. In a specific example, such as Figure 7 As shown, after the shared storage area is divided into multiple logical address segments, the producer device can perform writes, status updates, and SCR commits around the target logical address segment, and the HDF can perform dependency checks and release subsequent consumer tasks based on information such as the logical address segment status, batch number, and version number.

[0113] Specifically, such as Figure 7 As shown, after completing its producer task (i.e., the operator task executed by the producer device), the producer device writes the corresponding intermediate dependency tensor to the target logical address segment in the shared memory region. Optionally, the producer task can be a GPU front-end dense computation task or an FPGA front-end irregular processing task. The logical address segment pool in the shared memory region can hold multiple reusable and shareable logical address segments, such as S1, S2, S3, and S4. Each logical address segment can be uniformly organized and its status tracked by combining segment identifier, batch number, and version number.

[0114] Correspondingly, such as Figure 7As shown, after completing its consumer task (i.e., the operator task executed by the consumer device), the consumer device reads the intermediate dependency tensor after the target logical address segment meets the readable condition, and continues to execute subsequent stage calculations. Optionally, the consumer task can be a GPU subsequent calculation task or an FPGA subsequent processing task. The overall logic of the Hardware Dependency Fence (HDF) can be to receive SCR information and release the corresponding consumer task according to the logical address segment status and related dependency conditions. For example, the logical address segment status can include, but is not limited to, idle state, writing state, ready state, released state, consumed state, and reusable state. Among them, the idle state is used to indicate that the corresponding logical address segment is in an allocable or reusable state; the writing state is used to indicate that the producer task has occupied the corresponding logical address segment and is writing data; the ready state is used to indicate that the corresponding logical address segment has completed writing and meets the readable condition; the released state is used to indicate that the HDF has released the corresponding consumer task according to the dependency detection result; the consumed state is used to indicate that the consumer task has read the result in the target logical address segment; and the reusable state is used to indicate that the logical address segment can re-enter the idle pool or be remapped and used after consumption. Figure 7 The status field description examples show the status record items that can be associated with a logical address segment, such as segment identifier, batch number, version number, and status flags such as write complete, readable, released, consumed, and reusable.

[0115] like Figure 7 As shown, the SCR submission unit is used to submit the SCR after the producer task completes the writing of the shared logical address segment. The dependency determination unit in HDF can perform dependency determination on the received SCR. Optionally, the SCR fields may include, but are not limited to, logical address segment identifier, producer task identifier, batch number, version number, and status bits. The dependency detection conditions of HDF may include, but are not limited to, whether the logical address segments required by the consumer are all ready, whether the batch number and version number match, and whether the target device resources are available. If all dependency detection conditions are met, HDF can release the GPU successor task and / or FPGA successor task. The fusion scheduler, in conjunction with HDF's release operation of the successor task, continues to issue the corresponding operator task so that the corresponding operator task can be issued to the target device for execution after the operator task is released.

[0116] like Figure 7As shown, the computational process of the fusion operator in the computing system can be summarized as follows: The producer task first writes the intermediate result into the target logical address segment in the logical address segment pool of the shared storage area, causing the logical address segment to move from an idle state to a writing state. After the intermediate result is written, the logical address segment enters a ready state, and at the same time, the SCR submission unit submits an SCR containing relevant segment information to the HDF. The HDF performs dependency determination based on the SCR field and dependency detection conditions. When the determination meets the release conditions, the HDF marks the corresponding consumer task as released and generates a release result. Subsequently, the fusion scheduler continues to distribute the corresponding task to the target device for execution based on the release result. The consumer task reads the target logical address segment data and completes subsequent processing, causing the logical address segment to sequentially enter the consumed state and the reusable state.

[0117] It should be noted that, Figure 7 Solid lines in the code represent shared logical address segments, state transition paths, or task continuation paths; dashed lines represent SCR submission paths, state detection paths, or dependency release paths. Figure 7 To further clarify: the embodiments of the present invention do not coordinate cross-device execution based on coarse-grained software synchronization, but rather achieve fine-grained task release and heterogeneous collaborative execution for shared storage areas under the support of CXL consistency domains through shared logical address segment states, SCR commit information, and HDF dependency detection mechanisms.

[0118] The above-described fusion operator computation method involves the GPU executing a first rule operator task to obtain a first intermediate dependency tensor. This first intermediate dependency tensor is then written to the first logical address segment of the shared memory region, and a first shared segment commit record of the first intermediate dependency tensor is sent to the hardware dependency barrier. The hardware dependency barrier updates the state of the first logical address segment based on the first shared segment commit record. When the host-side control plane determines that the states of the first logical address segment and the FPGA satisfy the scheduling conditions, it issues a first irregular operator task to the FPGA. The FPGA reads the first target dependency data from the first logical address segment, executes the first irregular operator task based on the first target dependency data, obtains a second intermediate dependency tensor, writes the second intermediate dependency tensor to the second logical address segment of the shared memory region, and sends a second shared segment commit record of the second intermediate dependency tensor to the hardware dependency barrier. The hardware dependency barrier updates the state of the second logical address segment based on the second shared segment commit record. When the host-side control plane determines that the states of the second logical address segment and the GPU satisfy the scheduling conditions, it issues a second rule operator task to the GPU. The GPU reads the second target dependency data from the second logical address segment and executes the second rule operator task based on the second target dependency data. As can be seen, the above technical solution realizes zero-copy access to the shared storage area of ​​the CXL consistency domain, hardware-level parallel fusion of dense-irregular operators, and fine-grained consistency control with hardware-dependent fence driving, thereby helping to reduce the transfer of intermediate results across devices, reduce synchronization waiting overhead, and improve the computational efficiency of fusion operators.

[0119] It should be noted that the relevant information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for display, data used for analysis, etc.) involved in this disclosure are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data comply with the relevant laws, regulations and standards of the relevant regions, and do not violate public order and good morals.

[0120] It should be noted that any arrangement or combination of the technical features in the above embodiments also falls within the protection scope of this invention.

[0121] Example 3 Figure 8A schematic diagram of an electronic device 10, which can be used to implement embodiments of the present invention, is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0122] like Figure 8 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded into the RAM 13 from storage unit 18. The RAM 13 can also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0123] Optionally, the electronic device 10 may further include a computing system for the fusion operator described in any embodiment of the present invention. Figure 8 (Not shown).

[0124] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0125] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as the computational methods of fusion operators.

[0126] Optionally, the computation method of the fusion operator can be applied to the computation system of the fusion operator described in any embodiment of the present invention, including: the GPU executes a first regular operator task to obtain a first intermediate dependency tensor, writes the first intermediate dependency tensor into a first logical address segment of the shared storage region, and sends a first shared segment commit record of the first intermediate dependency tensor to the hardware dependency barrier; the hardware dependency barrier updates the state of the first logical address segment according to the first shared segment commit record; the host-side control plane, when determining that the state of the first logical address segment and the FPGA meets the scheduling conditions, issues a first irregular operator task to the FPGA; the FPGA reads a first target dependency from the first logical address segment. Based on the first target dependency data, the first irregular operator task is executed to obtain a second intermediate dependency tensor. The second intermediate dependency tensor is written into the second logical address segment of the shared storage region, and a second shared segment commit record of the second intermediate dependency tensor is sent to the hardware dependency barrier. The hardware dependency barrier updates the state of the second logical address segment based on the second shared segment commit record. When the host-side control plane determines that the state of the second logical address segment and the GPU meet the scheduling conditions, it issues a second rule operator task to the GPU. The GPU reads the second target dependency data from the second logical address segment and executes the second rule operator task based on the second target dependency data.

[0127] In some embodiments, the computation method of the fusion operator can be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the computation method of the fusion operator described above can be performed. Alternatively, in other embodiments, processor 11 can be configured to execute the computation method of the fusion operator by any other suitable means (e.g., by means of firmware).

[0128] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0129] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0130] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0131] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0132] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0133] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0134] This invention also discloses a computer program product, which includes a computer program that, when executed by a processor, implements the calculation method of the fusion operator provided in any embodiment of this invention. This program product and the calculation method of the fusion operator disclosed in the various embodiments of this invention belong to the same inventive concept, and therefore will not be described in detail here.

[0135] It should be understood that the various forms of processes shown above can be used to reorder, add, or delete steps. For example, the steps described in the embodiments of the present invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in the embodiments of the present invention can be achieved, and this is not limited herein.

[0136] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A computational system for fusion operators, characterized in that, Configured on heterogeneous computing nodes, including the host-side control plane, graphics processing unit (GPU), field-programmable gate array (FPGA), shared memory region, and hardware-dependent fence; wherein: The host-side control plane is communicatively connected to the GPU, the FPGA, the shared storage area, and the hardware dependency barrier. It is used to identify regular and irregular operators in the model to be executed, generate a fusion operator graph corresponding to the regular and irregular operators, determine the current operator task to be assigned based on the fusion operator graph and the task assignment condition detection result, and assign the current operator task to the corresponding target operator execution device according to the operator type of the current operator task. The target operator execution device includes the GPU and / or the FPGA; the task assignment condition detection result includes the state detection result of the target device and the target intermediate dependency tensor. The GPU is used to execute the operator tasks corresponding to the rule operators; The FPGA is used to execute the operator tasks corresponding to the irregular operator; The shared storage area is a shared intermediate data work area in the CXL consistency domain, configured for joint access by the GPU and the FPGA, used to temporarily store, exchange and reuse intermediate dependency tensors generated and / or used by the GPU and the FPGA during the execution of operator tasks; The hardware dependency barrier is used to detect the stored data in the shared storage area to determine the state detection result of the target intermediate dependency tensor.

2. The system according to claim 1, characterized in that, The host-side control plane is also used for: The detection host, the GPU, the FPGA, and interconnect resources; Based on the detection results, the storage address range, device access relationships, and device dependencies of the shared storage area are determined.

3. The system according to claim 2, characterized in that, The host-side control plane is also used for: Within the storage address range of the shared storage region, address planning is performed on the shared storage region to generate multiple logical address segments for the shared storage region; The mapping relationship between each logical address segment and the GPU and FPGA is established by calculating the Fast Link CXL memory protocol.

4. The system according to any one of claims 1-3, characterized in that: After completing the operator task, the producer device writes an intermediate dependency tensor to the target logical address segment of the shared memory region; wherein, the producer device includes the GPU and / or the FPGA; After determining that the intermediate dependency tensor is written to the target logical address segment, the producer device submits a shared segment submission record to the hardware dependency barrier. The hardware dependency barrier is also used to update the state of the target logical address segment and the consistency maintenance actions of the CXL core transaction layer protocol of the target logical address segment based on the shared segment commit record. The hardware dependency barrier is also used to trigger a consistency maintenance action of the CXL core transaction layer protocol for consumer devices when the state detection result of the target intermediate dependency tensor meets the task delivery conditions.

5. The system according to any one of claims 1-3, characterized in that, The host-side control plane is also used for: If it is determined that the current operator task to be assigned is a dependent operator task, the current operator task to be assigned is written into the target queue of the target device according to the operator type of the current operator task to be assigned. If it is determined that the currently assigned operator task is an independent operator task, the independent operator task is sent to the target device in parallel according to the operator type of the independent operator task.

6. A method for calculating a fusion operator, characterized in that, A computing system applied to the fusion operator according to any one of claims 1-5, comprising: The GPU executes the first rule operator task to obtain the first intermediate dependency tensor, writes the first intermediate dependency tensor into the first logical address segment of the shared storage area, and sends the first shared segment commit record of the first intermediate dependency tensor to the hardware dependency fence. The hardware dependency barrier updates the state of the first logical address segment based on the first shared segment commit record; When the host-side control plane determines that the state of the first logical address segment and the FPGA meets the scheduling conditions, it issues the first irregular operator task to the FPGA. The FPGA reads the first target dependency data from the first logical address segment, executes the first irregular operator task according to the first target dependency data, obtains the second intermediate dependency tensor, writes the second intermediate dependency tensor into the second logical address segment of the shared storage area, and sends the second shared segment commit record of the second intermediate dependency tensor to the hardware dependency barrier. The hardware dependency barrier updates the state of the second logical address segment based on the second shared segment commit record; When the host-side control plane determines that the second logical address segment and the state of the GPU meet the scheduling conditions, it issues a second rule operator task to the GPU. The GPU reads the second target dependency data from the second logical address segment and executes the second rule operator task based on the second target dependency data.

7. The method according to claim 6, characterized in that, The first regular operator task, the first irregular operator task, and the second regular operator task constitute the first batch of batch operator tasks. Simultaneously with the FPGA writing the second intermediate dependency tensor into the second logical address segment of the shared memory region, the method further includes: The host-side control plane issues a third rule operator task to the GPU; wherein, the third rule operator task is a batch operator task of the second batch; The GPU executes the third rule operator task to obtain the third intermediate dependency tensor, writes the third intermediate dependency tensor into the third logical address segment of the shared memory region, and sends the third shared segment commit record of the third intermediate dependency tensor to the hardware dependency barrier. The hardware dependency barrier updates the state of the third logical address segment based on the third shared segment commit record; When the host-side control plane determines that the state of the third logical address segment and the FPGA meets the scheduling conditions, it issues a second irregular operator task to the FPGA; wherein, the second irregular operator task is a second batch of batch operator tasks; The FPGA reads the third target dependency data from the third logical address segment, executes the second irregular operator task based on the third target dependency data to obtain the fourth intermediate dependency tensor, writes the fourth intermediate dependency tensor into the fourth logical address segment of the shared memory region, and sends the fourth shared segment commit record of the fourth intermediate dependency tensor to the hardware dependency barrier.

8. An electronic device, characterized in that, The electronic device includes a computing system that incorporates the fusion operator according to any one of claims 1-5: The computation system of the fusion operator is capable of executing the computation method of the fusion operator as described in claim 6 or 7.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the computation method of the fusion operator as described in claim 6 or 7.

10. A computer program product, characterized in that, It includes a computer program / instruction, wherein the computer program / instruction, when executed by a processor, implements the computation method of the fusion operator as described in claim 6 or 7.

Images