Data flow management method for computing tasks and computing execution hardware

Abstract

The application provides a data flow management and control method of a computing task and computing execution hardware. The method comprises the following steps: a first processing node in a computing core array reads and parses a computing package from a persistent cache resident area, and obtains resource binding information; the computing task comprises a general computing task and a graph computing task; the first processing node locates a logical address in the persistent cache resident area according to the resource binding information; the persistent cache resident area stores data generated by a second processing node in the computing core array and original data migrated from a storage unit; the first processing node reads the data based on the logical address, and executes the computing through a thread register; and the first processing node writes the computing result back to the persistent cache resident area. The application reduces the waste of on-chip storage resources and improves the cache utilization rate.

Description

Technical Field

[0001] This application relates to the field of computer technology, and more specifically, to a data flow management method and computing execution hardware for computing tasks. Background Technology

[0002] In traditional graphics processing unit (GPU) architectures, the graphics rendering pipeline relies on a large number of dedicated intermediate buffers (such as depth buffers, color buffers, primitive buffers, etc.) to temporarily store intermediate computation results at each stage. These dedicated buffers are usually customized for specific graphics tasks, have fixed hardware resources, and only serve the graphics rendering pipeline. They cannot be reused for general computing tasks (such as AI inference, physics simulation, etc.), resulting in low utilization of valuable on-chip storage resources and wasted silicon area.

[0003] In addition, in traditional architectures, intermediate data is frequently moved back and forth between global video memory and dedicated buffers. Due to the limited bandwidth and high access latency of global video memory, the system bandwidth pressure is high and the energy efficiency is low. Summary of the Invention

[0004] The purpose of this application is to provide a data flow management method and computing execution hardware for computing tasks, so as to reduce the waste of on-chip storage resources and improve bandwidth-energy efficiency ratio.

[0005] In a first aspect, embodiments of this application provide a data flow control method for computing tasks, applied to computing execution hardware. The computing execution hardware includes a processing unit and a storage unit. The processing unit includes a computing core array; the storage unit includes global memory and a persistent cache resident area. The method includes: The first processing node in the computing core array reads and parses the computing packets from the persistent cache to obtain resource binding information. The computing packets are obtained by decomposing the computing tasks. The computing tasks include graphics computing tasks and general computing tasks. The resource binding information is used to indicate the data resources that the current computing packet needs to access during execution and their corresponding relationships. The first processing node locates the logical address in the persistent cache resident area based on the resource binding information; the persistent cache resident area stores the data generated by the second processing node in the computing core array and the original data migrated from the storage unit; The first processing node reads data based on the logical address and performs calculations through the thread register; The first processing node writes the calculation results back to the persistent cache.

[0006] This application's embodiments introduce a persistent cache resident area to replace the traditional dedicated buffer mechanism, enabling dynamic sharing and flexible reuse of graphics computing tasks and general computing tasks on a unified high-speed cache resource. This effectively reduces the waste of on-chip storage resources and improves cache utilization. By parsing the resource binding information in the computing packet, the first processing node is precisely guided to locate and read the required data, enhancing the controllability and efficiency of data access and improving bandwidth-energy efficiency.

[0007] In one possible implementation of the first aspect, the method further includes: After obtaining the computation packet, acquire the creation timestamp and expiration timestamp of the computation packet allocation; The computed packet containing the creation timestamp and expiration timestamp is stored in the persistent cache resident area.

[0008] This application embodiment achieves fine-grained management of the cached data lifecycle by assigning creation timestamps and expiration timestamps to computation packets and storing computation packets with timestamp information in a persistent cache resident area. With the help of the timestamp mechanism, the system can accurately track the validity period and dependency status of each computation packet, thereby promptly reclaiming the cache resources it occupies when all consumers have completed reading or when the preset expiration time has been reached.

[0009] In one possible implementation of the first aspect, the method further includes: Monitor the consumption status of computation packets; the consumption status is used to indicate whether the first processing node that needs to consume the computation packet has completed the reading operation; If, based on the consumption status, it is determined that all first processing nodes that need to consume computation packets have completed the operation of reading computation packets, or the current time has reached the expiration timestamp, then the computation packet is marked as invalid, and the resources in the persistent cache area occupied by the computation packet are restored to an available state.

[0010] This application's embodiments introduce a packet lifecycle management mechanism to monitor the consumption status of computation packets in real time. By combining creation timestamps and expiration timestamps for dual determination, and ensuring that all relevant consumers have completed data reading or have exceeded the preset lifecycle, the computation packets are promptly marked as invalid and their occupied persistent cache residency resources are reclaimed. This achieves automated and refined management of cache resources, effectively avoiding memory leaks and resource waste, and improving cache utilization and overall system operating efficiency.

[0011] In one possible implementation of the first aspect, the computation result is written back to the persistent cache resident area, including: Obtain the packet dependency graph; the packet dependency graph includes multiple nodes and edges; nodes are used to represent computation packets, and edges are used to represent the dependencies between computation packets; Based on the packet dependency graph, the third processing unit in the computing core array that has a dependency relationship with the first processing node corresponding to the calculation result is obtained; The calculation results are written to the shared memory of the third processing unit; where the shared memory is part of the persistent cache resident area.

[0012] This application embodiment obtains a packet dependency graph to clarify the dependencies between various computation packets, and writes the computation results into the shared memory of the third processing unit with the dependency relationship based on the dependency graph. This allows subsequent consumers to access the required data nearby and in a timely manner. Since the shared memory is part of the persistent cache resident area, it avoids the high bandwidth consumption and access latency caused by writing intermediate results back to global memory, significantly improving data locality and cache utilization.

[0013] In one possible implementation of the first aspect, the packet dependency graph is constructed as follows: Obtain the preceding and succeeding packets for each computation packet; Construct a packet dependency graph based on the predecessor and successor packets of each computation packet.

[0014] This application embodiment obtains the predecessor and successor packets of each computation packet and constructs a packet dependency graph accordingly, which can accurately depict the data dependencies and timing constraints between computation tasks, thereby achieving fine-grained scheduling and management of the computation process.

[0015] In one possible implementation of the first aspect, the computational execution hardware further includes an asynchronous memory transfer accelerator, and the method further includes: The asynchronous memory transport accelerator receives asynchronous multicast load requests and moves the original data from global memory to the persistent cache resident area within the asynchronous multicast load request.

[0016] This application's embodiments introduce an asynchronous memory transport accelerator to initiate asynchronous multicast loading requests, which efficiently transports the original data from global memory to the persistent cache resident area, realizing the parallelization of data prefetching and computation execution, effectively hiding memory access latency; at the same time, it supports a multicast mechanism, which can concurrently load the same data to multiple target locations, significantly reducing duplicate data transmission, reducing global memory bandwidth pressure, improving data supply efficiency, and thus enhancing the overall system's throughput and energy efficiency.

[0017] In one possible implementation of the first aspect, the method further includes: Monitor the types of tasks currently pending execution by the computing hardware; task types include graphics processing tasks and general computing tasks. The persistent cache residency area is functionally divided according to task type.

[0018] This application embodiment dynamically senses the current load type to achieve on-demand allocation and flexible partitioning of the persistent cache resident area. When the system is running graphics processing tasks, larger cache space is prioritized for intermediate graphics data such as geometry and fragments; when running general computing tasks (such as AI inference and scientific computing), resources are allocated to data storage such as matrices and vectors. This mechanism improves the utilization rate of cache resources, avoids the resource waste caused by fixed allocation of dedicated buffers in traditional architectures, enhances the hardware's adaptability to diverse workloads, and effectively supports the efficient coexistence and dynamic sharing of graphics and general computing on the same cache resource.

[0019] In one possible implementation of the first aspect, the method further includes: After the computation task is completed, the final result is written to global memory.

[0020] This application embodiment avoids frequent reading and writing of intermediate data by uniformly writing the final result back to global memory after the task is completed, reduces the number of accesses to the high-latency, high-power global memory, and effectively reduces system bandwidth pressure and energy consumption; at the same time, it ensures that only valid results that have undergone complete processing are persistently stored, improves data consistency and system reliability, and is conducive to achieving efficient memory management and result output control.

[0021] Secondly, embodiments of this application provide computing execution hardware, including a processing unit and a storage unit. The processing unit includes a computing core array, and the storage unit includes global memory and a persistent cache resident area. The global memory stores computer program instructions, which are read and executed by the processing unit to perform the method of the first aspect.

[0022] Thirdly, embodiments of this application provide an electronic device, including: computing execution hardware as described in the second aspect.

[0023] Fourthly, embodiments of this application provide a non-transitory computer-readable storage medium, comprising: A non-transitory computer-readable storage medium stores computer instructions that cause the computer to perform the methods in the various possible implementations of the first aspect.

[0024] Fifthly, embodiments of this application provide a computer program product, including computer program instructions, which, when read and executed by a processor, perform the methods in various possible implementations of the first aspect.

[0025] Other features and advantages of this application will be set forth in the following description and will be apparent in part from the description or may be learned by practicing embodiments of this application. The objectives and other advantages of this application may be realized and obtained by means of the structures particularly pointed out in the written description and the accompanying drawings. Attached Figure Description

[0026] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0027] Figure 1 This application provides a schematic diagram of a computing execution hardware structure as an embodiment. Figure 2 A schematic flowchart of a data flow control method for a computing task provided in an embodiment of this application; Figure 3 A schematic diagram of a computational packet lifecycle management process is provided for embodiments of this application; Figure 4 This is another data flow architecture diagram provided for an embodiment of this application. Detailed Implementation

[0028] The embodiments of the technical solution of this application will now be described in detail with reference to the accompanying drawings. These embodiments are only used to more clearly illustrate the technical solution of this application and are therefore merely examples, and should not be used to limit the scope of protection of this application.

[0029] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs; the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit this application; the terms “comprising” and “having”, and any variations thereof, in the specification and the foregoing description of the drawings are intended to cover non-exclusive inclusion.

[0030] In the description of the embodiments of this application, technical terms such as "first" and "second" are used only to distinguish different objects and should not be construed as indicating or implying relative importance or implicitly specifying the number, specific order, or primary and secondary relationship of the indicated technical features. In the description of the embodiments of this application, "multiple" means two or more, unless otherwise explicitly defined.

[0031] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0032] In the description of the embodiments in this application, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this document generally indicates that the preceding and following related objects have an "or" relationship.

[0033] In the description of the embodiments of this application, the term "multiple" refers to two or more (including two), similarly, "multiple sets" refers to two or more (including two sets), and "multiple pieces" refers to two or more (including two pieces).

[0034] In the description of the embodiments of this application, unless otherwise expressly specified and limited, technical terms such as "installation," "connection," "joining," and "fixing" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components. For those skilled in the art, the specific meaning of the above terms in the embodiments of this application can be understood according to the specific circumstances.

[0035] Traditional GPU graphics rendering pipelines rely on dedicated intermediate caches at each stage, such as depth buffers, color buffers, and primitive buffers. These caches are entirely tailored for graphics tasks, fixing hardware resources and leaving no room for general-purpose computing such as AI inference and physics simulation. As a result, a large amount of on-chip storage space is idle, wasting valuable silicon area. Nowadays, graphics processing and AI computing often run simultaneously on the same chip, and this hard-isolated buffer structure is clearly inadequate for the need for flexible resource sharing.

[0036] Looking at performance, intermediate data is frequently moved back and forth between global video memory and dedicated buffers. Video memory bandwidth is inherently limited, and access latency is high, which not only exacerbates bandwidth pressure but also reduces energy efficiency. From data generation to consumption, the lack of effective locality management makes it difficult to utilize the high-speed shared storage between adjacent computing units, naturally resulting in low access efficiency. Existing caching strategies are largely passive, failing to proactively determine whether data should be retained or released based on the task's lifecycle, further exacerbating resource contention and redundant read / write problems.

[0037] Based on the above problems, this application provides a data flow management method for computing tasks. Specifically, it establishes a unified persistent cache area, allowing different types of workloads to reuse cache resources. At the same time, it strengthens data locality and improves access efficiency by combining fine-grained lifecycle management and dependency sorting, thereby reducing the bandwidth burden of global memory and ultimately achieving both high performance and high energy efficiency in heterogeneous computing scenarios.

[0038] The following is a detailed description of a data flow control method for computing tasks provided by an embodiment of this application.

[0039] It should be noted that this method is applied to computing execution hardware, which is the specific execution hardware of a computing task processing system. Figure 1 This is a schematic diagram of a computing execution hardware structure provided in an embodiment of this application. The computing execution hardware includes a processing unit 101 and a storage unit 102. The processing unit 101 integrates a large-scale computing core array 1011, each computing core array having an independent thread register file, supporting fine-grained parallel execution. The processor can be an integrated circuit chip with signal processing capabilities. The aforementioned processing unit 101 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute various methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor, etc.

[0040] Storage unit 102 includes global memory 1021 and persistent cache resident area 1022. Global memory 1021 has a large capacity but high access latency and limited bandwidth; it is used to store raw input data, long-term resident model parameters, and final output results. Persistent cache resident area 1022 is located in on-chip high-speed storage, serving as a unified data transfer and reuse pool. This area does not distinguish between graphics-specific buffers and general-purpose computing temporary data; instead, it manages them uniformly using the same logical address space, supporting dynamic allocation, lifecycle management tracking, and cross-task data sharing. Each processing unit (such as a thread block or workgroup) in the computing core array can produce, consume, and transfer data through this persistent cache resident area 1022, thereby breaking down the resource barriers between traditional graphics and general-purpose computing.

[0041] The storage unit 102 may include, but is not limited to, random access memory (RAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), etc.

[0042] Figure 2 This is a flowchart illustrating a data flow control method for a computing task provided in an embodiment of this application. The method includes: Step 201: The first processing node in the computing core array reads and parses the computing packets from the persistent cache resident area to obtain resource binding information.

[0043] The computation packet is formed by a high-level task scheduler decomposing the original computation task (including graphics pipeline stages such as vertex shading and pixel shading, as well as AI operators such as matrix multiplication and convolution) into several independent or semi-independent subtasks based on data dependencies and parallel granularity. Each subtask is encapsulated as a computation packet. The packet header contains metadata to guide its execution on the hardware. Computation tasks include graphics computation tasks and general-purpose computation tasks.

[0044] Resource binding information indicates the data resources and their corresponding relationships that the current computation packet needs to access during execution. This information, recorded in the form of a descriptor table or handle, includes data resources such as textures, buffers, tensors, and constants, as well as their logical address mappings, access permissions (read / write), expected data formats, and dependent version numbers within the persistent cache. The first processing node uses this information to locate the corresponding data and complete subsequent reading and computation. Therefore, resource binding information decouples computational logic from physical storage location, providing a foundation for subsequent dynamic data layout.

[0045] Step 202: The first processing node locates the logical address in the persistent cache resident area based on the resource binding information.

[0046] The first processing node queries the address translation table in the persistent cache resident area based on the logical address identifier in the resource binding information to obtain the actual physical offset of each data resource in the on-chip high-speed storage. If the required data is not currently resident (e.g., for the first access or has been evicted), a page fault / data fault exception is triggered, and the hardware management unit asynchronously migrates the data from global memory or lower-level storage to the resident area and updates the address mapping.

[0047] The persistent cache residency area can be a persistent secondary cache or shared content within a thread cluster. It stores data generated by the second processing node in the computational core array, such as primitive attributes output from the previous stage in the graphics pipeline, or intermediate feature maps from general computation. Additionally, the data stored in the persistent cache residency area can also be raw data migrated from other memory locations, such as initial inputs prefetched or loaded on demand from global memory, model weights, and fixed constants.

[0048] Understandably, before reading data, the first processing node will also check the dependencies attached to the resource binding information, such as waiting for a certain event to complete, semaphores, etc., to ensure data version consistency and reduce read / write conflicts.

[0049] Step 203: The first processing node reads data based on the logical address and performs calculations through the thread register.

[0050] The first processing node reads the required data blocks from the persistent cache resident area based on the located physical address and loads them into the vector or scalar units of its internal thread register. Since the resident area is located on-chip, the access latency is much lower than that of global memory, and it supports high-bandwidth concurrent reads.

[0051] The threads in the core array execute the computational instructions described by the packet in parallel using SIMD / SIMT. For graphics tasks, this could be performing pixel shading or geometric transformations; for general-purpose tasks, it could be performing matrix operations or nonlinear activations. All operands are derived from registers, avoiding frequent access to external memory and thus significantly improving energy efficiency.

[0052] If a temporary scalar or vector result is generated, it is preferentially kept in a register for use by subsequent instructions; if it needs to be persisted or shared by other cores, the next step of the write-back process is initiated.

[0053] Step 204: The first processing node writes the calculation result back to the persistent cache resident area.

[0054] After the calculation is complete, the first processing node writes the final result data, such as color values, depth values, and convolution outputs, back to the corresponding location in the persistent cache according to the logical address specified in the resource binding information. The write-back can adopt either active refresh or delayed write-back mode to balance performance and consistency.

[0055] This application's embodiments introduce a persistent cache resident area to replace the traditional dedicated buffer mechanism, enabling dynamic sharing and flexible reuse of graphics computing tasks and general computing tasks on a unified high-speed cache resource. This effectively reduces the waste of on-chip storage resources and improves cache utilization. By parsing the resource binding information in the computing packet, the first processing node is precisely guided to locate and read the required data, enhancing the controllability and efficiency of data access and improving bandwidth-energy efficiency.

[0056] Based on the above embodiments, the method further includes: After obtaining the computation packet, acquire the creation timestamp and expiration timestamp of the computation packet allocation; The computed packet containing the creation timestamp and expiration timestamp is stored in the persistent cache resident area.

[0057] This solution achieves fine-grained control over intermediate data in computing tasks by introducing a packet lifecycle management mechanism. Specifically, before a computing task is scheduled for execution, the control / scheduling logic responsible for managing the lifecycle of computing packets assigns a unique lifecycle identifier to each packet, namely a creation timestamp and an expiration timestamp. The creation timestamp records the starting point when the packet data enters the cache, marking the beginning of its validity period that can be accessed by consumers; while the expiration timestamp presets the maximum lifespan of the data, preventing cached resources from being occupied by invalid or expired data for extended periods.

[0058] After the computation packets with timestamp information are stored in the persistent cache, the control / scheduling logic responsible for managing the lifecycle of the computation packets can dynamically monitor the status of each packet at runtime. When a subsequent consumer (such as a graphics shader, AI computing unit, etc.) finishes reading the data of the packet, or when the current system time has reached the preset expiration timestamp, the control / scheduling logic responsible for managing the lifecycle of the computation packets can automatically determine that the packet is no longer needed, mark it as invalid, and release the cache space it occupies.

[0059] This timestamp-based lifecycle management approach makes the allocation and reclamation of cached resources predictable and automated, avoiding resource leaks or redundant residency issues caused by the lack of explicit management in traditional architectures.

[0060] Based on the above embodiments, this mechanism supports collaborative management in multi-consumer scenarios, namely, monitoring the consumption status of computation packets; the consumption status is used to characterize whether the first processing node that needs to consume the computation packet has completed the reading operation; if it is determined based on the consumption status that all the first processing nodes that need to consume the computation packet have completed the reading operation of the computation packet, or the current time has reached the expiration timestamp, then the computation packet is marked as invalid, and the resources in the persistent cache area occupied by the computation packet are restored to an available state.

[0061] The first processing node also includes a lifecycle management module for computation packets. This module continuously monitors the consumption status of computation packets, tracking which first processing nodes (as consumers) need to read the data generated by the computation packet and monitoring whether these consumers have completed the data reading operation. This monitoring ensures that data is not prematurely released when there are still dependent parties that need it. The lifecycle management module of the computation packet will only determine that the computation packet has entered an invalid state when all relevant first processing nodes have successfully read the required data, or even if there are incomplete reads but the current time has exceeded the preset expiration timestamp (to prevent resource deadlock due to anomalies).

[0062] Once marked as invalid, the persistent cache resources it occupied will be immediately reclaimed and restored to an available state. The core advantage of this mechanism lies in its automated, on-demand reclamation of cache resources, avoiding the long-term cache occupation or resource leakage problems caused by the lack of a clear lifecycle in traditional architectures. This allows valuable on-chip storage resources to dynamically flow and be reused between different computing tasks (whether graphics rendering or AI computing), significantly improving cache utilization, reducing the frequency of global memory access, and thus optimizing the overall system's energy efficiency and performance.

[0063] Figure 3 This document provides a schematic diagram of a computation packet lifecycle management process for embodiments of this application. The process revolves around a persistent cache residency area, achieving on-demand production, efficient consumption, and timely recycling of computational data through collaborative interaction among producers, consumers, and the lifecycle management module. The entire process uses computation packets as data carriers and lifecycle management as its core, ensuring that data resides in the on-chip cache in stages and resources are rapidly released after all users have completed their tasks. The specific steps are as follows: Step 301: The producer receives the computation packets after the upper-layer task decomposition, writes the corresponding result data or intermediate data into the persistent cache area, and creates associated packet metadata for the data, such as logical address, data size, version number, dependency list, etc., and registers the initial lifecycle state of the packet, wherein the initial lifecycle state can be a readable state.

[0064] Step 302: The producer registers the packet lifecycle with the lifecycle management module, and the lifecycle management module generates a creation timestamp and an expiration timestamp for the computation packet.

[0065] Step 303: The lifecycle management module monitors the consumption status of the computing packet in real time.

[0066] Step 304: The consumer reads the computation packet from the persistent cache residency area and notifies the lifecycle management module that the reading is complete after the reading is finished.

[0067] Step 305: The lifecycle management module checks whether all consumers have completed reading the computation packet within the validity period.

[0068] Step 306: If all consumers have finished reading, then reclaim the cache resources, that is, delete the computation packet from the persistent cache area, or restore the logical address occupied by the computation packet to availability.

[0069] Step 307: The lifecycle management module sends a message to the producer to restore the logical address occupied by the computation packet to a usable address.

[0070] Step 308: If there are still consumers who have not finished reading and the expiration time has not yet arrived, the lifecycle management module will continue to monitor.

[0071] This application's embodiments introduce a packet lifecycle management mechanism to monitor the consumption status of computation packets in real time. By combining creation timestamps and expiration timestamps for dual determination, and ensuring that all relevant consumers have completed data reading or have exceeded the preset lifecycle, the computation packets are promptly marked as invalid and their occupied persistent cache residency resources are reclaimed. This achieves automated and refined management of cache resources, effectively avoiding memory leaks and resource waste, and improving cache utilization and overall system operating efficiency.

[0072] Based on the above embodiments, after the first processing node generates the calculation result, the calculation result can be written back to the persistent cache area in the following way: Obtain the packet dependency graph; the packet dependency graph includes multiple nodes and edges; nodes are used to represent computation packets, and edges are used to represent the dependencies between computation packets; Based on the packet dependency graph, the third processing unit in the computing core array that has a dependency relationship with the first processing node corresponding to the calculation result is obtained; The calculation results are written to the shared memory of the third processing unit; where the shared memory is part of the persistent cache resident area.

[0073] In the specific implementation process, after completing the current computing task, when the first processing node writes the computing results back to the persistent cache, it does not store them randomly, but performs intelligent routing based on the pre-built packet dependency graph.

[0074] Packet dependency graphs can be the result of previous compilation, scheduling, or analysis phases. A packet dependency graph is a directed graph structure in which each node represents a computation packet and each edge represents the data dependency between two computation packets.

[0075] When constructing a packet dependency graph, the predecessor and successor packets of each computation packet can be obtained. A predecessor packet is a packet that must have completed execution before the current computation packet begins execution. The execution of the current computation packet depends on the data or state generated by the predecessor packet. In the packet dependency graph, the predecessor packet is upstream of the current packet, and directed edges point from the predecessor packet to the current computation packet. It can be understood that a computation packet can have multiple predecessor packets. During execution, the current computation packet can only enter the ready queue to await scheduling after all predecessor packets have completed. A successor packet is a packet that depends on the execution result of the current computation packet. After the current packet completes, it may unlock the dependencies of one or more successor packets. In the dependency packet graph, the successor packet is downstream of the current computation packet, and directed edges point from the current computation packet to the successor packet. During execution, when the current computation packet is completed, the graphics processing hardware checks the dependency count of its successor packets. If all the predecessor packets of a successor packet have been completed, the successor packet becomes ready and waits for scheduling.

[0076] Once the predecessor and successor packets for each computation packet are determined, a directed acyclic graph can be drawn to obtain a packet dependency graph, and the corresponding scheduling priority and resource requirements can be attached to each node of the packet dependency graph.

[0077] When the first processing node completes its computation and prepares to write the result back, the system first obtains the packet dependency graph corresponding to that packet and parses it to determine which subsequent packets (i.e., consumers) need to use the computation result. The execution units corresponding to these subsequent packets are the third processing units with dependencies. The system then determines the computing cluster or thread block to which these third processing units belong and locates their efficiently accessible shared memory regions. This shared memory is part of the persistent cache residency area and is typically located between adjacent computing units, supporting low-latency, high-bandwidth data sharing.

[0078] Subsequently, the system actively writes the calculation results into the shared memory, allowing the third processing unit to directly access the resident intermediate results without needing to read data from remote global memory or across cluster paths during task execution. This dependency-based targeted write mechanism not only improves data locality and memory access efficiency but also reduces redundant data copying and global memory bandwidth consumption.

[0079] Based on the above embodiments, the computing execution hardware further includes an asynchronous memory transfer accelerator, and the method further includes: The asynchronous memory transport accelerator receives asynchronous multicast load requests and moves the original data from global memory to the persistent cache resident area within the asynchronous multicast load request.

[0080] In the specific implementation process, the computing execution hardware not only includes a computing core array for performing computing tasks, but also integrates a dedicated asynchronous memory transfer accelerator, such as a Tensor Memory Accelerator (TMA) or a similar DMA hardware unit. This asynchronous memory transfer accelerator runs independently of the main computing pipeline and can perform data transfer operations in parallel.

[0081] During task execution, when a computing task is about to start, the system generates an asynchronous multicast loading request in advance. This request is issued by the scheduler or control logic and sent to the asynchronous memory transport accelerator. The request contains the location information of the raw data to be loaded, such as the physical or virtual address in global memory, the data size, the target residency region (i.e., the logical address in the persistent cache residency region), the data layout format, and possible multicast targets. This means loading the same data simultaneously into a shared region accessible to multiple computing units to support parallel consumption.

[0082] Upon receiving the request, the asynchronous memory transfer accelerator initiates a data transfer process in the background. This process efficiently moves the required raw data, such as image textures, neural network weights, geometric vertex data, or initial input tensors, from bandwidth-constrained, high-latency global memory to a high-speed on-chip storage area closer to the computing core—the persistent cache resident area. The entire transfer process does not block the computing core's operation, thus achieving parallelization of computation and data transfer.

[0083] Furthermore, after data transfer is complete, the asynchronous memory transfer accelerator sends a completion notification to the relevant consumer computing unit (the first processing node) via hardware interrupts, status register updates, or event signals. This allows the consumer to begin computation immediately upon data readiness, avoiding resource waste caused by polling and waiting. In this way, bandwidth pressure and access latency caused by frequent access to global memory are reduced, improving data locality and cache utilization.

[0084] Building upon the above embodiments, the persistent cache residency area can be divided into an area for graphics processing tasks and an area for general computing tasks. The size of these two areas is dynamically adjustable, meaning that the current workload is identified as primarily graphics processing or general computing by monitoring the types of tasks currently pending execution by the computing hardware at runtime. This monitoring can be implemented using various methods, such as task queue analysis, packet type statistics, scheduler context information, or hardware performance counters. For example, when a large number of rasterization or shading stage computation packets are detected being submitted, it is determined to be a graphics-intensive workload; when a large number of matrix multiply-accumulate (GEMM) or tensor core tasks are detected, it is determined to be an AI / general computing-intensive workload.

[0085] Based on the identified task type, the functional allocation strategy of the persistent cache residency area is dynamically adjusted. In scenarios dominated by graphics workloads, a larger proportion of the persistent cache space is automatically allocated to store geometric data (such as vertex attributes and primitive information) and fragment data (such as color, depth, and stencil values) to meet the needs of high-throughput rendering pipelines. In scenarios dominated by general computing workloads, more cache space is allocated to the residency area for matrix and vector data, serving activation caching, weight reuse, or intermediate feature map storage in AI inference.

[0086] In addition, when dividing the persistent cache residency area, factors such as geometric / fragment data volume, matrix tensor size, cache hit rate, bandwidth pressure, latency metrics, and task queue ratio can be considered.

[0087] In this context, geometry / fragment data volume refers to the number of transformed vertices output by the vertex shader, the list of triangles assembled from primitives, the number of fragments (pixels) generated by rasterization, and the related buffered data such as depth, color, and stencil data in the graphics rendering pipeline. This data typically exhibits significant inter-frame variations (e.g., scene complexity). If the current frame has high geometric complexity (a large number of triangles), more space needs to be reserved for the vertex buffer, primitive buffer, and depth buffer to prevent frequent data eviction. Conversely, if fragments are dense (high resolution), the demand for intermediate data in the color buffer and fragment shading increases. The partitioning module should estimate the required capacity based on information provided by the graphics API (such as viewport size and number of draw calls) and dynamically expand the corresponding partitions.

[0088] The matrix tensor size refers to the size of the input feature maps, weight matrices, gradients, and intermediate activation tensors involved in typical operations (such as convolution and matrix multiplication) in AI / general computing tasks. This data is typically stored in NCHW or NHWC formats, and can reach hundreds of MB or even GB in size, but on-chip caches only store the sub-tiles required for the current computation window. The tensor size determines the minimum cache capacity required for a single resident operation. For example, if block matrix multiplication is used, space needs to be allocated for two input matrix tiles and one output tile; if convolution uses a sliding window, multiple rows of input features and weights need to be cached. When partitioning, the instantaneous working set size needs to be calculated based on the task configuration (such as batch size and number of channels), and the general computing partition threshold needs to be adjusted accordingly.

[0089] Cache hit rate reflects whether the current partition size can effectively accommodate the data working set. A low hit rate means frequent cache misses, causing a large number of accesses to be redirected to global memory. The partitioning strategy should monitor the hit rate of each partition in real time. When the hit rate of a partition consistently falls below a preset threshold (e.g., 90%), it indicates insufficient capacity, and some capacity needs to be transferred from other partitions with higher hit rates or less space. Conversely, if the hit rate of a partition is extremely high (close to 100%), its space can be appropriately reduced to conserve resources. This feedback mechanism is similar to the dynamic cache partitioning of modern CPUs, but it pays more attention to task type.

[0090] Bandwidth pressure refers to the actual utilization rate of global memory bandwidth. If a certain type of task generates a large number of global memory read and write requests, causing bandwidth to approach saturation, then the corresponding on-chip cache capacity needs to be increased to absorb the access and reduce contention on the global bus. The number of memory transactions and bandwidth usage for each task are obtained through hardware performance counters. When the bandwidth usage of graphics rendering spikes (such as with a large amount of texture sampling or depth testing), the graphics partition is increased; when weight loading for AI inference becomes a bottleneck, the general computing partition is increased. Partitioning decisions need to consider both bandwidth pressure and hit rate, because high bandwidth pressure is often a sign of insufficient caching.

[0091] Latency metrics characterize the varying sensitivities of different tasks to access latency. For example, real-time graphics rendering has strict requirements for frame latency (it must be completed within 16ms), while offline training tasks may tolerate higher latency. Latency-sensitive jobs should receive more cache resources to ensure that almost all of their core data is hit on-chip, avoiding triggering longer global memory access latencies. The partitioning module can allocate additional capacity to high-priority tasks based on their QoS level or deadline, and even reserve dedicated space to prevent interference. Simultaneously, latency metrics also reflect whether the partition size is appropriate; if the average memory access latency of a task abnormally increases, it indicates the need for expansion.

[0092] The task queue percentage refers to the ratio of graphics computation packets to general computation packets in the scheduling queue. This reflects future workload trends and serves as a basis for feedforward prediction. If the percentage of graphics packets in the queue increases sharply (e.g., a large number of drawing calls are about to be processed), the scheduler will inform the cache management module in advance to expand the graphics partition, avoiding the overhead of temporary allocation. Conversely, if AI tasks accumulate, the general partition will be expanded first. Dynamic partitioning can be combined with queue depth to predict memory requirements in the next few minutes, thus enabling a smooth transition.

[0093] Based on the above factors, a layered hybrid partitioning strategy can be adopted. For example, during power-on or idle periods, an initial partition ratio (e.g., 60% for graphics and 40% for general applications) can be set according to the chip's typical application scenarios (e.g., gaming or inference), while ensuring a minimum capacity for each partition (e.g., each occupying 20%). During operation, the hardware performance monitoring unit (PMU) periodically (e.g., every frame or every 100μs) collects the hit rate, bandwidth utilization, and average access latency of each partition, while simultaneously obtaining the number of both types of packets in the current queue from the task scheduler. Weights are assigned to each factor, and a resource scarcity score for each partition is calculated through weighted summation. The scores of the image partition and the general partition are compared; if the score difference exceeds a threshold, partition adjustment is triggered. Adjustments can be made according to a preset step size, such as adjusting the total capacity by 5% each time. Alternatively, the adjustment magnitude can be determined based on how much the score difference exceeds the threshold.

[0094] The aforementioned dynamic functional partitioning not only improves the overall utilization of cache resources but also enhances the system's flexibility and adaptability. It allows limited on-chip high-speed storage to be allocated on demand among different types of tasks, avoiding resource bottlenecks or wasted resources due to fixed allocation in traditional architectures. Simultaneously, because the persistent cache resident area supports data residency and reuse across computation stages, combined with the dynamic partitioning mechanism, it can further reduce the frequency of access to global memory, lower bandwidth pressure and power consumption, thereby comprehensively improving the performance and energy efficiency of the computing execution hardware under mixed workloads.

[0095] Based on the above embodiments, after the computation task is completed, the final result is written to global memory.

[0096] Figure 4This is another data flow architecture diagram provided in this application embodiment. Global memory is used to store raw data and final results. A persistent cache resides instead of a traditional dedicated buffer, supporting the reuse of graphics and AI workloads. When executing computation tasks, raw data can be loaded from global memory into the persistent cache resides all at once via TMA asynchronous multicast. Producers can write graphics-related intermediate data and general computation-related intermediate data into the persistent cache resides. Graphics-related intermediate data includes geometric computation packets, raster computation packets, etc.; general computation-related intermediate data includes AI computation packets, etc. Consumers read the required data from the persistent cache resides into their consumer computing units, and after completing the computation, write the computation results back to the persistent cache resides. During this period, the packet lifecycle management module manages the computation packets in the persistent cache resides. After the last consumer completes the computation, the final result is written into global memory.

[0097] This application embodiment avoids frequent reading and writing of intermediate data by uniformly writing the final result back to global memory after the task is completed, reduces the number of accesses to the high-latency, high-power global memory, and effectively reduces system bandwidth pressure and energy consumption; at the same time, it ensures that only valid results that have undergone complete processing are persistently stored, improves data consistency and system reliability, and is conducive to achieving efficient memory management and result output control.

[0098] This application also provides an electronic device, including the computing execution hardware provided in this application embodiment. For example, the electronic device may be a server or server cluster, a PC, a mobile phone, a robot, etc.

[0099] This application also provides a computer program product, which includes computer program instructions. These computer program instructions are read and run by the computing execution hardware provided in this application to execute the data flow control method for the computing task provided in this application.

[0100] This application also provides a computer-readable storage medium storing computer program instructions. These computer program instructions are read and executed by the computing execution hardware provided in this application to perform the data flow control method for the computing task provided in this application.

[0101] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some communication interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.

[0102] Furthermore, the units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0103] Furthermore, the functional modules in the various embodiments of this application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0104] In this document, relational terms such as first and second are used only to distinguish one entity or operation from another entity or operation, without necessarily requiring or implying any such actual relationship or order between these entities or operations.

[0105] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A data flow control method for a computing task, characterized in that, The invention is applied to computing execution hardware, which includes a processing unit and a storage unit. The processing unit includes a computing core array, and the storage unit includes global memory and a persistent cache resident area. The method includes: The first processing node in the computing core array reads and parses the computing packets from the persistent cache resident area to obtain resource binding information; the computing packets are obtained by decomposing the computing tasks; the computing tasks include graphics computing tasks and general computing tasks; the resource binding information is used to indicate the data resources that the current computing packet needs to access during execution and their corresponding relationships. The first processing node locates the logical address in the persistent cache resident area according to the resource binding information; the persistent cache resident area stores data generated by the second processing node in the computing core array and original data migrated from the storage unit; The first processing node reads data based on the logical address and performs calculations through the thread register; The first processing node writes the calculation result back to the persistent cache resident area.

2. The method according to claim 1, characterized in that, The method further includes: After obtaining the computation packet, acquire the creation timestamp and expiration timestamp of the computation packet. The calculated packet containing the creation timestamp and the expiration timestamp is stored in the persistent cache resident area.

3. The method according to claim 2, characterized in that, The method further includes: Monitor the consumption status of the computation packet; the consumption status is used to indicate whether the first processing node that needs to consume the computation packet has completed the reading operation; If, based on the consumption status, it is determined that all first processing nodes that need to consume the computation packet have completed the operation of reading the computation packet, or the current time has reached the expiration timestamp, then the computation packet is marked as expired, and the resources of the persistent cache area occupied by the computation packet are restored to an available state.

4. The method according to claim 1, characterized in that, The step of writing the calculation result back to the persistent cache residency area includes: Obtain the packet dependency graph; the packet dependency graph includes multiple nodes and edges; the nodes are used to represent the computation packets, and the edges are used to represent the dependencies between computation packets; Based on the packet dependency graph, obtain the third processing unit in the computing core array that has a dependency relationship with the first processing node corresponding to the calculation result; The calculation result is written into the shared memory of the third processing unit; wherein the shared memory is part of the persistent cache resident area.

5. The method according to claim 4, characterized in that, The packet dependency graph is constructed using the following method: Obtain the preceding and succeeding packets for each computation packet; The packet dependency graph is constructed based on the predecessor and successor packets of each computation packet.

6. The method according to claim 1, characterized in that, The computing execution hardware also includes an asynchronous memory transfer accelerator, and the method further includes: The asynchronous memory transport accelerator receives asynchronous multicast loading requests and transports the original data from global memory to the persistent cache resident area in the asynchronous multicast loading request.

7. The method according to claim 1, characterized in that, The method further includes: Monitor the type of task currently pending execution by the computing execution hardware; the task types include graphics processing tasks and general computing tasks. The persistent cache residency area is functionally divided according to the task type.

8. The method according to any one of claims 1-7, characterized in that, The method further includes: After the computation task is completed, the final result is written to the global memory.

9. A computing execution hardware, characterized in that, It includes a processing unit and a storage unit. The processing unit includes a computing core array, and the storage unit includes global memory and a persistent cache resident area. The global memory stores computer program instructions, which are read and executed by the processing unit to perform the method as described in any one of claims 1-8.

10. An electronic device, characterized in that, Includes the computing execution hardware as described in claim 9.

11. A non-transitory computer-readable storage medium, characterized in that, The non-transitory computer-readable storage medium stores computer instructions that, when executed by a computer, cause the computer to perform the method as described in any one of claims 1-8.

12. A computer program product, characterized in that, It includes computer program instructions, which, when read and executed by a processor, perform the method as described in any one of claims 1-8.

Images