Cache management method and device, computer device, medium, chip and product

Abstract

This application relates to a cache management method, apparatus, computer device, medium, chip, and product. The method includes: acquiring a target task and a management command queue; the management command queue containing various management commands for the target task; when the last-level cache module does not contain dependency data required by a target subtask within the target task, in response to a first management command, reading the dependency data required by the target subtask from memory and storing the dependency data in the last-level cache module; in response to a second management command, initiating the execution of the target subtask; the target subtask includes an accelerator reading dependency data from the last-level cache module. This method can improve the processing efficiency of the target task.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a cache management method, apparatus, computer equipment, storage medium, chip, and computer program product. Background Technology

[0002] To improve the speed at which chips perform tasks, various types of accelerators are typically integrated into the chip. Accelerators are connected to the chip via a bus and can access the chip's memory through the bus.

[0003] In traditional technologies, an LLC (Last Level Cache) module is placed between the bus and the UMC (Unified Memory Controller) module. The LLC module is used to cache memory data that has been frequently accessed recently. When the accelerator needs to obtain dependent data, it first accesses the LLC module to obtain the dependent data. If the dependent data is not found in the LLC module, the accelerator then accesses the memory to obtain the dependent data.

[0004] However, in traditional technologies, since the LLC module only stores frequently accessed memory data, when the dependent data is not frequently accessed memory data, the accelerator needs to access memory to obtain the dependent data, resulting in low efficiency in obtaining dependent data and consequently a decrease in the chip's computing performance. Summary of the Invention

[0005] Therefore, it is necessary to provide a cache management method, apparatus, computer device, computer-readable storage medium, chip, and computer program product to address the aforementioned technical problems.

[0006] Firstly, this application provides a cache management method, the method comprising:

[0007] Obtain the target task and management command queue; the management command queue contains various management commands for the target task;

[0008] When the last-level cache module does not contain the dependency data required by the target subtask in the target task, in response to the first management command, the dependency data required by the target subtask is read from memory and the dependency data is stored in the last-level cache module;

[0009] In response to the second management command, the target subtask is initiated and executed; the target subtask includes the accelerator reading the dependency data from the last-level cache module.

[0010] In one embodiment, the step of acquiring the target task and managing the command queue includes:

[0011] Obtain the target task and the corresponding data lifecycle information; the target task includes various subtasks;

[0012] Based on the data lifecycle information, management commands corresponding to each subtask are generated, and a management command queue is constructed based on each management command.

[0013] In one embodiment, generating management commands corresponding to each of the subtasks based on the data lifecycle information, and constructing a management command queue based on each of the management commands, includes:

[0014] For each subtask, the category of the accelerator in the subtask is determined based on the data lifecycle information; the data lifecycle information includes the subtask association relationships between the subtasks.

[0015] Based on the category, generate the management commands corresponding to the subtask;

[0016] Based on the subtask relationships of each subtask and each management command, construct the management command queue for the target task.

[0017] In one embodiment, determining the category of the accelerator in the subtask based on the data lifecycle information includes:

[0018] In the data lifecycle information, the accelerator and associated data in the subtask are identified;

[0019] If the associated data contains only dependency data, the accelerator is classified as the first category.

[0020] If the associated data contains only result data, then the accelerator is classified as the second category;

[0021] If the associated data contains result data and dependency data, the accelerator is classified as the third category.

[0022] In one embodiment, generating the management commands corresponding to the subtasks according to the category includes:

[0023] If the category is the first category, generate the first management command, the fourth management command, and the fifth management command corresponding to the subtask;

[0024] If the category is the second category, generate the second management command and the third management command corresponding to the subtask;

[0025] If the category is the third category, generate the first management command, the second management command, the third management command, the fourth management command, and the fifth management command corresponding to the subtask.

[0026] In one embodiment, the data lifecycle information includes associated data of the subtask, and the associated data includes dependency data. If the category is a first category, generating a first management command, a fourth management command, and a fifth management command corresponding to the subtask includes:

[0027] If the category is the first category, the first management command is generated according to the location of the dependency data; the first management command is used to read the dependency data required by the subtask from memory to the last-level cache module;

[0028] A fourth management command corresponding to the subtask is generated; the fourth management command is used to instruct the accelerator to end the execution of the subtask;

[0029] A fifth management command is generated to generate the dependency data; the fifth management command is used to release the dependency data in the last-level cache to the memory.

[0030] In one embodiment, the data lifecycle information includes associated data of the subtask, the associated data including result data, and the step of generating a second management command and a third management command corresponding to the subtask if the category is a second category includes:

[0031] If the category is the second category, a third management command corresponding to the result data is generated; the third management command is used to pre-allocate cache lines for the result data in the last-level cache;

[0032] A second management command corresponding to the subtask is generated; the second management command is used to start the accelerator to execute the subtask.

[0033] In one embodiment, constructing the management command queue of the target task based on the subtask associations of each subtask and each management command includes:

[0034] Based on the dependencies of each subtask and each management command, a management command flowchart for the target task is generated; the management command flowchart includes the execution order of each management command and the command association relationship between each management command;

[0035] For each management command, based on the command association relationship, determine the number of preceding associated commands of the management command, and determine the subsequent associated commands of the management command;

[0036] A management command queue is constructed based on each management command, the number of its preceding associated commands, and its subsequent associated commands.

[0037] In one embodiment, the management command queue includes target management commands of target type and non-target management commands of non-target type; the target management commands include a second management command and a fourth management command; the non-target management commands include a first management command, a second management command, and a fifth management command.

[0038] In one embodiment, after acquiring the target task and managing the command queue, the method further includes:

[0039] In the management command queue, determine each target management command of the target type and generate an execution identifier for each target management command;

[0040] Based on the execution identifiers and target management commands, construct the subtask management table for the target task.

[0041] In one embodiment, after initiating the execution of the target subtask in response to the second management command, the method further includes:

[0042] Based on the generation probability of each of the management commands, at least one of the third, fourth, or fifth management commands is executed.

[0043] In one embodiment, the generation probability includes a first generation probability, a second generation probability, and a third generation probability. The step of executing at least one of the third, fourth, or fifth management commands based on the generation probabilities of each management command includes:

[0044] Based on the first generation probability, a third management command is executed, and in response to the third management command, a cache line is pre-allocated in the last-level cache module for the result data of the target subtask; the cache line is used to store the result data.

[0045] Based on the second generation probability, execute the fourth management command, and in response to the fourth management command, terminate the execution of the target subtask;

[0046] Based on the third generation probability, a fifth management command is executed, and in response to the fifth management command, the dependent data in the last-level cache is released to the memory.

[0047] In one embodiment, the method further includes:

[0048] The current management command is retrieved from the management command queue, and the number of preceding associated commands corresponding to the current management command is determined; the current management command is any one of the first management command, the second management command, the third management command, the fourth management command, and the fifth management command.

[0049] Based on the number of preceding associated commands, determine whether the current management command meets the execution conditions; the execution conditions are used to determine whether to respond to the current management command.

[0050] In one embodiment, the method further includes:

[0051] Determine whether the target management command was executed successfully;

[0052] If the target management command is executed successfully, the execution identifier corresponding to the target management command is updated to the successful execution identifier in the subtask management table; the successful execution identifier is used to instruct the cache management engine module to execute the next management command.

[0053] If the target management command is executed successfully, the execution identifier corresponding to the target management command is updated to the failure execution identifier in the subtask management table; the failure execution identifier is used to instruct the cache management engine module to continue executing the target management command until the execution identifier becomes the success execution identifier.

[0054] Secondly, this application also provides a cache management device, the device comprising:

[0055] The acquisition module is used to acquire the target task and the management command queue; the management command queue contains various management commands for the target task.

[0056] The reading module is used to read the dependency data required by the target subtask in the target task from memory and store the dependency data in the last-level cache module in response to a first management command when the last-level cache module does not have the dependency data required by the target subtask in the target task.

[0057] An execution module is used to initiate the execution of the target subtask in response to a second management command; the target subtask includes an accelerator reading the dependency data from the last-level cache module.

[0058] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0059] Obtain the target task and management command queue; the management command queue contains various management commands for the target task;

[0060] When the last-level cache module does not contain the dependency data required by the target subtask in the target task, in response to the first management command, the dependency data required by the target subtask is read from memory and the dependency data is stored in the last-level cache module;

[0061] In response to the second management command, the target subtask is initiated and executed; the target subtask includes the accelerator reading the dependency data from the last-level cache module.

[0062] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the following steps:

[0063] Obtain the target task and management command queue; the management command queue contains various management commands for the target task;

[0064] When the last-level cache module does not contain the dependency data required by the target subtask in the target task, in response to the first management command, the dependency data required by the target subtask is read from memory and the dependency data is stored in the last-level cache module;

[0065] In response to the second management command, the target subtask is initiated and executed; the target subtask includes the accelerator reading the dependency data from the last-level cache module.

[0066] Fifthly, this application also provides a chip, the chip including programmable logic circuitry and / or program instructions, which, when the chip is running, implement:

[0067] Obtain the target task and management command queue; the management command queue contains various management commands for the target task;

[0068] When the last-level cache module does not contain the dependency data required by the target subtask in the target task, in response to the first management command, the dependency data required by the target subtask is read from memory and the dependency data is stored in the last-level cache module;

[0069] In response to the second management command, the target subtask is initiated and executed; the target subtask includes the accelerator reading the dependency data from the last-level cache module.

[0070] Sixthly, this application also provides a computer program product, including a computer program that, when executed by a processor, performs the following steps:

[0071] Obtain the target task and management command queue; the management command queue contains various management commands for the target task;

[0072] When the last-level cache module does not contain the dependency data required by the target subtask in the target task, in response to the first management command, the dependency data required by the target subtask is read from memory and the dependency data is stored in the last-level cache module;

[0073] In response to the second management command, the target subtask is initiated and executed; the target subtask includes the accelerator reading the dependency data from the last-level cache module.

[0074] The aforementioned cache management method, apparatus, computer device, storage medium, chip, and computer program product acquire a target task and a management command queue. The management command queue contains various management commands for the target task. When the last-level cache module does not contain the dependency data required by the target subtask in the target task, in response to a first management command, the dependency data required by the target subtask is read from memory and stored in the last-level cache module. In response to a second management command, the execution of the target subtask is initiated. The target subtask includes the accelerator reading the dependency data from the last-level cache module. Using this method, the management command queue contains multiple management commands corresponding to the target task. Executing the management commands in the order of the management command queue can trigger the execution of the target subtask in the target task. When it is analyzed that the target subtask requires dependency data but the last-level cache module does not contain the dependency data, the dependency data required by the target subtask can be stored in the last-level cache from memory before executing the target step. This allows the accelerator to directly retrieve the dependency data from the last-level cache before each execution of the target subtask, avoiding retrieval from memory, improving the accelerator's data reading speed, and thus improving the chip's computing performance. Attached Figure Description

[0075] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0076] Figure 1 This is an application environment diagram of a cache management method in one embodiment;

[0077] Figure 2 This is a schematic diagram of the structure of a chip in the prior art in one embodiment;

[0078] Figure 3 This is a schematic diagram of the structure of a prior art chip containing an LLC module in one embodiment;

[0079] Figure 4 A system architecture diagram for a second existing technology computer;

[0080] Figure 5 This is a schematic diagram of the reconstructed core in one embodiment of the second conventional technique;

[0081] Figure 6 This is a flowchart illustrating a cache management method in one embodiment;

[0082] Figure 7 This is a schematic diagram of the process for obtaining the management command queue in one embodiment;

[0083] Figure 8 This is a schematic diagram of the process for generating a management command queue in one embodiment;

[0084] Figure 9 This is a schematic diagram of the data lifecycle of a target task in one embodiment;

[0085] Figure 10 This is a flowchart illustrating the process of determining the accelerator category in one embodiment;

[0086] Figure 11 This is a flowchart illustrating the process of generating management commands in one embodiment;

[0087] Figure 12 This is a flowchart illustrating the process of generating management commands under the first category in one embodiment;

[0088] Figure 13 This is a flowchart illustrating the process of generating management commands under the second category in one embodiment;

[0089] Figure 14 This is a flowchart illustrating the process of constructing a management command queue in one embodiment;

[0090] Figure 15 This is a flowchart of the management commands for a target task in one embodiment;

[0091] Figure 16 Here is a flowchart of the management commands for the target task in another embodiment;

[0092] Figure 17 This is a schematic diagram illustrating the management of the command queue in one embodiment;

[0093] Figure 18 This is a flowchart illustrating the process of constructing a subtask management table in one embodiment;

[0094] Figure 19 This is a schematic diagram of a subtask management table in one embodiment;

[0095] Figure 20 This is a flowchart illustrating the execution of management commands in one embodiment;

[0096] Figure 21 This is a flowchart illustrating the process of determining whether a management command meets the execution conditions in one embodiment;

[0097] Figure 22 This is a flowchart illustrating the execution of target management commands based on a subtask management table in one embodiment.

[0098] Figure 23 This is a schematic diagram of fine-grained lifecycle management in one embodiment;

[0099] Figure 24 This is a structural block diagram of a cache management device in one embodiment;

[0100] Figure 25 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0101] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0102] It should be noted that the terms "first," "second," etc., used in this application can be used to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish the first element from the second element. The terms "comprising" and "having," and any variations thereof, used in this application, are intended to cover non-exclusive inclusion. The term "multiple" used in this application refers to two or more. The term "and / or" used in this application refers to one of the embodiments, or any combination of multiple embodiments.

[0103] The cache management method provided in this application embodiment can be applied to, for example, Figure 1 The application environment shown is a chip. The chip includes a Last Level Cache (LLC, also known as the last-level cache module, although...) Figure 1 The example shown includes a first-level cache, but in reality, a cache can include more levels, UMC (Unified Memory Controller), CME module (Cache Management Engine), memory, bus, and multiple accelerators. Figure 1The components shown include: CPU (Central Processing Unit), GPU (Graphics Processing Unit), NPU (Neural Processing Unit), VCD (Video Codec Decoder), DPU (Display Processing Unit), VCE (Video Codec Encoder), and DMA (Direct Memory Access). In fact, accelerators can also include more than... Figure 1 (More or fewer varieties shown).

[0104] The memory is connected via UMC and LLC, the CME is connected to the cache, and the cache, CME, and various accelerators are connected via a bus. Figure 1 The example shown illustrates a connection between the CME and LLC (with the LLC, CME, and each accelerator connected via a bus). Each accelerator can access the cache to read and write data via the bus. The CME can interact with each accelerator via the bus to learn about their operating status and control the data stored in the cache by accessing it. This application does not limit the specific structure of the CME in its embodiments.

[0105] To improve the speed at which chips perform tasks, various types of accelerators are typically integrated into the chip. Accelerators are connected to the chip via a bus and can access the chip's memory through the bus. Figure 2 This is a schematic diagram of the structure of a chip in the prior art, as shown in one embodiment. Figure 2 As shown, a System-on-a-Chip (SoC) typically integrates many different types of accelerators. The SoC relies on these accelerators to implement various functionalities. Common accelerators include CPU, GPU, NPU, VCD, VCE, DPU, and DMA. Accelerators are connected to the SoC via a bus, allowing them to access the SoC's memory.

[0106] In a System-on-a-Chip (SoC), many functions are implemented through the cooperation of multiple accelerators, which necessitates frequent data exchange between them. This data exchange is achieved by accessing shared memory. For example, a VCD player writes the decoded video to shared memory, and the DPU reads the decoded video from shared memory and displays it on the screen. Therefore, accelerators frequently access memory.

[0107] The accelerator's operation depends on data read from memory. Therefore, during memory reads, the accelerator is in a paused state, waiting for data to be read back. Because the memory controller takes too long to access memory chips, the accelerator spends a significant amount of time waiting for data to be read from memory. Furthermore, the bus bandwidth in a SoC is typically greater than the memory bandwidth, causing congestion when the accelerator accesses memory. This further increases the latency of accelerator memory access, reduces the actual runtime of the accelerator, and degrades the overall performance of the SoC.

[0108] In the first traditional technology, to address the performance degradation caused by frequent memory accesses by accelerators in a SoC, an LLC module is typically placed between the bus and the UMC module. The LLC module caches frequently accessed memory data. When the accelerator needs to retrieve dependent data, it first accesses the LLC module to obtain the data. If the dependent data is not found in the LLC module, the accelerator then accesses the memory to retrieve it. Figure 3 This is a schematic diagram of the structure of a prior art chip containing an LLC module in one embodiment.

[0109] like Figure 3 As shown, in most cases, the accelerator only needs to access the LLC to read back the dependent data. The latency of the accelerator accessing the LLC is much smaller than the latency of accessing memory. In most situations, the accelerator only needs to access the LLC and not memory, which also reduces memory congestion. In summary, the LLC significantly reduces the time the accelerator waits for data, increases the accelerator's actual runtime, and improves the overall performance of the SoC.

[0110] However, in traditional technologies, since the LLC module only stores frequently accessed memory data, when the dependent data is not frequently accessed memory data, the accelerator needs to access memory to obtain the dependent data, resulting in low efficiency in obtaining dependent data and consequently a decrease in the chip's computing performance.

[0111] In other words, whether the accelerator can read back the dependent data when accessing the LLC depends on the LLC hit rate. In desktop processor scenarios, the LLC hit rate is typically 50%-70%. When an LLC access miss occurs, the accelerator still needs to access memory to retrieve the dependent data. Therefore, when an LLC miss occurs, the SoC's computing performance will decrease due to the excessive latency of the accelerator accessing memory.

[0112] In the second traditional technique, GPUs typically use prefetching to resolve cache misses. Figure 4 A system architecture diagram for a second existing technology computer. Figure 4In this context, "execution codes in the original kernel" refers to the executable code within the original kernel. ExecutionCode 1, ExecutionCode 2, and ExecutionCode 3 represent execution code 1, execution code 2, and execution code 3, respectively. "Compiler" refers to the compiler, and "GSE operation table" refers to the GSE (Global Scheduling Ether) operation table. Operation 1 and Operation 3 represent operation 1 and operation 3, respectively. "Computation codes" refers to computer code. "Sync Hook #1" is synchronization hook 1, and "Sync Hook #2" is synchronization hook 2. "Execution Code 2" is kernel execution task 2. "Runtime" refers to the environment or stage in which the program runs. "Driver" refers to the driver, "CP" is Co-Processor, "CU" is Control Unit, "L2 Cache" is the secondary cache, "DMA Controller" is the direct memory access controller, "Memory" (usually DRAM), "SDMA" is distributed DMA, "ASIC" is an application-specific integrated circuit, "FPG" is a field-programmable gate array, and "Accelerator" is a hardware accelerator.

[0113] like Figure 4 As shown, the GPU has an L2 cache used to cache data required for CU computation. The GSE module helps the CU offload non-computational operations. The GPU's raw code is compiled by the Compiler into a GSE operation table and computation codes. Specifically, code requiring CU computation is compiled into computation code; code unsuitable for CU computation, such as L2 Cache Prefetch and L2 Cache Flush, is compiled into the GSE operation table.

[0114] Figure 5 This is a schematic diagram of the reconstructed kernel in one embodiment of the second conventional technique. For example... Figure 5 As shown, the PreloadData code is compiled into a GSE operation table, and GSE is responsible for executing this part of the code; the computation code is compiled into computation codes, and the data dependency is compiled into a Sync Hook, and CU is responsible for executing this part of the code.

[0115] When the CU executes Sync Hook #1 (NS), it sends req #1 (request 1) to the GSE; the GSE will then instruct the CP to send the Preload Data A instruction to the L2Cache; after the L2Cache completes Preload Data A, it will notify the GSE, and then the GSE will notify the CU that Sync Hook #1 (NS, Next State) was executed successfully.

[0116] However, in the second traditional technology, L2 cache is designed only for GPU memory access. Other types of accelerators still need to access memory directly, resulting in significant memory access latency. When other types of accelerators perform computations, the SoC's computational performance will still degrade due to the high memory access latency. Prefetching only data that the GPU depends on can only optimize the GPU's memory access latency and computational efficiency. When there are multiple types of accelerators in the SoC, it is impossible to optimize memory access latency for other types of accelerators. When other types of accelerators perform computations, the SoC's computational performance will still degrade due to the high memory access latency. Prefetching data that the GPU depends on is based on control flow. That is, there is a sequential relationship between prefetching and computation, and prefetching and computation cannot be executed in parallel; there is a sequential relationship between prefetchings, and prefetchings cannot be executed in parallel.

[0117] This application provides a cache management method. The management command queue contains multiple management commands corresponding to a target task. Executing the management commands in the order of the queue triggers the execution of target subtasks within the target task. When it is determined that a target subtask requires dependent data, but this dependent data is not present in the last-level cache module, the dependent data required by the target subtask can be stored in the last-level cache from memory before executing the target step. This allows the accelerator to directly retrieve dependent data from the last-level cache before each execution of the target subtask, avoiding retrieval from memory, thus improving the accelerator's data reading speed and consequently enhancing the chip's computational performance. Furthermore, this application's pre-fetching is not specific to any particular accelerator; any accelerator can achieve this effect, enabling the pre-fetching of data required by each accelerator and significantly improving the chip's computational efficiency.

[0118] Before introducing specific embodiments of the present invention, the technical terms involved in the present invention will be explained:

[0119] SoC: System-on-Chip, is a highly integrated integrated circuit that integrates a processor core, memory, various functional modules (such as analog circuits, digital signal processing units, etc.) and interface circuits on a single chip to form a complete system-level solution. It is widely used in the core computing and control scenarios of electronic devices.

[0120] CPU: Central Processing Unit. As the computing and control center of a computer system, it is responsible for executing instructions and processing data, and is the "brain" of the system.

[0121] GPU: Graphics Processing Unit. It has a large number of computing cores and is good at processing massive amounts of data (such as pixel, vertex and other graphics data) at the same time. It has powerful parallel computing capabilities. It can be used as a graphics card to handle tasks such as 3D modeling, image rendering and video encoding. It can also be used in non-graphics fields such as scientific computing (such as deep learning and cryptography) and big data processing.

[0122] NPU: Neural Processing Unit, is a core specifically designed to accelerate neural network computation. It contains a large number of multiply-accumulate units (MACs) to support parallel processing of matrix multiplication and accumulation operations.

[0123] VCD: Video Codec Decoder. Its core function is to restore compressed image data to its original pixel information. It is the reverse process of image encoding, similar to decompression, but it must strictly follow the rules of a specific encoding standard for data parsing.

[0124] VCE: Video Codec Encoder. Its core function is to convert raw image data into a compressed encoding format; while preserving the visual quality of the image as much as possible, it reduces data storage space and transmission bandwidth, and at the same time ensures that the decoding end can accurately restore the image.

[0125] DPU: Display Processing Unit, mainly prepares and provides display data for the display interface; its main functions include preprocessing multiple layers from memory, blending multiple layers, post-processing of blending, and generating display interface timing for the Display Interface.

[0126] DMA: Direct Memory Access, a module that allows peripherals (such as hard drives, network cards, graphics cards, etc.) to directly transfer data with system memory without CPU intervention; common DMA types include SDMA, PCIeDMA, and SATADMA.

[0127] UMC: Unified Memory Controller, is a module responsible for managing the data interaction between the SoC and DDR.

[0128] DDR: Double Data Rate SDRAM, is a type of memory used in SoCs (System-on-a-Chip) for temporary storage of running programs and data.

[0129] LLC: Last Level Cache, is the last level cache in the SoC. When the accelerator needs to access memory, it will first access the LLC. If the data is hit in the LLC, it can be read directly; if it is not hit, it will access the memory.

[0130] In one exemplary embodiment, such as Figure 6 As shown, a cache management method is provided, which can be applied to... Figure 1 Taking the chip in the example, the explanation includes steps 602 to 606. Wherein:

[0131] Step 602: Obtain the target task and manage the command queue.

[0132] The management command queue contains various management commands for the target task.

[0133] In implementation, the chip acquires the target task to be executed and its data lifecycle information. Then, the chip generates a management command queue for the target task based on the data lifecycle information. The management commands within this queue are used to manage the execution process of the target task.

[0134] Specifically, the target task refers to the task that the chip needs to execute, such as video encoding / decoding, artificial intelligence computing, image processing, etc. The chip acquires the target task and its data lifecycle information. The target task can consist of multiple subtasks, with sequential order and dependencies between them. The data lifecycle information includes associated data for each subtask. Associated data includes at least one of dependent data and result data. Based on the sequential order of the subtasks, task relationships, and associated data, the chip generates management commands for each subtask and constructs a management command queue. The management command queue includes at least one of the following management commands: a first management command, a second management command, a third management command, a fourth management command, or a fifth management command. It should be noted that the chip includes a CME module and a CPU module. Each management command is compiled by software in the CPU module and then stored in the CME module. The CME module is used to execute the management commands in the management command queue.

[0135] Step 604: When the last-level cache module does not contain the dependency data required by the target subtask in the target task, in response to the first management command, the dependency data required by the target subtask is read from memory and stored in the last-level cache module.

[0136] The target task contains subtasks. A target subtask is a subtask that requires dependent data, and that dependent data is not in the last-level cache module. In other words, the associated data for this subtask includes dependent data, and this dependent data is stored in memory.

[0137] In implementation, the chip responds to the first management command, reads the dependency data required by the target subtask from memory, and stores the dependency data in the last-level cache module.

[0138] Specifically, the target task includes subtasks. The chip includes a CME module and a CPU. The CME module executes management commands in the order of the management command queue. During the execution of management commands, the execution of subtasks is triggered simultaneously. If the subtask is a target subtask, it needs to acquire dependent data before execution, and this dependent data exists in memory. To ensure the successful execution of the target subtask, when generating the management command corresponding to the target subtask, the program in the CPU analyzes whether the target subtask requires dependent data and whether the dependent data does not exist in the last-level cache module. Then, the program in the CPU generates the first and second management commands corresponding to the target subtask. During the execution of the target subtask, the CME module executes the first management command corresponding to the target subtask in the order of the management command queue. In response to the first management command, the CME module retrieves the dependent data from memory in advance and stores the dependent data in the LLC module. The first management command is a non-target management command used to manage the chip's cache.

[0139] In one exemplary embodiment, if the dependent data corresponding to the target subtask is Data A, and Data A is not in the last-level cache module, then the first management command is Prefetch Data A (i.e., prefetch data A). In response to Prefetch Data A, the CME module reads Data A from memory and stores Data A in the last-level cache module.

[0140] Step 606: In response to the second management command, start executing the target subtask.

[0141] The target subtask includes the accelerator reading dependency data from the last-level cache module.

[0142] In implementation, after executing the first management command, the chip executes the second management command in the order of the management command queue. In response to the second management command, the chip initiates the execution of the target subtask. The target subtask includes the accelerator reading dependent data from the last-level cache module and processing that data. That is, the accelerator in the chip reads dependent data from the last-level cache module and, based on processing that dependent data, continues execution to complete the target subtask.

[0143] Specifically, the chip includes a CME module and a CPU. The CME module executes management commands in the order of the management command queue. Each management command corresponding to the target subtask includes at least a first management command and a second management command, with the second management command following the first. The execution condition for the second management command includes the completion of the first management command. When the first management command completes execution, the execution condition for the second management command is met, and the CME module executes the second management command sequentially. The second management command is a response to the command `Wait XXX start flag #N` (waiting for the accelerator start flag #N). `XXX` represents the accelerator in the chip used to execute the target subtask. `flag #N` represents the index of the generated result data in the target subtask. When the accelerator of the target subtask successfully starts, the second management command is successfully executed and completes. The accelerator in the chip starts executing the target subtask, reads dependent data from the last-level cache module, and processes the dependent data.

[0144] For example, the second management command is "Wait VCD start flag #1". In response to the second management command, the CME module starts executing the target subtask, indicating that the VCD has successfully started. When the VCD successfully starts executing the target subtask (flag #1), the CME module determines that the second management command has been successfully executed and continues to execute the next management command according to the order of the management command queue. Simultaneously, the VCD retrieves dependent data from the last-level cache module and processes the dependent data accordingly.

[0145] In the aforementioned cache management method, the management command queue contains multiple management commands corresponding to the target task. Executing the management commands in the order of the queue can trigger the execution of the target subtask within the target task. When it is determined that the target subtask requires dependent data but this dependent data is not present in the last-level cache module, the dependent data required by the target subtask can be stored in the last-level cache from memory before executing the target step. This allows the accelerator to directly retrieve dependent data from the last-level cache before each execution of the target subtask, avoiding retrieval from memory, thus improving the accelerator's data reading speed and ultimately enhancing the chip's computing performance.

[0146] In one exemplary embodiment, such as Figure 7 As shown, the specific processing procedure of step 602 includes steps 702 to 704. Wherein:

[0147] Step 702: Obtain the target task and the corresponding data lifecycle information.

[0148] The target task includes various sub-tasks.

[0149] In implementation, the chip acquires the target task and its corresponding data lifecycle information. The target task can be broken down into multiple subtasks, each involving an accelerator. That is, each subtask requires accelerator processing. Furthermore, dependencies exist between the subtasks. For example, after subtask A completes, subtasks B and C need to be executed. The data lifecycle information includes associated data related to each subtask. This associated data includes at least one of dependent data and result data. From another perspective, this data lifecycle information contains all the data involved in the target task, as well as the generation and transformation nodes for each piece of data.

[0150] Step 704: Generate management commands corresponding to each subtask based on the data lifecycle information, and construct a management command queue based on each management command.

[0151] In implementation, for each subtask, the chip determines the accelerator category within that subtask based on data lifecycle information, and generates the corresponding management command for that subtask based on the category and data lifecycle information. Then, the chip constructs a management command queue based on these management commands.

[0152] Specifically, the chip contains a CPU. The CPU contains a program for generating a management command queue. For each subtask, the CPU determines the associated data corresponding to that subtask during its data lifecycle, and determines the accelerator category within that subtask based on that associated data. Then, the CPU generates management commands for that subtask based on the category. The number of management commands is at least one, and the accelerator category indicates whether the accelerator consumes data, generates data, or both. Based on the dependencies between subtasks and each management command, the CPU generates a management command flowchart for the target task, and constructs a management command queue based on the flowchart and the management commands.

[0153] In an exemplary embodiment, the target task includes subtasks. Each subtask contains information about an execution module, which is also an accelerator, specifying which accelerator is needed to execute the subtask. For each subtask, the CPU determines the associated data within its data lifecycle and, based on this associated data, determines the accelerator category within that subtask. If the category is a first category, the CPU generates a first management command and a fourth management command corresponding to the subtask. If the category is a second category, the CPU generates a second management command and a third management command corresponding to the subtask. If the category is a third category, the CPU generates a first management command, a second management command, a third management command, and a fourth management command corresponding to the subtask. The CPU generates a management command flowchart for the target task based on the dependencies between subtasks and the management commands. Then, based on the management command flowchart, the CPU determines the number of preceding associated commands and subsequent management commands for each management command. The CPU constructs a management command queue based on each management command, the number of preceding associated commands, and the subsequent associated commands. The first management command is a memory data read command. The second management command is a subtask start command. The third management command is a cache line allocation command. The fourth management command is the subtask termination command.

[0154] In an optional embodiment, during the generation of management commands, if the accelerator category in the subtask is the first category, the CPU generates the first, fourth, and fifth management commands corresponding to that subtask. If the accelerator category in the subtask is the third category, the CPU generates the first, second, third, fourth, and fifth management commands corresponding to that subtask. The fifth management command is a data release command.

[0155] In this embodiment, a management command queue is generated in advance based on the target task and its data lifecycle information. This allows for fine-grained management of the target task's execution during subsequent execution.

[0156] In one exemplary embodiment, such as Figure 8 As shown, the specific processing procedure of step 704 includes steps 802 to 806. Wherein:

[0157] Step 802: For each subtask, determine the type of accelerator in the subtask based on the data lifecycle information.

[0158] The data lifecycle information includes the subtask relationships between each subtask.

[0159] In implementation, for each subtask, the chip determines the associated data corresponding to that subtask in the data lifecycle information, and determines the type of accelerator in the subtask based on the associated data.

[0160] Specifically, each subtask requires a corresponding accelerator for execution. Therefore, each subtask contains its corresponding accelerator. The CPU in the chip, for each subtask in the target task, determines the associated data corresponding to that subtask during its data lifecycle. If the associated data only contains dependent data, the CPU determines the accelerator category in the subtask to be Category 1. If the associated data only contains result data, the CPU determines the accelerator category in the subtask to be Category 2. If the associated data includes both dependent and result data, the CPU determines the accelerator category in the subtask to be Category 3.

[0161] In one exemplary embodiment, Figure 9 This is a schematic diagram illustrating the data lifecycle of a target task in one embodiment. For example... Figure 9 As shown, the target task includes subtasks: generating Data D, generating Data E, generating Data F, generating Data G, and processing Data F and Data G. Taking the subtask of generating Data D as an example, since the VCD inputs Data A and generates Data D, Data A is the dependent data and Data D is the result data in this subtask. The CPU determines the generator VCD in this subtask to be of type three. As for processing Data F and Data G, since the DPU inputs Data F and Data G but does not generate result data, Data F and Data G are the dependent data for this subtask. The CPU determines the DPU to be of type two.

[0162] Step 804: Generate the management commands corresponding to the subtasks according to the category.

[0163] In implementation, the chip generates management commands corresponding to each subtask based on the accelerator type. These management commands are used to manage the execution of the subtask and the storage of data within it.

[0164] Specifically, if the category is Category 1, the CPU in the chip generates a first management command and a fourth management command corresponding to the subtask. If the category is Category 2, the CPU generates a second management command and a third management command corresponding to the subtask. If the category is Category 3, the CPU generates a first management command, a second management command, a third management command, and a fourth management command corresponding to the subtask. The first management command retrieves the dependent data corresponding to the subtask from memory and stores the dependent data in the last-level cache module. The second management command starts the execution of the subtask. The third management command allocates cache lines to store the result data generated by the accelerator. The fourth management command instructs the accelerator to terminate the subtask.

[0165] In an optional embodiment, if the category is the first category, the CPU in the chip generates the first management command, the fourth management command, and the fifth management command corresponding to the subtask. If the category is the third category, the CPU generates the first management command, the second management command, the third management command, the fourth management command, and the fifth management command corresponding to the subtask.

[0166] In another optional embodiment, if the category is the first category and the dependent data required by the accelerator is in the last-level cache module, the CPU directly generates the fourth and fifth management commands corresponding to the subtask. If the category is the first category, the dependent data required by the accelerator is in the last-level cache module, and the result data generated by the accelerator will be continuously used in subsequent subtasks, the CPU directly generates the fourth management command corresponding to the subtask.

[0167] If the category is the fifth category, and the dependent data required by the accelerator is in the last-level cache module, then the CPU directly generates the second, third, fourth, and fifth management commands corresponding to that subtask. If the category is the third category, and the dependent data required by the accelerator is in the last-level cache module, and the result data generated by the accelerator will continue to be used in subsequent subtasks, then the CPU directly generates the second, third, and fourth management commands corresponding to that subtask.

[0168] Step 806: Construct the management command queue for the target task based on the subtask relationships of each subtask and each management command.

[0169] In implementation, for each management command, the chip determines its preceding and following associated commands based on subtask relationships and other management commands. Then, based on each management command and its corresponding preceding and following associated commands, the chip constructs a management command queue for the target task.

[0170] Specifically, the CPU in the chip generates a management command flowchart corresponding to the target task based on the dependencies and management commands of each subtask. Then, based on the management command flowchart, the CPU determines the number of preceding and subsequent associated commands for each management command, and constructs a management command queue according to each management command, the number of preceding and subsequent associated commands for each management command.

[0171] In this embodiment, management commands for each subtask are generated using associated data, enabling fine-grained data management for each subtask and improving the accuracy of data management. Furthermore, a management command queue is constructed based on these management commands, providing a command foundation for the subsequent sequential management of target task execution.

[0172] In one exemplary embodiment, such as Figure 10 As shown, the specific processing procedure of step 802 includes steps 1002 to 1008. Wherein:

[0173] Step 1002: In the data lifecycle information, identify the accelerator and associated data in the subtask.

[0174] In implementation, for each subtask in the target task, the chip determines the accelerator and associated data for that subtask within the data lifecycle information. The accelerator executes the subtask, and the associated data is data related to that subtask, including at least one of dependent or related data. The dependent data is the data input to the accelerator, which is also the data the accelerator needs to process. The result data is the data generated by the accelerator.

[0175] Specifically, the lifecycle of a cacheline needs to be specified by the software during the compilation phase. For example... Figure 9 As shown, when the CPU in the chip compiles the subtasks of each accelerator, it compiles the data dependencies between each accelerator into a data flow graph, and then the life cycle of each data can be obtained from the data flow graph. Figure 9 The target task includes subtasks: generating Data D, generating Data E, generating Data F, generating Data G, and processing Data F and Data G.

[0176] Taking the subtask of generating Data D as an example, the CPU determines the accelerator of this subtask as VCD in the data lifecycle information, and determines the associated data of this subtask as Data A and Data D. Among them, Data D is the result data, and Data A is the dependent data.

[0177] Taking the subtask of generating Data E as an example, the CPU determines the GPU as the accelerator for this subtask from the data lifecycle information, and identifies the associated data for this subtask as Data B, Data D, and Data E. Among them, Data E is the result data, and Data B and Data D are the dependent data.

[0178] Taking the subtask of Data F and Data G as an example, the CPU determines the accelerator of this subtask as the DPU from the data lifecycle information, and identifies the associated data of this subtask as Data F and Data G. Both Data F and Data G are dependent data.

[0179] Step 1004: If the associated data only contains dependent data, determine the accelerator category as the first category.

[0180] In practice, if the associated data only contains dependency data, then the chip accelerator is classified as Category 1. If compared to a food chain in nature, the accelerator is classified as a consumer.

[0181] Specifically, the dependent data is the data that needs to be processed by the accelerator, and it is also the data input to the accelerator. If the associated data only contains the data input to the accelerator and not the result data, the CPU in the chip determines the category of the accelerator in this subtask as the first category.

[0182] In one exemplary embodiment, with Figure 9 Taking the processing of the subtask Data F and Data G as an example, since this subtask only contains two dependent data, Data F and Data G, the CPU in the chip determines the DPU accelerator as the first category.

[0183] Step 1006: If the associated data only contains result data, then the accelerator category is determined to be the second category.

[0184] In practice, if the associated data only contains result data, the chip accelerator is classified as Category II. If compared to a food chain in nature, the accelerator is classified as a producer.

[0185] Specifically, the resulting data is the data generated by the accelerator. If the associated data includes the data generated by the accelerator but does not contain data that needs to be input to the accelerator, the CPU in the chip determines that the accelerator in this subtask belongs to the second category.

[0186] Step 1008: If the associated data contains result data and dependency data, determine the accelerator category as the third category.

[0187] In practice, if the associated data includes both result data and dependency data, the chip determines the accelerator category in that subtask to be the third category. If compared to a food chain in nature, the accelerator is both a producer and a consumer.

[0188] Specifically, the resulting data is the data generated by the accelerator. If the associated data includes the data generated by the accelerator but does not contain data that needs to be input to the accelerator, the CPU in the chip determines that the accelerator in this subtask belongs to the second category.

[0189] In one exemplary embodiment, with Figure 9 Taking the subtask of generating Data E as an example, since this subtask includes two dependent data sets, Data B and Data D, as well as the generated data set, Data D, the CPU in the chip classifies the GPU accelerator as a third-category device.

[0190] Taking the food chain in nature as an example, for Data A, VCD is the consumer, and the lifecycle of Cacheline A begins with Prefetch Data A and ends at #2; for Data F, NPU is the producer, GPU is the consumer, and the lifecycle of Cacheline F begins at #5 and ends at #8.

[0191] Optionally, the associated data corresponding to the subtask may include result data, dependent data, or both, depending on the needs of the subtask. This embodiment of the application does not impose any limitations on this.

[0192] In this embodiment, the category of each accelerator is determined by the associated data corresponding to each subtask, clarifying the role of the accelerator in the associated data and providing a data foundation for the subsequent generation of management commands corresponding to the subtasks.

[0193] In one exemplary embodiment, such as Figure 11 As shown, the specific processing procedure of step 804 includes steps 1102 to 1106. Wherein:

[0194] Step 1102: If the category is the first category, generate the first management command, the fourth management command, and the fifth management command corresponding to the subtask.

[0195] In implementation, if the category is the first category, the chip generates the first management command, the fourth management command, and the fifth management command corresponding to the subtask based on the data lifecycle information.

[0196] Specifically, if the category is Category 1, the CPU in the chip determines the dependent data corresponding to the subtask from the data lifecycle information and checks whether the dependent data is in the last-level cache module. If the dependent data is not in the last-level cache module, it indicates that the dependent data is in memory, and the CPU determines that the subtask is the target subtask. Then, the CPU generates a first management command corresponding to the target subtask, thereby retrieving the dependent data from memory and storing it in the last-level cache module before executing the target subtask. If the dependent data is in the last-level cache module, the CPU does not generate a first management command corresponding to the subtask. The CPU directly generates a fourth management command corresponding to the subtask. This fourth management command is used to instruct the accelerator to end the execution of the subtask. Then, the CPU determines whether the dependent data will be used in subsequent subtasks. If the dependent data will not be used in subsequent subtasks, the CPU generates a fifth management command corresponding to the subtask. If the dependent data will be used in subsequent subtasks, the CPU does not generate a fifth management command corresponding to the subtask.

[0197] In an optional embodiment, during actual operation, the accelerator may execute multiple processes in a time-sharing manner. This could result in the output data produced by a previous accelerator remaining unconsumed in the LLC module for an extended period. The LLC module's replacement algorithm might evict the cacheline corresponding to this output data back into memory. Subsequently, when the accelerator needs to consume this output data, it must wait for the LLC module to read the data back from memory. Therefore, when compiling the CME's management commands, a Prefetch command (i.e., the first management command) can be added after the flag at the start of each accelerator. This allows the CME to perform the Prefetch operation simultaneously with the start of the accelerator's operation.

[0198] Step 1104: If the category is the second category, generate the second management command and the third management command corresponding to the subtask.

[0199] In implementation, if the category is the second category, it indicates that the subtask will generate result data. The chip generates the second management command corresponding to the subtask. Then, the chip directly generates the third management command corresponding to the subtask, thereby allocating a cache line in the last-level cache module to store the result data.

[0200] Step 1106: If the category is the third category, generate the first management command, the second management command, the third management command, the fourth management command, and the fifth management command corresponding to the subtask.

[0201] In implementation, if the category is the third category, the chip generates the first management command, the second management command, the third management command, the fourth management command and the fifth management command corresponding to the subtask based on the data lifecycle information.

[0202] Specifically, if the category is the third category, the CPU in the chip determines the dependent data corresponding to the subtask from the data lifecycle information and checks whether the dependent data is in the last-level cache module. If the dependent data is not in the last-level cache module, it indicates that the dependent data is in memory, and the CPU determines that the subtask is the target subtask. Then, the CPU generates a first management command corresponding to the target subtask, thereby retrieving the dependent data from memory and storing it in the last-level cache module before executing the target subtask.

[0203] If the dependent data is in the last-level cache module, the CPU will not generate the first management command for that subtask. Instead, the CPU will directly generate the second, third, and fourth management commands for that subtask. The second management command initiates the execution of the subtask. The third management command allocates a cache line for the result data corresponding to the subtask in the last-level cache module. The fourth management command instructs the accelerator to terminate the execution of the subtask.

[0204] The CPU then determines whether the dependent data will be needed in subsequent subtasks. If the dependent data will not be used in subsequent subtasks, the CPU generates the fifth management command corresponding to that subtask. If the dependent data will be used in subsequent subtasks, the CPU does not generate the fifth management command corresponding to that subtask.

[0205] In this embodiment, management commands corresponding to each subtask are generated based on the type of accelerator in the subtask, realizing personalized generation of management commands. This provides a command basis for the execution of subsequent personalized management subtasks, refines the management granularity, and improves the accuracy of cache management.

[0206] In one exemplary embodiment, the data lifecycle information includes associated data of subtasks, and the associated data includes dependency data, such as... Figure 12 As shown, the specific processing procedure of step 1102 includes steps 1202 to 1206. Wherein:

[0207] Step 1202: If the category is the first category, generate the first management command based on the location of the dependent data.

[0208] The first management command is used to read the dependency data required by the subtask from memory into the last-level cache module.

[0209] In implementation, if the category is Category 1, the chip determines the location of the dependent data. If the dependent data is in memory, the chip determines this subtask as the target subtask and generates the first management command for that target subtask. If the dependent data is in the LLC module, the chip does not generate the first management command corresponding to that subtask. The first management command is Prefetch Data XX, where Data XX represents the dependent data that needs to be prefetched.

[0210] In one exemplary embodiment, the chip includes a CPU, and the CPU contains a program for generating management commands. Figure 9 Taking the subtask of generating Data D as an example, if the CPU determines that Data A is located in memory, then the CPU determines that generating Data D is the target subtask. Then, the CPU generates the first management command corresponding to this target subtask, namely PrefetchData A. This first management command is used to prefetch Data A from memory and store Data A in the LLC module.

[0211] by Figure 9 Taking the subtask of generating Data E as an example, the CPU determines that Data B is located in memory and Data D is located in the LLC module. Since the dependent data Data B is in memory, the CPU determines that generating Data E is the target subtask. Then, the CPU only generates the first management command for Data B, namely Prefetch Data B. This first management command is used to retrieve Data B from memory and store Data B in the LLC module. The dependent data Data D is not processed.

[0212] In an optional embodiment, to prevent the LLC module's replacement algorithm from releasing dependent data from the LLC module into memory, causing the accelerator in the subtask to retrieve the dependent data from memory, the CPU determines the subtask as the target subtask even if the dependent data is in the LLC module, and generates a first management command for that target subtask.

[0213] For example, with Figure 9Taking the subtask of generating Data E as an example, the CPU determines that Data B is located in memory and Data D is located in the LLC module. Although the dependent data Data D is in the last-level cache module, to prevent the LLC module's replacement algorithm from releasing the dependent data Data D from the LLC module to memory, the CPU needs to generate the first management commands for Data D and Data B. Specifically, the CPU generates the first management command for Data B, namely PrefetchData B, and simultaneously generates the first management command for Data D, namely PrefetchData D. PrefetchData B is used to retrieve Data B from memory and store it in the LLC module. PrefetchDataD is used to retrieve Data D from memory and store it in the LLC module.

[0214] Optionally, if there are multiple dependent data, the CPU generates a first management command for each dependent data.

[0215] Step 1204: Generate the fourth management command corresponding to the subtask.

[0216] The fourth management command is used to instruct the accelerator to end the execution of the subtask.

[0217] In implementation, the chip generates a fourth management command corresponding to the accelerator in the subtask, thereby managing the execution status of the subtask based on the fourth management command. The fourth management command is "Wait VCD end flag #N," where flag #N is the index of the consumption dependency data in the subtask.

[0218] In one exemplary embodiment, with Figure 9 Taking the subtask of generating Data F as an example, the CPU in the chip generates the fourth management command corresponding to this subtask, namely Wait NPU end flag #6, based on the NPU accelerator in this subtask. Wait NPU end flag #6 is used to terminate the execution of the subtask by the accelerator. Specifically, the CME module in the chip determines whether the NPU has completed flag #6. If the NPU has completed flag #6, the CME module instructs the NPU to terminate the execution of the Data F subtask.

[0219] Step 1206: Generate the fifth management command for dependent data.

[0220] The fifth management command is used to release the dependency data in the last-level cache into memory.

[0221] In implementation, for each piece of dependent data, the chip generates a fifth management command based on the relationship between that dependent data and subsequent subtasks. This fifth management command is `Flush Cacheline XX`, where `XX` represents the dependent data that needs to be released.

[0222] Specifically, for each piece of dependent data, the chip determines whether that dependent data will be used in subsequent subtasks. If the dependent data will be used in subsequent subtasks, the chip does not generate the fifth management command corresponding to that dependent data. If the dependent data will be used in subsequent subtasks, the chip generates the fifth management command corresponding to that dependency. This fifth management command allows unused dependent data to be released to memory.

[0223] In one exemplary embodiment, with Figure 9 Taking the subtask of generating Data F as an example, it depends on Data D and Data C. The CPU in the chip determines that Data D will be used in subsequent subtasks; therefore, the CPU does not generate the fifth management command corresponding to Data D, meaning it stores Data D in the last-level cache module. The CPU determines that Data C will not be used in subsequent subtasks; therefore, the CPU generates the fifth management command corresponding to Data C, namely Flush Cacheline C. This fifth management command stores Data C in memory.

[0224] In this embodiment, when the accelerator is classified as Category 1, meaning it consumes dependent data, a first management command corresponding to the target subtask is generated. This allows the dependent data corresponding to the target subtask to be prefetched into the last-level cache module before subsequent processing of the target subtask, ensuring a near 100% hit rate for the accelerator's LLC reads, thereby improving the execution speed of the target task. Furthermore, by generating a fifth management command corresponding to the target subtask, the LLC cacheline can be released promptly when the accelerator no longer needs the data in the LLC, improving LLC utilization.

[0225] In one exemplary embodiment, the data lifecycle information includes associated data of subtasks, and the associated data includes result data, such as... Figure 13 As shown, the specific processing procedure of step 1104 includes steps 1302 to 1304. Wherein:

[0226] Step 1302: If the category is the second category, generate the third management command corresponding to the result data.

[0227] The third management command is used to pre-allocate cache lines for the result data in the last-level cache.

[0228] In implementation, if the category is Category 2, the chip directly generates the third management command corresponding to the result data, thereby allocating a storage address for the result data in the last-level cache module (also known as the last-level swap cache). The third management command is `Allocate Cacheline XX`, where XX represents the result data.

[0229] In one exemplary embodiment, with Figure 9 Taking the generation of Data E as an example, the CPU in the chip determines that the result data in this subtask is Data E, and generates the third management command corresponding to the result data Data E, namely AllocateCacheline E. This third management command allocates a cache line Cacheline E to Data E.

[0230] In an optional embodiment, if the category is the second category and there are multiple result data, the CPU generates a third management command corresponding to each result data.

[0231] Step 1304: Generate the second management command corresponding to the subtask.

[0232] The second management command is used to start the accelerator to execute subtasks.

[0233] In implementation, the chip generates a second management command corresponding to the accelerator in the subtask. This second management command enables the accelerator to execute the subtask. Specifically, the second management command is `Wait GPU startflag #N`, where `N` is the index of the generated result data in the subtask.

[0234] In one exemplary embodiment, with Figure 9 Taking the generation of Data E as an example, the CPU in the chip generates the second management command corresponding to the subtask using the GPU corresponding to that subtask, namely Wait GPU start flag #3.

[0235] In one exemplary embodiment, with Figure 9Taking the target task as an example, the subtasks within this target subtask are generating Data D, generating Data E, generating Data F, generating Data G, and processing Data F and Data G. Based on the data lifecycle, the CPU in the chip compiles the command flow graph for the CME module. If the data consumed by the accelerator exists in memory before it starts running, a Prefetch command is generated, such as Prefetch Data A. After the accelerator starts running, a Cacheline is allocated in the Cache to cache the generated data, such as Allocate Cacheline D. When the accelerator finishes consuming the corresponding data, the consumed data Cacheline is flushed back to memory, such as Flush Cacheline A. The CME module starts executing commands, which depend on some accelerator start or end flags, such as Wait VCD endflag #2 and Wait VCD start flag #1.

[0236] For the subtask of generating Data D, the generated management commands are Prefetch Data A (first management command, prefetch data A), Wait VCD start flag #1 (second management command, wait for video codec start flag #1), Allocate Cacheline D (third management command, pre-allocate cacheline D), Wait VCDend flag #2 (fourth management command, wait for video codec end flag #2), and Flush Cacheline A (fifth management command, flush cacheline A).

[0237] For the subtask of generating Data E, the generated management commands are Prefetch Data B (first management command, prefetch data B), Wait GPU start flag #3 (second management command, wait for GPU start flag #3), Allocate Cacheline E (third management command, pre-allocate cacheline E), Wait GPU end flag #4 (fourth management command, wait for GPU end flag #4), and Flush Cacheline B (fifth management command, flush cacheline B).

[0238] For the subtask of generating Data D, the generated management commands are Prefetch Data C (first management command, prefetch data C), Wait NPU start flag #5 (second management command, wait for the neural network processing unit to start flag #5), Allocate Cacheline F (third management command, pre-allocate cacheline F), Wait NPU endflag #6 (fourth management command, wait for the neural network processing unit to end flag #6), and Flush Cacheline C (fifth management command, flush cacheline C).

[0239] For the subtask of generating Data G, the generated management commands are Wait GPU startflag #7 (second management command, waiting for the graphics processor start flag #7), Allocate Cacheline G (third management command, pre-allocating cache line G), Wait GPU end flag #8 (fourth management command, waiting for the graphics processor end flag #8), Flush Cacheline E (fifth management command, flushing cache line E), and Flush Cacheline F (fifth management command, flushing cache line F).

[0240] For processing the subtasks Data F and Data G, the generated management commands are Wait DPUend flag #10 (fourth management command, waiting for the end flag of the display processing unit #10), Flush Cacheline D (fifth management command, flushing cacheline D), and Flush Cacheline D (fifth management command, flushing cacheline D).

[0241] In an optional embodiment, with Figure 9Taking the target task as an example, the subtasks within this target subtask are generating Data D, generating Data E, generating Data F, generating Data G, and processing Data F and Data G. In actual operation, the accelerator may execute multiple processes in a time-sharing manner. This may cause data produced by a previous accelerator to remain unconsumed in the LLC for an extended period, and the LLC's replacement algorithm may evict this cacheline back to memory. Subsequently, when the accelerator needs to consume this data, it must wait for the LLC to read the data back from memory. Therefore, when compiling the CME command, a Prefetch command can be added after the flag at the start of each accelerator. This allows the CME to perform the Prefetch operation simultaneously with the start of the accelerator's operation. Therefore, for each subtask, if there is dependent data related to that subtask, a first management command is generated for that dependent data. Specifically:

[0242] For the subtask of generating Data D, the generated management commands are Prefetch Data A (first management command, prefetch data A), Wait VCD start flag #1 (second management command, wait for video codec start flag #1), Prefetch Data A Allocate Cacheline D (first and third management commands, prefetch data A and pre-allocate cacheline D), Wait VCD end flag #2 (fourth management command, wait for video codec end flag #2), and Flush Cacheline A (fifth management command, flush cacheline A).

[0243] For the subtask of generating Data E, the generated management commands are Prefetch Data B (first management command, prefetch data B), Wait GPU start flag #3 (second management command, wait for GPU start flag #3), Prefetch Data B / D Allocate Cacheline E (first and third management commands, prefetch data B and pre-allocate cacheline E), Wait GPU end flag #4 (fourth management command, wait for GPU end flag #4), and Flush Cacheline B (fifth management command, flush cacheline B).

[0244] For the subtask of generating Data D, the generated management commands are Prefetch Data C (first management command, prefetch data C), Wait NPU start flag #5 (second management command, wait for the neural network processing unit start flag #5), Prefetch Data C / D Allocate Cacheline F (first and third management commands, prefetch data C and pre-allocate cacheline F), Wait NPU end flag #6 (fourth management command, wait for the neural network processing unit end flag #6), and Flush Cacheline C (fifth management command, flush cacheline C).

[0245] For the subtask of generating Data G, the generated management commands are Wait GPU startflag #7 (second management command, waiting for the graphics processor to start flag #7), Prefetch Data E / F AllocateCacheline G (first and third management commands, prefetching data E / F and pre-allocating cache line G), Wait GPUend flag #8 (fourth management command), Flush Cacheline E (fifth management command, flushing cache line E), and Flush Cacheline F (fifth management command, flushing cache line F).

[0246] For processing the subtasks Data F and Data G, the generated management commands are Wait DPUend flag #10 (fourth management command, waiting for the end flag of the display processing unit #10), Flush Cacheline D (fifth management command, flushing cacheline D), and Flush Cacheline D (fifth management command, flushing cacheline D).

[0247] In this embodiment, when the accelerator of the subtask is of the second category, that is, when the accelerator generates result data, the generation of the third management command can ensure that the result data is stored in the LLC, thus avoiding the loss of result data.

[0248] In one exemplary embodiment, such as Figure 14 As shown, the specific processing procedure of step 806 includes steps 1402 to 1406. Wherein:

[0249] Step 1402: Generate a flowchart of management commands for the target task based on the dependencies of each subtask and each management command.

[0250] The management command flowchart includes the execution order of each management command and the command relationships between them.

[0251] In implementation, the chip generates a management command flowchart for the target task based on the dependencies between subtasks, the execution order of management commands in the subtasks, and each management command.

[0252] Specifically, for a subtask, dependent data should first be stored in the last-level cache module before execution. Then, if the subtask generates results, cache lines are allocated for the generated data. Upon completion of the subtask, cached data that is no longer needed is released back into memory. Therefore, within a subtask, the execution order of management commands, from first to last, is: first management command, second management command, third management command, fourth management command, and fifth management command. The CPU in the chip then sets the execution order of the management commands within each subtask and the command dependencies between them. Finally, based on the management commands of each subtask and the dependencies between subtasks, the CPU generates a management command flowchart for the target task.

[0253] In one exemplary embodiment, Figure 15 This is a flowchart of the management commands for a target task in one embodiment. For example... Figure 15 As shown, in the subtask of generating Data D, the CPU determines the execution order and command associations between the various management commands. The order of the management commands from beginning to end is: Prefetch Data A, Wait VCD start flag #1, Allocate Cacheline D, Wait VCD end flag #2, and Flush Cacheline A. For the management command Prefetch Data A, its associated management commands are Wait VCD end flag #2 and Wait VCD start flag #1. For the management command Wait VCD end flag #2, its associated management commands are Flush Cacheline A and Wait VCD end flag #2. Then, based on the dependencies between subtasks, the CPU associates the management commands of different subtasks. Then, the CPU generates management command relationships according to the execution order and command associations between the management commands. For example, for the management command Wait VCD start flag #1, it is associated with the management commands Prefetch Data B and Prefetch Data C.

[0254] In one exemplary embodiment, Figure 16 This is a flowchart of the management commands for the target task in another embodiment. Figure 16 As shown, in the subtask of generating Data E, the CPU determines the execution order and command associations between the various management commands. The order of the management commands from beginning to end is: Prefetch Data B, Wait GPU start flag #3, Prefetch Data B / D Allocate Cacheline E, Wait GPU end flag #4, and Flush Cacheline B. For the management command Prefetch Data B, its associated management commands are Wait GPU end flag #4 and Wait GPU start flag #3. For the management command Wait GPU end flag #4, its associated management commands are Flush Cacheline B and Prefetch Data B. Then, based on the dependencies between subtasks, the CPU associates the management commands of different subtasks. Then, the CPU generates management command relationships according to the execution order and command associations between the management commands. For example, for the management command Prefetch Data B / D Allocate Cacheline E, this management command is associated with the management commands Wait GPU start flag #7 and Wait GPU end flag #8. The command association between the two management commands means that the second management command must be executed after the first management command has been completed.

[0255] Step 1404: For each management command, based on the command association relationship, determine the number of preceding associated commands of the management command, and determine the subsequent associated commands of the management command.

[0256] In implementation, for each management command, the chip determines the preceding management commands and their associated pre-command commands based on command relationships. The chip then counts these pre-command commands to obtain their number. Within each management command, the chip determines the management commands whose execution order is after the current management command and whose associated commands are considered subsequent associated commands. The number of pre-command commands for each management command is used to construct the execution conditions for that command. These execution conditions are used to determine whether to execute the management command.

[0257] Step 1406: Construct a management command queue based on each management command, the number of preceding associated commands, and subsequent associated commands.

[0258] In implementation, the chip generates command identifiers for each management command according to the execution order of the commands. Then, the chip constructs a management command queue based on each management command, its command identifier, the number of preceding associated commands, and subsequent associated commands.

[0259] Specifically, the CPU in the chip sequentially generates command identifiers for each management command according to the execution order. Then, for each management command, the CPU generates an entry for that management command based on its command identifier, the command itself, the number of preceding and subsequent related commands. Finally, the CPU constructs a management command queue for the target task based on the entries in the management command queue and the execution order of the management commands.

[0260] In one exemplary embodiment, Figure 17 This is a schematic diagram illustrating the management of a command queue in one embodiment. For example... Figure 17 As shown, the target task contains five management commands: Prefetch Data A (the first management command), WaitVCD end flag #2 (the fourth management command), Flush Cacheline A (the fifth management command), Wait VCD startflag #1 (the second management command), and Prefetch Data A (the second management command). For the first management command, the CPU determines that its preceding associated commands are empty and that the number of preceding associated commands is 0. Then, the CPU determines that the subsequent associated commands for this first management command are Wait VCD endflag #2 and Wait VCD start flag #1. Based on the command identifier of the first management command, the first management command itself, the number of preceding associated commands, and the command identifiers of the subsequent associated commands, the CPU constructs an entry for the first management command. Following the above execution process, the CPU generates entries for Wait VCD end flag #2, Flush Cacheline A, Wait VCD start flag #1, and the second management command. Then, the CPU combines the entries in the table according to the order of the management commands to generate the management command queue for the target task.

[0261] In this embodiment, a management command queue for the target task is constructed based on the dependencies between subtasks and various management commands. This management command queue enables the pre-fetching and release of cachelines and the lifecycle management of cachelines based on the data executed by the accelerator.

[0262] In an exemplary embodiment, the management command queue includes target management commands of target type and non-target management commands of non-target type; the target management commands include a second management command and a fourth management command; the non-target management commands include a first management command, a second management command, and a fifth management command.

[0263] Among them, target-type management commands are task-type target management commands used to manage the execution status of subtasks. Non-target-type management commands are cache-type non-target management commands used to manage associated data of subtasks.

[0264] In implementation, management commands are divided into target-type target management commands and non-target-type non-target management commands. Target-type target management commands are task-type task management commands used to start or stop the execution of subtasks. Therefore, target management commands include a second management command and a fourth management command. The second management command instructs the accelerator to start executing a subtask, and the fourth management command instructs the accelerator to stop executing a subtask. Non-target-type non-target management commands are cache-type cache management commands used to manage the associated data of subtasks. Non-target-type management commands include a first management command, a third management command, and a fifth management command. The first management command prefetches the dependency data required by the subtask from memory and stores it in the last-level cache module. The third management command allocates storage space (i.e., cache lines) for the result data corresponding to the subtask in the last-level cache module. The fifth management command releases the dependency data required by the subtask from the last-level cache module into memory.

[0265] Optionally, the management of the command queue can be implemented using, but is not limited to, linked lists, command queues, or ring buffers.

[0266] In this embodiment, by determining the type of management command, it is easier to manage the execution of the target task from both data caching and task perspectives, thereby refining the management granularity and improving management accuracy.

[0267] In an exemplary embodiment, after forming the management command queue, it is also necessary to generate a target subtask execution identifier table corresponding to the target management command, thereby managing the execution status of each target management command through the target subtask execution identifier table. For example... Figure 18 As shown, after step 602 is executed, the specific processing procedure of this cache management method includes steps 1802 to 1804. Wherein:

[0268] Step 1802: Determine the target management commands of each target type in the management command queue, and generate the execution identifier of each target management command.

[0269] In implementation, the chip filters target management commands of target type from the management command queue. Then, for each target management command, the chip generates an execution identifier corresponding to that target management command. The execution identifier indicates whether the target management command was successfully executed. When generating execution identifiers, each target management command's execution identifier is an identifier indicating that the target management command was not successfully executed, i.e., a failure execution identifier.

[0270] Specifically, the target management commands include second and fourth management commands. 0 indicates that the target management command was not executed successfully. 1 indicates that the target management command was executed successfully. The CPU in the chip determines each second and fourth management command from the management command queue. For each second management command, the CPU generates an execution flag corresponding to that second management command and sets the value of the execution flag to 0. Then, for each fourth management command, the CPU generates an execution flag corresponding to that fourth management command and sets the value of the execution flag to 0.

[0271] Step 1804: Construct a subtask management table for the target task based on each execution identifier and each target management command.

[0272] In implementation, the chip constructs a subtask management table for the target task based on the command index of each target management command and each execution identifier.

[0273] In one exemplary embodiment, Figure 19 This is a schematic diagram of a subtask management table in one embodiment. For example... Figure 19 As shown, Event ID is the command index of the target management command. Flag is the execution identifier. For the target management command with command index 2, its corresponding execution identifier is 0, indicating that the target management command did not execute successfully. The target management command with command index 0 is Wait NPU start flag #5. For this target management command, the execution identifier is 0 at the beginning of its generation, indicating that the target management command did not execute successfully.

[0274] In this embodiment, by generating a subtask management table, a data foundation is provided for the subsequent execution of target management commands, ensuring the sequential execution of target management commands, further ensuring the sequential execution of each subtask in the target task, and improving the execution success rate.

[0275] In an exemplary embodiment, after initiating the execution of the target subtask, a third management command, a fourth management command, and a fifth management command may be executed during the execution of subsequent target subtasks or other subtasks. Therefore, after step 604 is executed, the specific processing procedure of the cache management method includes:

[0276] Based on the generation probability of each management command, execute at least one of the third, fourth, or fifth management commands.

[0277] The generation probability is determined based on the requirements of the target task.

[0278] In practice, the chip executes at least one of the third, fourth, or fifth management commands according to the generation probability of each management command, thereby completing the execution of the target sub-task.

[0279] Specifically, the chip includes a CME module. For the target subtask, since the management commands corresponding to the target subtask include a first management command and a second management command, indicating that the accelerator in the target subtask belongs to the third category, the management commands corresponding to this target subtask also include a third management command and a fourth management command. Therefore, for the target subtask, the generation probability of the third and fourth management commands is 100% (percentage). The CME module executes the third and fourth management commands and executes the fifth management command according to the third generation probability. The third generation probability is determined based on the dependency data of the target subtask and the relationship with subsequent subtasks.

[0280] In this embodiment, the sequential execution of each management command is ensured by generating the management command based on its probability, thereby improving the success rate of the target sub-tasks.

[0281] In an exemplary embodiment, the generation probabilities include a first generation probability, a second generation probability, and a third generation probability, such as... Figure 20 As shown, the specific processing procedure for executing at least one of the third, fourth, or fifth management commands based on the generation probability of each management command includes steps 2002 to 2006. Wherein:

[0282] Step 2002: Based on the first generation probability, execute the third management command, and in response to the third management command, pre-allocate cache lines for the result data of the target subtask in the last-level cache module.

[0283] The cache line is used to store the result data.

[0284] In implementation, the chip executes a third management command based on the first generation probability. The final-level cache module in the chip responds to the third management command by pre-allocating a cache line for the result data generated by the target subtask. This cache line is used to store the result data.

[0285] Specifically, for the target subtask, the first generation probability is 100%. Since the management commands corresponding to the target subtask include a first management command and a second management command, the accelerator category in this target subtask is the third category. Therefore, it can be determined that the management commands corresponding to this target subtask include at least a first management command, a second management command, a third management command, and a fourth management command. The CME module in the chip executes the third management command based on the first generation probability. The last-level cache module in the chip responds to the third management command by pre-allocating cache lines for the result data generated by the target subtask.

[0286] In an optional embodiment, for non-target subtasks, the first generation probability is determined based on whether the non-target subtask will generate result data. If result data needs to be generated, the CPU generates a third management command. If result data does not need to be generated, the CPU does not generate a third management command. If a third management command is generated, the CME module executes it. The last-level cache module in the chip responds to the third management command by pre-allocating cache lines for the result data generated by the non-target subtask. After the result data is generated, the accelerator stores the result data in the cache lines.

[0287] Step 2004: Based on the second generation probability, execute the fourth management command, and in response to the fourth management command, terminate the execution of the target subtask.

[0288] In practice, the chip executes the fourth management command based on the second generation probability. The accelerator in the target subtask responds to the fourth management command and terminates the execution of that subtask.

[0289] Specifically, for the target subtask, the second generation probability is 100%. Since the management commands corresponding to the target subtask include a first management command and a second management command, the accelerator in this target subtask is classified as a third category. Therefore, it can be determined that the management commands corresponding to this target subtask include at least a first management command, a second management command, a third management command, and a fourth management command. The CME module in the chip executes the fourth management command based on the second generation probability. The accelerator in the target subtask responds to the fourth management command and terminates the execution of the target subtask.

[0290] In an optional embodiment, for non-target subtasks, the second generation probability is determined based on whether the non-target subtask processes dependent data. If dependent data processing is required, the CPU generates a fourth management command. If dependent data processing is not required, the CPU does not generate a fourth management command. If the CPU generates a fourth management command, the CME module executes it. The accelerator in the non-target subtask responds to the fourth management command and stops executing the non-target subtask.

[0291] Step 2006: Based on the third generation probability, execute the fifth management command, and in response to the fifth management command, release the dependent data in the last-level cache into memory.

[0292] In implementation, the chip executes the fifth management command based on the third generation probability. The accelerator in the target subtask responds to the fourth management command by releasing the dependent data from the last-level cache back into memory.

[0293] Specifically, for the target subtask, the third generation probability is determined based on whether subsequent subtasks will use the dependency data required by the target subtask. If subsequent subtasks will use the dependency data required by the target subtask, the CPU in the chip will not generate the fifth management command corresponding to that dependency data. If subsequent subtasks will not use the dependency data required by the target subtask, the CPU will generate the fifth management command corresponding to that dependency data. The CME module executes the fifth management command based on the third generation probability. The accelerator in the target subtask responds to the fifth management command by releasing the cache line corresponding to the dependency data in the last-level cache, thereby saving the dependency data to memory.

[0294] In an optional embodiment, for non-target subtasks, the third generation probability is determined based on whether the non-target subtask processes dependent data and whether subsequent subtasks of the non-target subtask will use that dependent data. If the non-target subtask does not need to process dependent data, the CPU does not generate the fifth management command. If the non-target subtask needs to process dependent data, the CPU determines whether subsequent subtasks will use the dependent data required by the non-target subtask. If subsequent subtasks will use the dependent data required by the non-target subtask, the CPU does not generate the fifth management command corresponding to that dependent data. If subsequent subtasks will not use the dependent data required by the non-target subtask, the CPU generates the fifth management command corresponding to that dependent data. The CME module executes the fifth management command according to the third generation probability. The accelerator in the target subtask responds to the fifth management command by releasing the cache line corresponding to the dependent data in the last-level cache, thereby saving the dependent data to memory.

[0295] In this embodiment, the third management command ensures that the result data is stored in the LLC in a timely manner, preventing data loss. By executing the fifth management command, the LLC cacheline can be released promptly when the accelerator no longer needs the data in the LLC, improving the utilization rate of the LLC.

[0296] In an exemplary embodiment, during the execution of each management command based on the management command queue, it is also necessary to determine whether the current management command meets the execution conditions. Therefore, as... Figure 21As shown, the specific processing steps of this cache management method further include steps 2102 to 2104. Wherein:

[0297] Step 2102: Obtain the current management command from the management command queue and determine the number of preceding associated commands corresponding to the current management command.

[0298] The current management command is any one of the following: the first management command, the second management command, the third management command, the fourth management command, and the fifth management command.

[0299] In practice, the chip retrieves the current management command in the order of the management command queue and determines the number of preceding associated commands corresponding to the current management command.

[0300] Step 2104: Determine whether the current management command meets the execution conditions based on the number of preceding associated commands.

[0301] The execution condition is used to determine whether to respond to the current management command. The execution condition is that the number of preceding commands is 0.

[0302] During implementation, the chip determines whether the number of preceding associated commands is 0. If the number of preceding associated commands is 0, the chip determines that the current management command meets the execution conditions and executes the current management command. If the number of preceding associated commands is not 0, the chip determines that the current management command does not meet the execution conditions.

[0303] In one exemplary embodiment, the management command queue contains entries for each management command. For example... Figure 17 As shown, each entry contains four pieces of information: Cmd ID (command identifier), Previous Counter (number of preceding related commands), Subsequent List (number of subsequent related commands), and Cmd (management command). The Cmd ID identifies the entry. Previous Counter indicates how many preceding entries the current entry depends on; when Previous Counter is 0, it means the current entry can be executed. Subsequent List represents the IDs of subsequent entries that depend on the current entry; after CME executes each entry, it decrements the Previous Counter by 1 based on the IDs listed in Subsequent List. Cmd is the command that CME needs to execute.

[0304] The CME module in the chip retrieves the current management command in the order of the management command queue. After executing the current management command, the CME module determines the next management command to be executed based on its subsequent associated commands and updates the current management command accordingly. Once the CME module determines the current management command, it checks the number of preceding associated commands in the corresponding table. The CME module then checks if the number of preceding associated commands is 0. If it is, the CME module executes the current management command. If it is not 0, the CME module continues executing the preceding associated commands until the number of preceding associated commands is 0, at which point it executes the current management command. After executing the current management command, the CME module decrements the PreviousCounter value in the table of subsequent associated commands by 1.

[0305] In this embodiment, the number of preceding associated commands is used as the execution condition to determine whether the current management command can be executed. If it can be executed, the current management command is executed, which ensures the sequential execution of each management command and improves the accuracy of management command execution.

[0306] In an exemplary embodiment, during the execution of target management commands, it is also necessary to manage the target management commands through a subtask management table. Therefore, as... Figure 22 As shown, the specific processing steps of this cache management method further include steps 2202 to 2206. Wherein:

[0307] Step 2202: Determine whether the target management command was executed successfully.

[0308] During implementation, if the management command is a target management command, the chip determines whether the target management command was executed successfully. If the target management command was executed successfully, the chip proceeds to step 2204 below. If the target management command failed to execute, the chip proceeds to step 2206 below.

[0309] Step 2204: If the target management command is executed successfully, update the execution flag corresponding to the target management command in the subtask management table to the successful execution flag.

[0310] The successful execution flag is used to instruct the cache management engine module in the chip to execute the next management command.

[0311] In practice, if the target management command is executed successfully, the chip updates the execution flag corresponding to the management command in the subtask management table to the successful execution flag and determines that the next management command can be executed.

[0312] Specifically, an execution flag value of 0 indicates a failed execution, while an execution flag value of 1 indicates a successful execution. If the target management command is executed successfully, the CME module in the chip updates the execution flag value corresponding to the management command to 1 in the subtask management table and continues to execute the next management command in the target management command queue.

[0313] Step 2206: If the target management command is executed successfully, update the execution flag corresponding to the target management command in the subtask management table to the failure execution flag.

[0314] The failure execution flag is used to instruct the cache management engine module in the chip to continue executing the target management command until the execution flag becomes the success execution flag.

[0315] In practice, if the target management command fails to execute, the chip updates the execution flag of the management command in the subtask management table to the failure execution flag, and continues to execute the target management command until the target management command is executed successfully, at which point the execution flag of the target management command changes to the success execution flag.

[0316] Specifically, an execution flag value of 0 indicates a failed execution, while an execution flag value of 1 indicates a successful execution. If the target management command fails to execute, the CME module in the chip updates the execution flag value corresponding to the management command to 0 in the subtask management table and continues to execute the target management command until it succeeds, at which point the execution flag value of the target management command becomes 1.

[0317] In one exemplary embodiment, such as Figure 19 As shown, CME synchronizes via an Event table (subtask management table). Each entry in the Event table contains a 1-bit flag, and the Event is the index of that entry. Taking the NPU as an example, when the NPU starts working (i.e., successfully executes the target management command (i.e., the second management command)), it writes 1 to the flag with Event id (command index) 0. When the CME executes the corresponding Wait command, it polls the flag with Event id 0. When polling reaches a flag with Event id 0 that is 1, it indicates that the Wait command was executed successfully, and the CME can continue executing other commands after the Wait command. If polling reaches a flag with Event id 0 that is 0, it indicates that the Wait command failed, and the Wait command will block subsequent commands until it succeeds.

[0318] In this embodiment, the sequential execution of each subtask is managed by a subtask management table, which ensures the sequential execution of each subtask and improves the success rate of subtask execution.

[0319] In one exemplary embodiment, if the accelerator's task supports data parallel switching, then the CME module in the chip can implement fine-grained cacheline lifecycle management. Figure 23 This is a schematic diagram illustrating fine-grained lifecycle management in one embodiment. For example... Figure 23 As shown, the neural network processing unit consumes data A and produces data B. Data A can be switched to smaller granular data A0, A1, A2, and A3, and data B can also be switched to smaller granular data B0, B1, B2, and B3. When the neural network processing unit is working, it can first consume A0 and produce B0, and after execution, it can consume A1 and produce B1, and so on. The CME can then manage the cacheline according to the granularity of A0, A1, A2, A3 and B0, B1, B2, and B3. That is, the CME module can release the cacheline A0 after A0 is consumed. In the non-splitting scheme, the CME module needs to wait for the complete A to be consumed before it can release the cacheline A.

[0320] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps. It is understood that the steps in different embodiments can be freely combined as needed, and all non-contradictory solutions formed by such combinations are within the scope of protection of this application.

[0321] Based on the same inventive concept, this application also provides a cache management device for implementing the cache management method described above. The solution provided by this device is similar to the implementation described in the above method; therefore, the specific limitations in one or more cache management device embodiments provided below can be found in the limitations of the cache management method described above, and will not be repeated here.

[0322] In one exemplary embodiment, such as Figure 24 As shown, a cache management device 2400 is provided, including: an acquisition module 2401, a reading module 2402, and an execution module 2403, wherein:

[0323] The acquisition module 2401 is used to acquire the target task and the management command queue; the management command queue contains various management commands for the target task.

[0324] The reading module 2402 is used to read the dependency data required by the target subtask from memory and store the dependency data in the last-level cache module in response to the first management command when the last-level cache module does not have the dependency data required by the target subtask in the target task.

[0325] Execution module 2403 is used to initiate the execution of a target subtask in response to a second management command; the target subtask includes the accelerator reading dependent data from the last-level cache module.

[0326] In one exemplary embodiment, the acquisition module 2401 includes:

[0327] The first acquisition submodule is used to acquire the target task and the corresponding data lifecycle information; the target task includes various subtasks.

[0328] The first generation submodule is used to generate management commands corresponding to each subtask based on data lifecycle information, and to build a management command queue based on each management command.

[0329] In one exemplary embodiment, the first generation submodule includes:

[0330] The first determination submodule is used to determine the type of accelerator in each subtask based on the data lifecycle information; the data lifecycle information includes the subtask association relationships between each subtask.

[0331] The second generation submodule is used to generate management commands corresponding to subtasks based on the category.

[0332] The first construction submodule is used to construct the management command queue of the target task based on the subtask relationships of each subtask and each management command.

[0333] In an exemplary embodiment, the first determining submodule is specifically configured to: determine the accelerator and associated data in the subtask from the data lifecycle information; if the associated data contains only dependent data, determine the accelerator category as a first category; if the associated data contains only result data, determine the accelerator category as a second category; if the associated data contains both result data and dependent data, determine the accelerator category as a third category.

[0334] In one exemplary embodiment, the second generation submodule includes:

[0335] The third generation submodule is used to generate the first, fourth, and fifth management commands corresponding to the subtask if the category is the first category.

[0336] The fourth generation submodule is used to generate the second and third management commands corresponding to the subtask if the category is the second category.

[0337] The fifth generation submodule is used to generate the first, second, third, fourth, and fifth management commands corresponding to the subtask if the category is the third category.

[0338] In one exemplary embodiment, the data lifecycle information includes associated data of subtasks, which includes dependency data. The third generation submodule is specifically used to: if the category is a first category, generate a first management command based on the location of the dependency data; the first management command is used to read the dependency data required by the subtask from memory to the last-level cache module; generate a fourth management command corresponding to the subtask; the fourth management command is used to instruct the accelerator to end the execution of the subtask; generate a fifth management command for the dependency data; and the fifth management command is used to release the dependency data in the last-level cache to memory.

[0339] In an exemplary embodiment, the data lifecycle information includes associated data of the subtask, which includes result data. The fourth generation submodule is specifically used to: if the category is the second category, generate a third management command corresponding to the result data; the third management command is used to pre-allocate cache lines for the result data in the last-level cache; generate a second management command corresponding to the subtask; and the second management command is used to start the accelerator to execute the subtask.

[0340] In an exemplary embodiment, the first construction submodule is specifically configured to: generate a management command flowchart for the target task based on the dependencies of each subtask and each management command; the management command flowchart includes the execution order of each management command and the command association relationship between each management command; for each management command, determine the number of preceding associated commands based on the command association relationship, and determine the subsequent associated commands of the management command; and construct a management command queue based on each management command and the number of preceding associated commands and subsequent associated commands of each management command.

[0341] In one exemplary embodiment, the management command queue includes target management commands of target type and non-target management commands of non-target type; the target management commands include a second management command and a fourth management command; the non-target management commands include a first management command, a second management command, and a fifth management command.

[0342] In an exemplary embodiment, the cache management device 2400 is further configured to: determine each target management command of a target type in the management command queue, and generate an execution identifier for each target management command; and construct a subtask management table for the target task based on each execution identifier and each target management command.

[0343] In one exemplary embodiment, the cache management device 2400 further includes:

[0344] The execution module is used to execute at least one of the third, fourth, or fifth management commands based on the generation probability of each management command.

[0345] In an exemplary embodiment, the generation probability includes a first generation probability, a second generation probability, and a third generation probability. The execution module is specifically configured to: execute a third management command based on the first generation probability, and in response to the third management command, pre-allocate cache lines in the last-level cache module for the result data of the target subtask; the cache lines are used to store the result data; execute a fourth management command based on the second generation probability, and in response to the fourth management command, terminate the execution of the target subtask; and execute a fifth management command based on the third generation probability, and in response to the fifth management command, release the dependent data in the last-level cache to memory.

[0346] In an exemplary embodiment, the cache management device 2400 is further configured to: obtain a current management command from a management command queue and determine the number of preceding associated commands corresponding to the current management command; the current management command is any one of the first management command, the second management command, the third management command, the fourth management command, and the fifth management command; determine whether the current management command meets the execution conditions based on the number of preceding associated commands; the execution conditions are used to determine whether to respond to the current management command.

[0347] In an exemplary embodiment, the cache management device 2400 is further configured to: determine whether the target management command has been executed successfully; if the target management command has been executed successfully, update the execution identifier corresponding to the target management command in the subtask management table to a successful execution identifier; the successful execution identifier is used to instruct the cache management engine module in the chip to execute the next management command; if the target management command has been executed successfully, update the execution identifier corresponding to the target management command in the subtask management table to a failed execution identifier; the failed execution identifier is used to instruct the cache management engine module in the chip to continue executing the target management command until the execution identifier is a successful execution identifier.

[0348] Each module in the aforementioned cache management device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can invoke and execute the operations corresponding to each module.

[0349] In one exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 25As shown, this computer device includes a processor, memory, input / output interfaces (I / O), and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores data required for a cache management method. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. When the computer program is executed by the processor, it implements a cache management method.

[0350] Those skilled in the art will understand that Figure 25 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0351] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0352] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.

[0353] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0354] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0355] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0356] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A cache management method, characterized in that, The method includes: Obtain the target task and management command queue; the management command queue contains various management commands for the target task; When the last-level cache module does not contain the dependency data required by the target subtask in the target task, in response to the first management command, the dependency data required by the target subtask is read from memory and the dependency data is stored in the last-level cache module; In response to the second management command, the target subtask is initiated and executed; the target subtask includes the accelerator reading the dependency data from the last-level cache module.

2. The method according to claim 1, characterized in that, The process of acquiring the target task and managing the command queue includes: Obtain the target task and the corresponding data lifecycle information; the target task includes various subtasks; Based on the data lifecycle information, management commands corresponding to each subtask are generated, and a management command queue is constructed based on each management command.

3. The method according to claim 2, characterized in that, The step of generating management commands corresponding to each of the subtasks based on the data lifecycle information, and constructing a management command queue based on each of the management commands, includes: For each subtask, the category of the accelerator in the subtask is determined based on the data lifecycle information; the data lifecycle information includes the subtask association relationships between the subtasks. Based on the category, generate the management commands corresponding to the subtask; Based on the subtask relationships of each subtask and each management command, construct the management command queue for the target task.

4. The method according to claim 3, characterized in that, Determining the category of the accelerator in the subtask based on the data lifecycle information includes: In the data lifecycle information, the accelerator and associated data in the subtask are identified; If the associated data contains only dependency data, the accelerator is classified as the first category. If the associated data contains only result data, then the accelerator is classified as the second category; If the associated data contains result data and dependency data, the accelerator is classified as the third category.

5. The method according to claim 3, characterized in that, The step of generating management commands corresponding to the subtasks based on the category includes: If the category is the first category, generate the first management command, the fourth management command, and the fifth management command corresponding to the subtask; If the category is the second category, generate the second management command and the third management command corresponding to the subtask; If the category is the third category, generate the first management command, the second management command, the third management command, the fourth management command, and the fifth management command corresponding to the subtask.

6. The method according to claim 5, characterized in that, The data lifecycle information includes associated data for the subtask, which includes dependency data. If the category is the first category, the generation of a first management command, a fourth management command, and a fifth management command corresponding to the subtask includes: If the category is the first category, the first management command is generated according to the location of the dependency data; the first management command is used to read the dependency data required by the subtask from memory to the last-level cache module; A fourth management command corresponding to the subtask is generated; the fourth management command is used to instruct the accelerator to end the execution of the subtask; A fifth management command is generated to generate the dependency data; the fifth management command is used to release the dependency data in the last-level cache to the memory.

7. The method according to claim 5, characterized in that, The data lifecycle information includes associated data for the subtask, which includes result data. If the category is the second category, generating a second management command and a third management command corresponding to the subtask includes: If the category is the second category, a third management command corresponding to the result data is generated; the third management command is used to pre-allocate cache lines for the result data in the last-level cache; A second management command corresponding to the subtask is generated; the second management command is used to start the accelerator to execute the subtask.

8. The method according to claim 3, characterized in that, The step of constructing the management command queue for the target task based on the subtask relationships and management commands of each subtask includes: Based on the dependencies of each subtask and each management command, a management command flowchart for the target task is generated; the management command flowchart includes the execution order of each management command and the command association relationship between each management command; For each management command, based on the command association relationship, determine the number of preceding associated commands of the management command, and determine the subsequent associated commands of the management command; A management command queue is constructed based on each management command, the number of its preceding associated commands, and its subsequent associated commands.

9. The method according to claim 1, characterized in that, The management command queue includes target management commands of target type and non-target management commands of non-target type; the target management commands include the second management command and the fourth management command; the non-target management commands include the first management command, the second management command and the fifth management command.

10. The method according to claim 9, characterized in that, After acquiring the target task and managing the command queue, the method further includes: In the management command queue, determine each target management command of the target type and generate an execution identifier for each target management command; Based on the execution identifiers and target management commands, construct the subtask management table for the target task.

11. The method according to claim 1, characterized in that, After responding to the second management command and initiating the execution of the target subtask, the method further includes: Based on the generation probability of each of the management commands, at least one of the third, fourth, or fifth management commands is executed.

12. The method according to claim 11, characterized in that, The generation probability includes a first generation probability, a second generation probability, and a third generation probability. The step of executing at least one of the third, fourth, or fifth management commands based on the generation probability of each management command includes: Based on the first generation probability, a third management command is executed, and in response to the third management command, a cache line is pre-allocated in the last-level cache module for the result data of the target subtask; the cache line is used to store the result data. Based on the second generation probability, execute the fourth management command, and in response to the fourth management command, terminate the execution of the target subtask; Based on the third generation probability, a fifth management command is executed, and in response to the fifth management command, the dependent data in the last-level cache is released to the memory.

13. The method according to any one of claims 1 or 11, characterized in that, The method further includes: The current management command is retrieved from the management command queue, and the number of preceding associated commands corresponding to the current management command is determined; the current management command is any one of the first management command, the second management command, the third management command, the fourth management command, and the fifth management command. Based on the number of preceding associated commands, determine whether the current management command meets the execution conditions; the execution conditions are used to determine whether to respond to the current management command.

14. The method according to any one of claim 1 or 10, characterized in that, The method further includes: Determine whether the target management command was executed successfully; If the target management command is executed successfully, the execution identifier corresponding to the target management command is updated to the successful execution identifier in the subtask management table; the successful execution identifier is used to instruct the cache management engine module in the chip to execute the next management command. If the target management command is executed successfully, the execution identifier corresponding to the target management command is updated to the failure execution identifier in the subtask management table; the failure execution identifier is used to instruct the cache management engine module in the chip to continue executing the target management command until the execution identifier becomes the success execution identifier.

15. A cache management device, characterized in that, The device includes: The acquisition module is used to acquire the target task and the management command queue; the management command queue contains various management commands for the target task. The reading module is used to read the dependency data required by the target subtask in the target task from memory and store the dependency data in the last-level cache module in response to a first management command when the last-level cache module does not have the dependency data required by the target subtask in the target task. An execution module is used to initiate the execution of the target subtask in response to a second management command; the target subtask includes an accelerator reading the dependency data from the last-level cache module.

16. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 14.

17. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 14.

18. A chip, characterized in that, The chip includes programmable logic circuitry and / or program instructions that, when the chip is running, implement the steps of the method according to any one of claims 1 to 14.

19. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 14.

Images