Method and apparatus for dynamic cache control

Abstract

A method and related apparatus for dynamic cache control are provided. The method can include a processing unit generating a first instruction for a first processing stage, wherein the first instruction is configured to assign a buffer identifier (ID) for a target data of a dedicated allocation type; a cache unit setting a state of the buffer ID to a dedicated allocation state to allow the target data to be stored in a cache space corresponding to the buffer ID; the processing unit generating a second instruction for a second processing stage, wherein the second instruction is configured to change the state of the buffer ID to a release state; and the cache unit setting the state of the buffer ID to the release state to allow the target data stored in the cache space corresponding to the buffer ID to be replaced.

Description

[Technical Field] This disclosure generally relates to a cache management technique, and more particularly to a method and related apparatus for dynamic cache control. [Background Technology] Today, mobile applications (apps) typically offer a variety of games that require significant system resources, making it increasingly challenging to provide a smooth and satisfying user experience. High-resolution games with large, high-quality textures significantly increase the demand for memory access bandwidth. When this demand exceeds the bandwidth provided by the mobile platform, it becomes a system bottleneck and can lead to unstable frame rates. According to relevant technologies, computer processors are typically equipped with various types and levels of cache hardware to reduce memory bandwidth usage and improve data access efficiency. However, these caches are often designed as general-purpose resources and shared across different units of the system, and the caching mechanisms are not optimized for graphics rendering. Therefore, data frequently contends for cache space, resulting in repeated cache flushes and write-back operations. In particular, data may be frequently written back to memory and then reloaded or reallocated to the cache, causing additional memory bandwidth overhead. Therefore, a novel approach and related architecture are needed to solve the above problems without introducing any side effects, or to solve the problems in a way that is less likely to introduce side effects. [Summary of the Invention] The purpose of this disclosure is to provide a method and apparatus for dynamic cache control to solve problems in related technologies.

[0001] At least one embodiment of this disclosure provides a method for dynamic cache control, the method comprising: a processing unit generating a first instruction for a first processing stage, wherein the first instruction is configured to assign a buffer identifier (ID) to target data of a dedicated allocation type; a caching unit setting the state of the buffer ID to a dedicated allocation state to allow the target data to be stored in a cache space corresponding to the buffer ID; the processing unit generating a second instruction for a second processing stage, wherein the second instruction is configured to change the state of the buffer ID to a released state; and the caching unit setting the state of the buffer ID to the released state to allow the target data stored in the cache space corresponding to the buffer ID to be replaced.

[0002] In some embodiments, when the state of the buffer ID is set to the dedicated allocation state, the target data stored in the cache space corresponding to the buffer ID should not be replaced by subsequently arriving data.

[0003] In some embodiments, the replaced target data is not written back to external memory or dynamic random access memory (DRAM).

[0004] In some embodiments, the processing unit is a graphics processing unit (GPU).

[0005] In some embodiments, the first processing stage and the second processing stage are different rendering channels, and the target data of the dedicated allocation type includes a depth image.

[0006] In some embodiments, the second processing stage is the last rendering pass using the depth image.

[0007] In some embodiments, the target data of this dedicated allocation type includes all data accessed by one or a combination of a block buffer and a texture buffer.

[0008] In some embodiments, the target data of the dedicated allocation type is associated with a predefined access channel type.

[0009] In some embodiments, the processing unit is a neural processing unit (NPU), and the first processing stage and the second processing stage correspond to the operations of different layers of the neural network.

[0010] In some embodiments, the state of the buffer ID is set to remain in the dedicated allocation state across multiple processing stages until the cache unit receives a signal to change the state of the buffer ID to the release state.

[0011] In some embodiments, the dedicated allocation state is non-exclusive, allowing multiple buffer IDs to remain in the dedicated allocation state simultaneously within the same processing phase, with each buffer ID occupying a corresponding cache quota.

[0012] At least one embodiment of this disclosure provides an apparatus for dynamic cache control, wherein the apparatus includes at least one processing circuit, the processing circuit including a processing unit configured to perform at least one processing operation, and a cache unit configured to perform at least one cache operation. For example, the processing unit may be configured to generate a first instruction for a first processing stage, wherein the first instruction is configured to allocate a buffer ID to target data of a dedicated allocation type. The cache unit may be configured to set the state of the buffer ID to a dedicated allocation state to allow the target data to be stored in a cache space corresponding to the buffer ID. Furthermore, the processing unit may be configured to generate a second instruction for a second processing stage, wherein the second instruction is configured to change the state of the buffer ID to a released state. Additionally, the cache unit may be configured to set the state of the buffer ID to the released state to allow the target data stored in the cache space corresponding to the buffer ID to be replaced.

[0013] The advantage of this disclosure lies in the fact that, through reasonable design, the proposed method and related apparatus can provide a novel cache control mechanism, enabling specific data to be dedicated to cache storage during processing and achieving task-level fine-grained cache control. Specifically, the proposed mechanism can be implemented using existing cache units with appropriate hardware support, or by adding a dedicated cache unit for this mechanism. Furthermore, the proposed method and related apparatus can solve related technical problems without introducing any side effects or in a manner that makes it unlikely to introduce side effects.

[0014] These and other objects of this disclosure will undoubtedly become apparent to those skilled in the art upon reading the following detailed description of the preferred embodiments illustrated in conjunction with the figures. [Attached Image Description] Figure 1 This is a schematic diagram of an apparatus for dynamic cache control according to an embodiment of the present disclosure, wherein the apparatus is configured to operate based on a method for dynamic cache control.

[0015] Figure 2 A rendering control scheme according to an embodiment of the present disclosure is shown.

[0016] Figure 3 It shows Figure 2 The rendering control scheme shown here includes some implementation details.

[0017] Figure 4A The diagram illustrates the corresponding operations of a graphics processing unit (GPU) and its GPU driver in a cache quota management control scheme according to an embodiment of the present disclosure.

[0018] Figure 4B It shows Figure 1 The cache unit shown is in Figure 4A The cache quota management control scheme shown includes some operations related to memory.

[0019] Figure 5 A basic case control scheme for the method according to an embodiment of this disclosure is shown.

[0020] Figure 6 A multi-data control scheme according to an embodiment of the present disclosure is illustrated.

[0021] Figure 7 A configuration timing control scheme for a method according to an embodiment of the present disclosure is shown.

[0022] Figure 8 A channel-level control scheme for a method according to an embodiment of this disclosure is shown.

[0023] Figure 9 The main workflow of the method according to an embodiment of this disclosure is shown.

Detailed Implementation Methods

[0024] The method and related apparatus proposed in this invention can improve the overall performance of electronic devices (e.g., mobile devices, controllers, etc.) running high-resolution and large-scale high-quality textured games. For ease of understanding, in modern applications, a rendering flow may include multiple processing stages, such as depth map generation, geometry buffer (G-buffer) construction, post-processing, user interface (UI) compositing, and final display output for on-screen display. These processing stages can be performed by a graphics processing unit (GPU) using various types of jobs, including render-pass jobs, compute jobs, and transfer jobs. In more advanced applications, artificial intelligence (AI) computational tasks based on deep learning neural networks (such as deep learning inference operations) may also run on a neural processing unit (NPU) and participate in the pipeline. During rendering, some data may exhibit a producer-consumer relationship. For example, data generated in an earlier pass may be consumed (or "used") in a later pass; that is, data generated in an earlier pass is used by a later pass. Based on this behavior, the method proposed in this invention can set or allocate such data as dedicated-allocated (which can also be interchangeably described herein as "dedicated," "dedicated-allocated state," or "dedicated state") when it is generated, and release the dedicated allocation after the data is last consumed. During the interval between the time the data is generated and the time it is consumed, the data may be accessed multiple times (e.g., reads and / or writes) because it is in a dedicated-allocated state in the cache and there are cache hits. A higher cache hit rate (or more cache hits) can reduce memory access operations, thereby reducing memory bandwidth usage. In addition, some data may be immediate data, meaning that once consumed, it will not be used again.For data exhibiting this behavior, after releasing the dedicated allocation setting, the data can be discarded and the write-back operation to memory can be skipped (i.e., no write-back operation is performed after releasing the dedicated allocation configuration), allowing its cache quota to be reclaimed and replaced by other data (such as data arriving later). In addition to the discard operation, the original refresh operation is also supported. The proposed method and related apparatus provide a novel cache control mechanism that allows specified data (e.g., selected data or target data) to be set to a dedicated allocation state in the cache during operations (such as rendering or computation), thereby achieving job-level fine-grained cache management. For ease of description and understanding, embodiments of this disclosure are illustrated using image rendering via a GPU as an example, but this disclosure is not limited thereto. For example, the techniques of this disclosure can be applied to any other suitable processing task / process. In the example of GPU-rendered images, this mechanism may include two main cache control operations: (1) allocating frequently used GPU rendering data in the cache; and (2) releasing the cache quota of the target data when the target data's lifecycle ends (or is completed). In the first of these two operations, frequently accessed GPU rendering data (e.g., device memory objects of image type and / or buffer type) may be allocated in the cache, making the cache quota (or cache space) occupied by the target data dedicated, preventing other data from flushing the target data out of the cache. This behavior can improve the cache hit rate of such frequently used data, thereby reducing memory bandwidth usage. The target data is not limited to a single image / buffer type object, but can also be a group of objects, or even all access traffic of a channel type, which is not limited in this invention. For example, all texture-read access traffic of a render pass may be dedicated, and / or, all depth-test access traffic of a render pass may be dedicated. In the second of these two operations, when the target data's lifecycle ends (i.e., it is consumed for the last time), the dedicated cache quota for the target data can be released to allow the target data to be replaced by other data (e.g., data arriving subsequently). (For example, the target data will be discarded from the cache and the write-back operation will be skipped, meaning no write-back operation is required, thus avoiding writing to other / external memory, unlike traditional methods.) By eliminating memory write-backs for such data, memory bandwidth usage can be further reduced.Furthermore, this mechanism can be implemented using existing cache units with appropriate hardware support, or by adding new hardware dedicated to this mechanism. In this case, the cache can be integrated as a layer in the system's memory hierarchy. Remaining unused cache quotas can still be used by other GPUs in the system and / or shared by other processing units (e.g., NPU or central processing unit (CPU)).

[0025] Figure 1 This is a schematic diagram of an apparatus for dynamic cache control according to an embodiment of the present disclosure, configured to operate based on a method for dynamic cache control. The apparatus may be implemented as an electronic device 100, including a processing circuit 110, a display module (such as a touch display module 130), and an audio output module 140. The processing circuit 110 includes a processing unit 111 and a cache unit 112, respectively configured to perform at least one processing operation and at least one cache operation. In the case where the processing unit 111 and cache unit 112 are implemented as processing unit 111A and cache unit 112A, respectively, the cache unit 112A may be located outside the processing unit 111A. In the case where the processing unit 111 and cache unit 112 are implemented as processing unit 111B and cache unit 112B, respectively, the cache unit 112B may be located inside the processing unit 111B. Examples of the electronic device 100 include, but are not limited to, mobile devices, such as multifunction mobile phones.

[0026] In electronic device 100, processing unit 111 generates a first command for a first processing stage, wherein the first command is configured to allocate a buffer identifier (buffer ID, i.e., the buffer ID is associated with the target data) to dedicated-allocated type target data; and caching unit 112 sets the state of the buffer ID to a dedicated-allocate state (also interchangeably described as a "dedicated state") to allow (e.g., only allow) the target data (its access traffic) to be stored in the cache space (also described as a "cache quota") corresponding to the buffer ID. For example, when the state of the buffer ID is set to dedicated-allocate state, the target data stored in the cache space corresponding to the buffer ID should not be replaced by subsequently arriving data (e.g., target data in a non-dedicated-allocate state). Furthermore, in the electronic device 100, the processing unit 111 generates a second instruction for the second processing stage, wherein the second instruction is configured to change the state of the buffer ID to a released state, and the caching unit 112 sets the state of the buffer ID to a released state to allow the target data stored in the cache space (quota) corresponding to the buffer ID to be replaced by subsequently arriving data. For example, the first processing stage and the second processing stage may be different rendering channels, and the target data of the dedicated allocation type may include a depth image. As another example, the second processing stage may be the last rendering channel using the target data (such as a depth image). It should be noted that the target data of the dedicated allocation type may include all data accessed by one or a combination of a tile buffer and a texture buffer. In some implementations, the target data may be associated with data transfer between on-chip buffers and memory. For example, in tile-based rendering, the rendering result may be written from the tile buffer (such as an on-chip tile buffer) to memory, and then read from the memory into a texture buffer (such as an on-chip texture buffer) for further processing. The access traffic generated by such write and read operations will pass through the cache hierarchy. Therefore, data accessed by either the block buffer and the texture buffer or a combination thereof can be regarded as target data, and the special allocation mechanism proposed in this invention is applied to such target data in the cache.In addition, the electronic device 100 may also include a memory located outside the processing circuit 110. Figure 1 (Not shown), this memory is typically considered as external memory or main memory. Preferably, such memory can be implemented using dynamic random access memory (DRAM), while the described cache can be implemented using faster static random access memory (SRAM), but the present invention is not limited to this. In the solution of the present invention, the replaced target data is not written back to DRAM or external memory (e.g., other memory further away / slower than the cache) because, in the solution proposed in the present invention, the write-back operation for the replaced target data is skipped, unlike conventional solutions where the target data in the cache is written back to other memory (such as external memory or other caches with slower access speeds) when replaced by other data.

[0027] exist Figure 1 In the architecture shown, the processing unit 111 may be a graphics processing unit (GPU), but this disclosure is not limited thereto. For example, the processing unit 111 may be a neural processing unit (NPU). In particular, the first processing stage and the second processing stage may correspond to the operations of different layers of a neural network, and the neural network may run on the NPU.

[0028] Figure 2 A rendering control scheme according to an embodiment of the present disclosure is shown, wherein the display device can be implemented via a touch screen and can serve as... Figure 1 An example of the touch display module 130 shown. Devices (such as electronic device 100) can operate based on a rendering control scheme to optimize cache usage for processing target data (such as depth image 201). Figure 2 As shown, a simple rendering may include at least three rendering channels, such as rendering channels 1, 2, and 3 (simplified as "channel 1", "channel 2", and "channel 3" respectively) 210, 220, and 230: (1) Rendering channel 1: Draw depth image 201 (e.g., write data to the storage space of depth image 201, the process of which will be cached). (2) Rendering channel 2: Perform some processing based on depth image 201, where this operation will read the contents of depth image 201 (access traffic). (3) Rendering channel 3: Complete the game scene to fit the screen and add one or more user interfaces (UI) to display on the display device (step 240). This is the last time the depth image 201 will be used, and the content will not be used again afterward. However, this disclosure is not limited thereto. For example, in practical applications, additional channels can be inserted between these channels. In this example, depth image 201 can serve as the target data mentioned above.

[0029] Figure 3 It shows Figure 2 The rendering control scheme shown includes some implementation details. In this embodiment, the depth image 201 can be configured to carry the depth of all objects, such as... Figure 3 The subgraph (a) is shown and can serve as an example of target data for a specific allocation type. Figure 3 As shown in sub-figure (b), the G-buffer (G-buffer) 202 used in rendering channel 2 can include image information 320, and more specifically, can store image information 320. For ease of understanding, the G-buffer 202 can include various attachments, such as albedo / diffuse, normal, metallic / roughness, depth, motion vector (MV), and specular, etc. Figure 3 As shown in sub-figure (c), the scene 203 with UI in rendering channel 3 (e.g., a scene with one or more UIs) may include various UIs 331, 332, and 333 for different purposes. This is for illustrative purposes only and is not intended to limit the scope of this disclosure. According to some embodiments, the depth image 201, the image information 320 in the G buffer 202, and the scene with UI 203 and UIs 331, 332, and 333 therein may vary.

[0030] Based on a rendering control scheme that implements dynamic cache control, job-level cache operations can be applied to reduce memory bandwidth through dedicated allocation or, more specifically, discard-after-use behavior. In rendering channel 1, regarding drawing depth image 201 (i.e., writing data to the memory space of depth image 201 after caching), cache unit 112 sets depth image 201 to a dedicated allocation state, ensuring that no other data can evict it from the cache. In rendering channel 2, regarding processing based on depth image 201, processing operations to read depth image 201 are performed. Due to dedicated allocation, all read accesses achieve cache hits, reducing access to external memory. In rendering channel 3, regarding completing the game scene to fit the screen and adding one or more UI elements for display, final scene rendering and UI compositing are performed. This is the last channel to use depth image 201. However, the operation of changing the buffer ID associated with depth image 201 to a released state is not required to be performed in this rendering channel. In some embodiments, the buffer ID can be changed to a released state in a subsequent processing stage, provided that the release operation is scheduled after all accesses to the depth image 201 have been completed, to ensure the correct access order and avoid access requirements after the depth image 201 has been discarded (replaced) from the cache. After this channel, the depth image 201 becomes unnecessary in the cache. Therefore, the cache quota corresponding to the depth image 201 (i.e., the target data) is released, allowing subsequent data to replace it, and the released target data is discarded without being written back to other memory (such as dynamic random access memory (DRAM) or other cache units), thereby further reducing memory bandwidth.

[0031] Preferably, an interface is provided to allow upper-layer software to hint to the graphics processing unit (GPU) driver that certain target data requires a buffer ID. Cache unit 114 uses / by virtue of the buffer ID to determine whether a given access flow belongs to target data of a specific allocation type. When the GPU configures properties for a device memory object (a common technical term in the relevant art), the buffer ID is synchronized to cache unit 112 via the GPU-Cache interface. For example, this mechanism can be implemented by adding a user-level application programming interface (API) to the graphics or computing language interface (e.g., in Vulkan's vkAllocateMemory call). Applications can use such APIs to hint to the GPU driver that a specific device memory object (also often simply referred to as device memory) requires a buffer ID to be allocated as target data, and the GPU driver then synchronizes the corresponding buffer ID with cache unit 112. Multiple data objects can share the same buffer ID. In this case, these data objects can be considered as a group controlled within the cache. For example, in a Vulkan implementation, multiple images can be bound to the same device memory object. The use of buffer IDs is not limited to a specific data range / object. Furthermore, buffer IDs can also be configured in a channel-wise manner, where dedicated allocation data can represent all access traffic through this channel (e.g., all data accessed by the depth testing unit or all data accessed by the texture unit). In the above embodiments, the timing of buffer ID allocation is not limited to the memory-allocation stage. For example, a buffer ID can be allocated immediately before the corresponding resource is used (e.g., during resource preparation for the rendering target, such as during a vkBindImageMemory call).

[0032] In addition, an interface can be provided that allows upper-layer software components to notify the lower layers of the GPU to perform dedicated allocation and / or release of cache quotas for target data of the corresponding GPU task. It is assumed that the target data has been allocated a buffer ID. For example, this mechanism can be implemented by adding a user-level API to the graphics or computation language interface (e.g., in Vulkan's `vkCmdBeginRenderPass` call). Applications can call such APIs when recording rendering, computation, transport, or neural processing instructions to notify the GPU driver, such as the GPU driver itself, to generate appropriate cache control instructions. The timing of sending / issuing cache control instructions is not limited to the command recording stage. For example, cache control instructions can also be sent / issued immediately before the GPU executes action instructions (e.g., during Vulkan's `vkQueueSubmit` call). After the cache quota for the target data is released, the buffer ID setting remains stored in the data attribute table, allowing subsequent accesses to the target data to be dedicated to cache allocation again (e.g., when the application draws to the same image every frame). The buffer ID setting is cleared only when the corresponding device memory object is reclaimed by the application; afterwards, the buffer ID can be used for other data. If multiple data objects are assigned to a dedicated allocation and attempt to occupy the cache simultaneously, and the available cache size is insufficient to hold all objects, cache replacement may occur. The following describes an example of replacement behavior, but this disclosure is not limited thereto: (1) Replacement of data between dedicated allocations does not use a discard operation; (2) A later dedicated allocation request can flush out (or replace) part of the data in the earlier request, so that this part of the data is written back to memory; (3) After the replacement is completed, the earlier data may be partially retained in the cache, while the later data will be fully cached.

[0033] It should be noted that the replacement behavior described above is merely an illustrative example. When the cache size is insufficient, other replacement strategies may be applied depending on implementation requirements. For example, in some implementations, previously allocated data may be retained in the cache, while subsequent allocated data is prevented from occupying cache space already occupied by previously allocated data. Alternatively, other replacement strategies may be employed without departing from the spirit and scope of this disclosure.

[0034] Figure 4AThis illustrates the corresponding operation of the graphics processing unit (GPU) 420 and its GPU driver 401 in a cache quota management control scheme according to an embodiment of the present disclosure. Figure 4B It shows in Figure 4A Under the cache quota management control scheme shown, cache unit 112 (for example, Figure 1 The cache units 112A / 112B shown are partially operational with memory 450, wherein the operation is used to indicate... Figure 4A The GPU driver 401 and GPU 420 shown are... Figure 4B The arrows indicating the interactions between the cache units 112 can be connected via nodes N1 to N5. In this embodiment, the processing unit 111 and the external memory can be implemented as a GPU 420 and a memory 450, respectively.

[0035] In operation 410, GPU driver 401 can bind resources to GPU instructions to configure (or "configure") related data attributes (for brevity, labeled "GPU instructions to configure data"). Specifically, it can control cache unit 112 to add a buffer ID attribute in the case of target data (in other words, the resource corresponds to the target data). For example, cache unit 112 can maintain a data property table 430 that records attribute information for each data object or predefined channel type. In an example implementation, data property table 430 may include multiple data attribute entries, such as entry 431 for data A (target data), entry 432 for data B (normal, non-target data), entry 433 for data C (target data, but already released), entry 434 for predefined channel type X (target channel type), and entry 435 for predefined channel type Y (normal, non-target channel type). Regarding data A (target data), entry 431 includes attribute information and memory address of data A, a buffer ID field allocated with buffer ID a, and a status field indicating that data A is in a dedicated state (also interchangeably described as "dedicated allocation state" in this embodiment). Regarding data B (ordinary non-target data), entry 432 includes attribute information and memory address of data B. Since data B is not target data, there is no buffer ID field or corresponding status field. Regarding data C (target data but released), entry 433 includes attribute information and memory address of data C, a buffer ID field allocated with buffer ID c, and a status field indicating that data C is in a released state. Regarding predefined channel type X (target channel), entry 434 includes attribute information for predefined channel type X, a buffer ID field allocated with buffer ID x, and a status field indicating a dedicated state. In this case, the target data corresponds to all access traffic belonging to this channel type. Regarding predefined channel type Y (ordinary, non-target channel), entry 435 includes attribute information for predefined channel type Y. Since this channel type was not specified as target data, the buffer ID or status field is not included.

[0036] Furthermore, the GPU driver 401 can control the cache unit 112 to update the status of the buffer ID and / or allocate a buffer ID to a preset channel. The cache unit 112 can also query the data attribute table 430 to obtain / retrieve attribute information of the corresponding data. For example, related operations may include: (1) In response to the instruction of GPU task 1 (labeled as “Instruction of GPU task 1” in the figure) 411, buffer ID 412 (e.g., buffer ID a) has a “dedicated” state (labeled as “Buffer ID a = dedicated” for simplicity), GPU 420 writes data A, and cache unit 112 allocates cache space corresponding to cache quota 441 (e.g., cache quota 1) to the data, wherein cache quota 441 (e.g., cache quota 1) is set to valid (labeled as “Cache quota 1: valid” in the figure), and the content 442 stored in the cache space corresponding to cache quota 441 is data related to buffer ID a (labeled as “Data of buffer ID a” for simplicity). (2) In response to the instruction of GPU task 2 (labeled as “Instruction of GPU task 2” in the figure) 413, buffer ID 414 (e.g., buffer ID x) has a “dedicated” state (labeled as “Buffer ID x = dedicated” for simplicity), GPU 420 reads data related to channel type X from memory 450, and cache unit 112 allocates cache space corresponding to cache quota 443 (e.g., cache quota 2) when GPU 420 reads data of channel X, wherein cache quota 443 (e.g., cache quota 2) is set to valid (labeled as “Cache quota 2: valid” in the figure), and the content 444 stored in the cache space corresponding to cache quota 443 is data related to buffer ID x (labeled as “Data of bufferID x” for simplicity). (3) In response to the instruction of GPU Task 3 (labeled "Instruction of GPU Task 3" in the figure) 415, buffer ID 416 (e.g., buffer ID c) has a "released" state (labeled "Buffer ID c = released" for brevity), allowing any type of GPU access (e.g., any access of GPU 420), and because the buffer ID state shown in data attribute table 430 is "released," the cache quota 445 (e.g., cache quota N) associated with buffer ID c becomes invalid (labeled "Cache quota N: invalid" in the figure). Therefore, the data associated with buffer ID c (labeled "Data of buffer ID c" for brevity) can be discarded / replaced, such as... Figure 4B The dashed and strikethrough lines are used to indicate this for ease of understanding. Under the control of cache unit 112, other data can replace this quota (i.e., the cache quota corresponding to buffer ID c), regardless of whether the access traffic of other data is generated by the GPU, memory or any other processing or memory access source through the cache.

[0037] In the above embodiments, a cache quota can refer to a logical allocation related to data using cache space. When data is in a dedicated state, the cache space corresponding to the cache quota can store the target data. Furthermore, in the data attribute table 430, data attribute entries including an additional buffer ID attribute (or "buffer ID attribute") are considered targets, indicating that data with the additional buffer ID attribute is considered target data. Once the cache unit 112 identifies access traffic belonging to the target data, it marks the corresponding buffer ID on the cache quota. The initial state of the buffer ID is "released". Target channel types are predefined, limited in number, and the buffer IDs associated with these channel types can be updated subsequently based on the executing GPU task.

[0038] Figure 5 A basic case control scheme for a method according to an embodiment of this disclosure is illustrated, wherein the GPU driver 520 and cache unit 530 can serve as examples of GPU driver 401 and cache unit 112, respectively, and can operate under the control of upper-layer software components, such as an upper-layer software stack 510 including but not limited to applications, game engines, and / or graphics or computing API layers (labeled "APP / game engine / layer" for brevity), which can issue instructions to the GPU driver 520. Figure 5As shown, the upper-layer software stack 510 can issue GPU instructions to the GPU driver 520 through an upper-layer graphics or computing language interface (e.g., Vulkan). The GPU driver 520 can interpret these instructions and communicate with the cache unit 530 through a lower-layer cache-control interface. Through these interfaces, buffer ID settings and cache control states (e.g., dedicated state or released state) are passed to the cache unit 530.

[0039] like Figure 5 As further shown, the upper-layer software stack 510 can execute a series of operations 511, 513, and 515. Each operation may include a corresponding API-level sub-operation, denoted as operations 512, 514, and 516, respectively. Operation 511 indicates that the upper-layer software stack 510 allocates a device memory object, and the internal sub-operation 512 (shown as "+API: target data requires buffer ID hint") allows the upper-layer software stack 510 to issue an API-level indication that the allocated device memory object corresponds to target data that requires a buffer ID. Operation 513 indicates that the instruction for GPU task 1 is recorded, and the internal sub-operation 514 (shown as "+API: set target data to private allocation") enables the upper-layer software stack 510 to call the API to request the GPU driver 520 to set the buffer ID state of the target data to a private state for task 1. Operation 515 records the instructions for GPU Task 2. This operation occurs after all usage of the target data in Task 1 is complete. The internal sub-operation 516 (shown as "+API: Set target data to release quota") enables the upper-layer software stack 510 to call the API to request GPU Task 2 to change the buffer ID state of the target data to the released state. The arrows labeled "Target data has a buffer ID" and "All usage of target data complete" indicate the logical progression between the above operations; that is, the target data acquires a buffer ID between operations 511 and 513, and all usage of the target data is completed between operations 513 and 515.

[0040] Furthermore, GPU driver 520 can execute a series of operations 521, 523, and 525 corresponding to the upper-level operations (such as operations 511, 513, and 515). Each GPU driver operation (such as operations 521, 523, and 525) includes a corresponding internal sub-operation, as shown in operations 522, 524, and 526, respectively. Operation 521 represents the configuration stage, in which GPU driver 520 receives an instruction from operation 511 that the corresponding resource is target data (labeled for brevity as "GPU instruction to configure target data"), and the internal sub-operation 522 (shown as "Assign a buffer ID to target data") configures the data attributes within GPU driver 520 by assigning a buffer ID to the target data, so that the buffer ID information can be synchronized with cache unit 530 afterwards. Operation 523 is used to build instructions for GPU task 1, with input from operation 513, which indicates that GPU task 1 is recording and the target data should be placed in a dedicated allocation state. Internal sub-operation 524 (shown as "Change the state of the buffer ID to dedicated") enables GPU driver 520 to build instructions for GPU task 1 along with cache control instructions to set the buffer ID state of the target data to dedicated. Operation 525 is used to build instructions for GPU task 2, with input from operation 515, which indicates that all use of the target data has been completed and the target data should be released. Internal sub-operation 526 (shown as "Change the state of the buffer ID to released") enables GPU driver 520 to build instructions for GPU task 2 along with corresponding cache control instructions to set the buffer ID state of the target data to released. The arrows from operations 511, 513, and 515 to operations 521, 523, and 525 respectively represent upper-level intentions passed to GPU driver 520. Specifically, operation 511 triggers operation 521 to allocate a buffer ID, operation 513 triggers operation 523 to set a dedicated state for task 1, and operation 515 triggers operation 525 to set a release state for task 2.

[0041] Furthermore, cache unit 530 can execute a series of operations 532, 534, and 536 corresponding to the operations of the GPU driver (such as operations 521, 523, and 525). In operation 532, cache unit 530 receives an instruction from operation 521 and stores the buffer ID associated with the target data for subsequent identification of whether a certain access traffic belongs to the target data. In operation 534, cache unit 530 receives an instruction from operation 523 notifying cache unit 530 that the buffer ID has changed to a private allocation state, and updates the state of the buffer ID to "private state" (e.g., in...). Figure 4BIn the data attribute table 430 shown, cache unit 530 is used as an example of cache unit 112. The arrow from operation 534 to operation 536 is labeled "Dedicated allocation when accessing data of buffer ID", indicating that when GPU or memory access traffic arrives, cache unit 530 can allocate cache space corresponding to the cache quota for the data associated with the buffer ID. For example, such dedicated allocation may involve cache unit 530 with the GPU and external memory (e.g., respectively in...). Figure 4A and Figure 4B This refers to input / output (I / O) interactions between either the GPU 420 or the memory 450 shown in the diagram. In operation 536, cache unit 530 receives from operation 525 an indication that the buffer ID has transitioned to a "released" state, and updates the buffer ID state to "released" (e.g., in...). Figure 4B In the data attribute table 430 shown, cache unit 530 is used as an example of cache unit 112. Once in this state (referred to as the released state), the associated cache quota can be replaced. The arrow pointing outward from operation 536 is labeled "Replace data of buffer ID when other data wants to be allocated on the cache," indicating that when other data requests cache allocation, cache unit 530 can replace the data associated with the released buffer ID. This replacement process may involve I / O interactions between cache unit 530 and either the GPU or external memory (e.g., GPU 420 and memory 450 mentioned above), depending on whether the data is discarded or written back.

[0042] Figure 6 A multiple data control scheme according to an embodiment of the present disclosure is illustrated, wherein the GPU driver 620 and cache unit 630 can serve as examples of GPU driver 401 and cache unit 112, respectively, and can operate under the control of upper-layer software components, such as upper-layer software stack 610, including but not limited to applications, game engines, and / or graphics or computing API layers (labeled "APP / game engine / layer" for brevity), which can issue instructions to the GPU driver 620. Figure 6 As shown, the upper-layer software stack 610 can issue GPU instructions to the GPU driver 620 through an upper-layer graphics or computing language (such as Vulkan), and the GPU driver 620 can interpret these instructions and communicate with the cache unit 630 through the lower-layer cache control interface, through which buffer ID settings and cache control states (such as dedicated allocation or release) are passed to the cache unit 630.

[0043] like Figure 6As further shown, the upper-layer software stack 610 can execute a series of operations 611, 613, 615, and 617, each with corresponding API-level sub-operations 612, 614, 616, and 618. These operations illustrate a multi-data control scenario where target data A and target data B are independently allocated and released. Operation 611 represents allocating device memory objects for data A and data B, and the internal sub-operation 612 (shown as "+API: Target data A, B require buffer ID hint") enables the upper-layer software stack 610 to issue an API-level hint, indicating that both data A and data B need to be allocated buffer IDs as target data. The arrow from operation 611 to operation 613 (labeled "Target data has buffer ID") indicates that buffer IDs for data A and data B have been allocated before recording GPU task 1. Operation 613 indicates that GPU task 1 is recorded, and the internal sub-operation 614 (shown as "+API: Set target data A to dedicated allocation") enables the upper-layer software stack 610 to request that the buffer ID state of data A be set to dedicated allocation state for execution of GPU task 1. Operation 615 indicates that GPU task 2 is recorded, and the internal sub-operation 616 (shown as "+API: Set target data B to dedicated allocation") enables the upper-layer software stack 610 to request that the buffer ID state of data B be set to dedicated allocation state for execution of GPU task 2. This indicates that data A and data B can be dedicated independently in different GPU tasks. Operation 617 indicates that GPU task 3 is recorded, and the internal sub-operation 618 (shown as "+API: Set target data A to free quota") enables the upper-layer software stack 610 to request that the buffer ID state of data A be changed to free state to indicate that all use of data A has been completed and its cache quota can be replaced.

[0044] Furthermore, GPU driver 620 can execute a series of operations 621, 623, 625, and 627, corresponding to upper-level operations such as operations 611, 613, 615, and 617, respectively. Each GPU driver operation, such as operations 621, 623, 625, and 627, includes a corresponding internal sub-operation, as shown in operations 622, 624, 626, and 628. Operation 621 is used to prepare target data for GPU instructions (labeled for brevity as "GPU instructions to prepare target data"), its input comes from operation 611, which indicates that device memory objects for data A and data B have been allocated and identified as target data, and the internal sub-operation 622 (shown as "allocate buffer IDs a, b for target data A, B") indicates configuring the data attribute entries of the GPU driver to assign buffer ID a to data A and buffer ID b to data B. These buffer IDs can then be synchronized with cache unit 630. Operation 623 generates instructions for GPU Task 1, taking input from operation 613, which indicates that GPU Task 1 is being recorded and that data A should be placed in a dedicated allocation state in Task 1. An internal sub-operation 624 (shown as "Change the state of buffer ID a to dedicated state") enables GPU driver 620 to generate instructions for GPU Task 1 along with corresponding cache control instructions to set the buffer ID state of data A to a dedicated state. Operation 625 generates instructions for GPU Task 2, taking input from operation 615, which indicates that GPU Task 2 is being recorded and that data B should be placed in a dedicated allocation state in Task 2. An internal sub-operation 626 (shown as "Change the state of buffer ID b to dedicated state") enables GPU driver 620 to generate instructions for GPU Task 2 along with corresponding cache control instructions to set the buffer ID state of data B to a dedicated allocation state. Operation 627 is used to generate GPU task 3 instructions, which take input from operation 617, which indicates that all use of data A has been completed and its cache quota should be released. An internal sub-operation 628 (shown as "change the state of buffer ID a to the released state") enables GPU driver 620 to generate GPU task 3 instructions with cache control instructions to set the buffer ID state of data A to the released state.

[0045] The arrows from operations 611, 613, 615, and 617 to operations 621, 623, 625, and 627 indicate that the upper-level intent is passed to the GPU driver 620. Specifically, operation 611 triggers the allocation of the buffer ID in operation 621, operation 613 triggers the dedicated allocation of data A in operation 623, operation 615 triggers the dedicated allocation of data B in operation 625, and operation 617 triggers the release of data A in operation 627.

[0046] Furthermore, cache unit 630 can perform a series of operations 632, 634, 636, and 638, corresponding to GPU driver operations, such as operations 621, 623, 625, and 627, respectively. Each operation reflects how cache unit 630 interprets the buffer ID allocation and buffer ID state update for target data A and B. In operation 632, cache unit 630 receives an instruction from operation 621 and stores the buffer ID a of data A and the buffer ID b of data B for subsequent identification of their traffic, more specifically, for identifying whether subsequent incoming access traffic corresponds to target data A or target data B. In operation 634, cache unit 630 receives an instruction (e.g., an update) from operation 623 to notify cache unit 630 that buffer ID a has been transitioned to a dedicated allocation state for GPU task 1, and cache unit 630 maintains the dedicated state of buffer ID a, for example, updating the state of buffer ID a to "dedicated allocation state" (e.g., in...). Figure 4B In the data attribute table 430 shown, cache unit 630 is used as an example of cache unit 112. The arrow from operation 634 to operation 636 is labeled "Dedicated allocation of data to buffer ID a: quota 1 allocated to it", indicating that when the access traffic belongs to buffer ID a, cache unit 630 can allocate cache space corresponding to cache quota 1 and store the corresponding data in that cache space. For example, this dedicated allocation may involve cache unit 630 and GPU or external memory (e.g., respectively in...). Figure 4A and Figure 4B Input / output (I / O) interaction between either GPU 420 or memory 450 (as shown in the diagram). In operation 636, cache unit 630 receives an indication (e.g., an update) from operation 625 to notify cache unit 630 that buffer ID b has been transitioned to a dedicated allocation state for GPU task 2, and cache unit 630 maintains the state of buffer ID b as dedicated, for example, updating the state of buffer ID b to "dedicated allocation state" (e.g., in...). Figure 4B In the data attribute table 430 shown, cache unit 630 is used as an example of cache unit 112. The arrow from operation 636 to operation 638 is labeled "Dedicated allocation of data to buffer ID b: quota 2 allocated to it", indicating that for the access traffic corresponding to buffer ID b, cache unit 630 can allocate cache space corresponding to cache quota 2 and store the corresponding data in that space. For example, this dedicated allocation may involve cache unit 630 and GPU or external memory (e.g., respectively in...). Figure 4A and Figure 4BThis refers to input / output (I / O) interactions between either the GPU 420 or the memory 450 shown in the diagram. This illustrates that multiple target data objects (e.g., data A and B) can be independently and dedicated, and can simultaneously occupy different cache quotas. In operation 638, cache unit 630 receives an indication (e.g., an update) from operation 627 to notify cache unit 630 that buffer ID a has transitioned to a released state for GPU task 3, and cache unit 630 maintains the released state of buffer ID a, for example, by updating the state of buffer ID a to "released state" (e.g., in...). Figure 4B In the data attribute table 430 shown, cache unit 630 is used as an example of cache unit 112. Once the state of buffer ID is "released" (i.e., released state), the associated cache quota (e.g., quota 1) will meet the replacement condition. The arrow pointing outward from operation 638 is marked "Replace buffer ID a data (e.g., quota 1 space) when other data access traffic arrives", indicating that cache unit 630 can replace the data of buffer ID a stored in the cache space corresponding to quota 1 with other data requesting cache allocation. This replacement process may involve input / output (I / O) interaction between cache unit 630 and either the GPU or external memory (e.g., GPU 420 and memory 450 mentioned above), depending on the discard or write-back behavior.

[0047] Furthermore, dedicated allocation mechanisms are not limited to a single buffer ID or a single task. For example, when a GPU task carries access traffic to multiple target data objects, the buffer IDs associated with these data objects (e.g., buffer ID a and buffer ID b) can be configured to dedicated allocation simultaneously, causing their corresponding data to be allocated to the cache concurrently, each occupying a corresponding cache quota. In other words, dedicated allocation is not limited to a single task or a single buffer ID. For example, in the corresponding Figure 6 During the time interval from operation 636 to 638, buffer ID a and buffer ID b can remain in a dedicated allocation state simultaneously, allowing access traffic for data A and data B to be allocated to different cache quotas at the same time.

[0048] In the above embodiments, if the cache is large enough to accommodate data A and B—for example, when the cache capacity is sufficient to hold data objects (such as data A and data B)—each data object can maintain a fully dedicated allocation in the cache, and their cache usage will not overlap. However, if the cache capacity is insufficient to hold data objects (such as data A and data B) simultaneously, cache replacement may occur depending on the replacement strategy of the specific implementation. In one example, the dedicated data B that appears later may partially clear the cache space corresponding to data A, causing part of the contents of data A to be written back to memory 450. Because partial clearing occurred in this example, data A may not fully benefit from the use-and-discard feature.

[0049] Figure 7 A configuration timing control scheme for a method according to an embodiment of this disclosure is illustrated, wherein the GPU driver 720 and cache unit 730 may serve as examples of GPU driver 401 and cache unit 112, respectively, and may operate under the control of upper-layer software components, such as upper-layer software stack 710, including but not limited to applications, game engines, and / or graphics or computing API layers (labeled for simplicity as "APP / game engine / layer"), which are capable of issuing instructions to the GPU driver 720. Figure 7 As shown, the upper-layer software stack 710 can issue GPU instructions to the GPU driver 720 through an upper-layer graphics or computing language (such as Vulkan). The GPU driver 720 can interpret these instructions and communicate with the cache unit 730 through the lower-layer cache control interface. Through these interfaces, the buffer ID settings and cache control states (such as dedicated or released) are passed to the cache unit 730.

[0050] like Figure 7As further illustrated, the upper-layer software stack 710 can execute a series of operations 711, 713, and 715, each with a corresponding API-level sub-operation, shown as operations 712, 714, and 716, respectively. These operations illustrate another configuration timing control scenario where the allocation and state transition of buffer IDs occur at command-submission time, rather than command-record time. Operation 711 indicates that the upper-layer software stack 710 binds a resource (e.g., an image or buffer) to a device storage object. The internal sub-operation 712 (shown as "+API: Target data requires buffer ID hint") enables the upper-layer software stack 710 to issue an API-level hint indicating that the bound resource corresponds to target data that requires a buffer ID allocation. The arrow from operation 711 to operation 713 is labeled "Target data has a buffer ID," indicating that the target data has obtained a buffer ID before submitting GPU task 1. Operation 713 indicates that the upper-layer software stack 710 submits the instruction for GPU task 1, and the internal sub-operation 714 (shown as "+API: Set target data to private allocation") allows the upper-layer software stack 710 to call the API to request that the buffer ID state of the target data be set to a private state for execution of GPU task 1. Operation 715 indicates that the upper-layer software stack 710 submits the GPU task instruction, and the internal sub-operation 716 (shown as "+API: Set target data to free quota") enables the upper-layer software stack 710 to request that the buffer ID state of the target data be changed to a free state after all use of the target data is completed. The arrow from operation 713 to operation 715 is marked "Complete all use of target data", indicating that the upper-layer software stack 710 submits the instruction for GPU task 2 after determining that all accesses to the target data in the instruction for task 1 have been completed.

[0051] Furthermore, the GPU driver 720 can execute a series of operations 721, 723, and 725, corresponding to upper-level operations such as operations 711, 713, and 715, respectively. Each GPU driver operation, such as operations 721, 723, and 725, includes internal sub-operations, shown as operations 722, 724, and 726, respectively. Operation 721 is used to map resources to memory, its input being from operation 711, indicating that the resource has been bound to a device storage object and identified as target data. The internal sub-operation 722 (shown as "Assign Buffer ID to Target Data") indicates configuring the data attributes on the GPU side so that the target data is assigned a buffer ID. This buffer ID information can then be used by the cache unit 730 when performing cache control operations. Operation 723, as an operation prior to the execution of GPU task 1 (labeled "Before the execution of GPU task 1" for simplicity), takes input from operation 713, indicating that GPU task 1 is about to begin execution. An internal sub-operation 724 (shown as "Change the state of the buffer ID to the dedicated state") enables GPU driver 720 to issue an instruction before executing GPU task 1 to update the buffer ID state of the target data to the dedicated state. This state information is then passed to cache unit 730 via the cache control interface. Operation 725, as an operation prior to the execution of GPU task 2 (labeled "Before the execution of GPU task 2" for simplicity), takes input from operation 715, indicating that all usage of the target data for task 1 has been completed and GPU task 2 is about to begin. An internal sub-operation 726 (shown as "Change the state of the buffer ID to the freed state") enables GPU driver 720 to issue an instruction before executing GPU task 2 to update the buffer ID state of the target data to the freed state. Therefore, the corresponding cache quota meets the replacement condition.

[0052] The arrows pointing from operations 711, 713, and 715 to operations 721, 723, and 725 respectively indicate that upper-layer timing signals trigger corresponding GPU driver updates. Specifically, operation 711 triggers the buffer ID allocation in operation 721, operation 713 triggers the dedicated allocation for task 1 in operation 723, and operation 715 triggers the release state setting for task 2 in operation 725.

[0053] Furthermore, cache unit 730 can execute a series of operations 732, 734, and 736 corresponding to GPU driver operations 721, 723, and 725, respectively. These operations illustrate how cache unit 730 applies buffer ID allocation and buffer ID state transitions during instruction commit control. In operation 732, cache unit 730 receives an instruction from operation 721 and stores the buffer ID allocated to the target data for subsequent identification of its traffic, specifically determining whether any incoming access traffic belongs to the target data. In operation 734, cache unit 730 receives an instruction (e.g., an update) from operation 723 to notify cache unit 730 that the buffer ID for the target data has been transitioned to a private state, and cache unit 730 updates the state of the buffer ID to "private state" (e.g., in...). Figure 4B In the data attribute table 430 shown, cache unit 730 is assumed to be an example of cache unit 112. The arrows from operation 734 to operation 736 are labeled "Dedicated allocation when accessing data for buffer ID," indicating that when access traffic corresponding to a buffer ID is detected, cache unit 730 allocates cache space in response, storing the data associated with the buffer ID in the corresponding cache quota. For example, such dedicated allocation may involve cache unit 730 with the GPU and external memory (e.g., respectively in…). Figure 4A and Figure 4B This refers to I / O interactions between either the GPU 420 or the memory 450 shown in the diagram. In operation 736, cache unit 730 receives an indication (e.g., an update) from operation 725 to notify cache unit 730 that the buffer ID has transitioned to a released state, and cache unit 730 updates the state of the buffer ID to "released state" (e.g., in...). Figure 4B In the data attribute table 430 shown, assuming cache unit 730 is an example of cache unit 112. Once in a released state, the cache quota of data associated with the buffer ID becomes replaceable (i.e., meets the replacement condition). The outer arrow of operation 736, labeled "Replace buffer ID data when accessing other data," indicates that when access traffic for other data requests cache allocation, cache unit 730 can replace the cache space corresponding to the released buffer ID data (i.e., its corresponding buffer ID state is released). This replacement process can involve I / O interactions between cache unit 730 and either the GPU or external memory (e.g., GPU 420 and memory 450 mentioned above), depending on the discard or write-back behavior.

[0054] It should also be noted that the timing configuration of the GPU driver 720 and cache unit 730 is not limited to... Figure 7The embodiments described above. In practical applications, the configuration timing of buffer ID settings and cache control operations, such as when the GPU driver 720 configures the buffer ID settings and when the cache unit 730 applies the corresponding cache control operations, may vary depending on the specific implementation of the GPU driver, graphics or computing API layer, or underlying hardware architecture.

[0055] Figure 8 A channel control scheme according to an embodiment of the present disclosure is illustrated, wherein the GPU driver 820 and cache unit 830 may serve as examples of GPU driver 401 and cache unit 112, respectively, and can operate under the control of upper-layer software components, such as upper-layer software stack 810, including but not limited to applications, game engines, and / or graphics or computing API layers (labeled "APP / game engine / layer" for brevity), which are capable of issuing instructions to the GPU driver 820. Figure 8 As shown, the upper-layer software stack 810 can issue GPU instructions to the GPU driver 820 through an upper-layer graphics or computing language (such as Vulkan). The GPU driver 820 can interpret these instructions and communicate with the cache unit 830 through the lower-layer cache control interface. Through these interfaces, the buffer ID settings and cache control states (such as dedicated allocation or release) are passed to the cache unit 830.

[0056] like Figure 8As further shown, the upper-layer software stack 810 can execute a series of operations 811 and 813, each with a corresponding API-level sub-operation, denoted as operations 812 and 814, respectively. This embodiment illustrates channel-wise control, where cache control is based on predefined channel types (e.g., depth-access traffic) rather than on data objects. In the case of channel-wise allocation, buffer IDs are not assigned to specific data objects. Instead, buffer IDs may be allocated later based on the associated GPU task. Since buffer IDs are not limited to specific data, there is no need to update the data attribute table 430 first. For example, dedicated allocation behavior may apply to all access traffic of a specified channel type, but this disclosure is not limited thereto. In particular, operation 811 represents the stage at which the upper-layer software stack 810 records instructions for GPU task 1, and the internal sub-operation 812 (denoted as "+API: Set subsequent depth-access traffic to dedicated allocation") causes the upper-layer software stack 810 to issue an API-level request to indicate that all subsequent traffic corresponding to the specified channel, including but not limited to depth-access traffic of GPU task 1, should be treated as dedicated allocation in cache unit 830. In this channel-level configuration, the dedicated state applies to the entire access channel (e.g., depth test unit access), rather than a specific data object. Operation 813 indicates the stage at which the upper software stack 810 records GPU Task 2 instructions, and the internal sub-operation 814 (shown as "+API: Depth access traffic allocated to release quota") allows the upper software stack 810 to issue an API-level request to cause the channel-level buffer ID of the depth access traffic to transition to a released state. The arrow from operation 811 to operation 813 is labeled "Complete all usage of target data," indicating that GPU Task 2 is recorded after all depth access operations requiring dedicated allocation are completed in GPU Task 1.

[0057] Furthermore, GPU driver 820 can execute a series of operations 821 and 823 corresponding to upper-layer operations (such as operations 811 and 813). Each GPU driver operation (such as operations 821 and 823) includes internal sub-operations, shown as operations 822 and 824, respectively. Operation 821 generates instructions for GPU task 1, with input from operation 811, indicating that GPU task 1 has been recorded by upper-layer software stack 810 and that subsequent depth-access traffic should be dedicated allocation. Internal sub-operation 822 (shown as "Assign a buffer ID to the depth-access channel type and set its state to dedicated") causes GPU driver 820 to assign a buffer ID to the depth-access channel type and set the corresponding buffer ID state to dedicated for executing GPU task 1. This channel-level configuration allows cache unit 830 to process depth-access traffic based on the state of the buffer ID, such that all depth-access traffic occurring after the buffer ID is set to dedicated state (e.g., by task 1) and before the buffer ID is changed to free state (e.g., by task 2) is considered dedicated allocation. The tasks configuring dedicated and released states do not need to be consecutive, and the same strategy can be applied to depth access traffic generated by intermediate tasks. Operation 823 generates instructions for GPU task 2, with input from operation 813. Operation 813 indicates that the upper software stack 810 has completed the use of all depth access traffic requiring dedicated allocation and that GPU task 2 is being recorded. An internal sub-operation 824 (shown as "change the state of buffer ID to released state") enables the GPU driver 820 to generate instructions for GPU task 2 along with cache control instructions to change the state of the depth channel-level buffer ID to the released state. Therefore, the cache quota associated with the depth access traffic becomes replaceable by other data.

[0058] The arrows from operations 811 and 813 to operations 821 and 823 indicate that the upper-level channel cache behavior configuration is passed to the GPU driver 820. Operation 811 triggers the dedicated channel configuration in operation 821, while operation 813 triggers the release channel configuration in operation 823.

[0059] Furthermore, cache unit 830 can execute a series of operations 832 and 834 corresponding to GPU driver operations (such as operations 821 and 823), respectively. These operations illustrate the application of channel-level buffer ID control in cache unit 830 when a buffer ID is assigned to a channel type rather than a specific data object. Before operation 832, it should be noted that in the case of channel-level control, there is no need to initially update the data attribute table because the buffer ID is not limited to a specific data object. Instead, the buffer ID can be applied to all access traffic of the corresponding channel type as a GPU task is issued. In operation 832, cache unit 830 receives an instruction from operation 821, which assigns a buffer ID to the deep access channel type and sets its state to a dedicated state; and cache unit 830 stores the buffer ID and its dedicated state, enabling cache unit 830 to identify when access traffic belongs to that channel type. The arrow from operation 832 to operation 834 is labeled "Dedicated allocation when access traffic for buffer ID arrives," indicating that when access traffic corresponding to a specified channel type is detected, cache unit 830 can allocate cache space for the data of the channel-level buffer ID according to a dedicated allocation strategy. For example, this allocation may involve cache unit 830 with the GPU and external memory (e.g.,...). Figure 4A and Figure 4B I / O interaction between either GPU 420 or memory 450 (shown separately). In operation 834, cache unit 830 receives an indication (e.g., an update) from operation 823 to notify cache unit 830 that the state of the channel-level buffer ID has changed to a released state, and cache unit 830 updates the state of the buffer ID to "released state" (e.g., in...). Figure 4B In the data attribute table 430 shown (in the example case where cache unit 830 is cache unit 112), once in the released state, the cache quota associated with the data of the channel-level buffer ID becomes replaceable. The arrow pointing outward from operation 834 is labeled "Replace the data of the buffer ID when access traffic for other data arrives," indicating that when access traffic for other data requests cache allocation, cache unit 830 can replace the cache space corresponding to the released buffer ID data. This replacement can involve I / O interactions between cache unit 830 and either the GPU or external memory (such as GPU 420 and memory 450 mentioned above), depending on the discard or write-back behavior.

[0060] Figure 9 The main workflow of the method according to embodiments of this disclosure is illustrated. The above-described apparatus for dynamic cache control, for example... Figure 1 The electronic device 100 shown can be configured according to Figure 9 Follow the workflow shown.

[0061] In step S11, the electronic device 100 (e.g., the processing unit 111 in the electronic device 100) generates a first instruction for the first processing stage (e.g., Figure 5 The instructions for GPU task 1 shown are configured to allocate a buffer ID to target data of a special allocation type.

[0062] In step S12, the electronic device 100 (e.g., the cache unit 112 in the electronic device 100) sets the state of the buffer ID to a dedicated allocation state to allow the target data to be stored in the cache space corresponding to the buffer ID.

[0063] In step S13, the electronic device 100 (e.g., the processing unit 111 in the electronic device 100) generates a second instruction for the second processing stage (e.g., Figure 5 The instructions for GPU task 2 shown are configured to change the state of the buffer ID to a released state.

[0064] In step S14, the electronic device 100 (e.g., the cache unit 112 in the electronic device 100) sets the state of the buffer ID to a released state to allow the target data stored in the cache space / quota corresponding to the buffer ID to be replaced.

[0065] To better understand this method, it can be... Figure 9 The workflow shown is for illustrative purposes only, but this disclosure is not limited thereto. According to some embodiments, it is possible to... Figure 9 Add, delete, or change one or more steps in the workflow shown.

[0066] According to some implementations, dynamic cache control can be used not only by the graphics processing unit (GPU) but also by other processing units, and cache quotas can be shared among different processing units. This provides two advanced use cases as described below: (1) Dynamic cache control used independently by at least one other processing unit: Another processing unit (e.g., NPU) may develop or expose its own control application programming interface (API) to interface with the dynamic cache control mechanism. For example, in this scenario, the NPU may allocate data in a dedicated cache, which is accessed by the NPU and subsequently replaced by other data according to the cache unit's replacement policy, thereby enabling non-GPU processing units to benefit from, but not limited to: cache residency control for frequently accessed NPU data, reduced bandwidth during neural network inference or training, and use-and-discard efficiency when the NPU subsequently transitions the state of the buffer identifier (ID) to release; (2) Data sharing across processing units via caching: Dynamic cache control also allows for direct data sharing across processing units via caching, without the need for data exchange through external dynamic random access memory (DRAM). For example, data can first be dedicated to the cache by the GPU, and then accessed by the GPU and / or NPU through the cache (because both the GPU and NPU can recognize the same buffer ID or channel specification). Finally, the cache quota can be released by the NPU. In this case, data transfer between the GPU and NPU can be done directly through the cache without involving DRAM, thereby reducing overall memory bandwidth consumption and latency.

[0067] For the sake of brevity, similar descriptions of these implementation methods will not be repeated in detail here.

[0068] The foregoing outlines the features of several embodiments, enabling those skilled in the art to fully understand various aspects of this disclosure. Those skilled in the art should recognize that this disclosure provides a basis for designing or modifying other processes and structures to achieve substantially the same functionality and / or substantially the same results as the embodiments described herein. Furthermore, these equivalent solutions do not depart from the spirit and scope of this disclosure, and various changes, substitutions, and modifications can be made without departing from this disclosure.

Claims

1. A method for dynamic cache control, comprising: The processing unit generates a first instruction for the first processing stage, wherein the first instruction is configured to assign a buffer identifier ID to target data of a special allocation type; The cache unit sets the state of the buffer ID to a dedicated allocation state to allow the target data to be stored in the cache space corresponding to the buffer ID; The processing unit generates a second instruction for the second processing stage, wherein the second instruction is configured to change the state of the buffer ID to a released state; and The cache unit sets the state of the buffer ID to the released state to allow the target data stored in the cache space corresponding to the buffer ID to be replaced.

2. The method as described in claim 1, wherein, When the state of the buffer ID is set to the dedicated allocation state, the target data stored in the cache space corresponding to the buffer ID should not be replaced by subsequent data.

3. The method as described in claim 1, wherein, The replaced target data will not be written back to external memory or dynamic random access memory (DRAM).

4. The method of claim 1, wherein, This processing unit is a graphics processing unit (GPU).

5. The method of claim 1, wherein, The first processing stage and the second processing stage are different rendering channels, and the target data of the dedicated allocation type includes a depth image.

6. The method of claim 5, wherein, This second processing stage is the last rendering pass that uses the depth image.

7. The method of claim 1, wherein, The target data for this special allocation type includes all data accessed by one or a combination of block buffers and texture buffers.

8. The method of claim 1, wherein, The target data for this dedicated allocation type is associated with a predefined access channel type.

9. The method of claim 1, wherein, The processing unit is a neural processing unit (NPU), and the first processing stage and the second processing stage correspond to the operations of different layers of the neural network.

10. The method of claim 1, wherein, The state of the buffer ID is set to remain in the dedicated allocation state across multiple processing stages until the cache unit receives a signal to change the state of the buffer ID to the release state.

11. The method of claim 1, wherein, This dedicated allocation state is non-exclusive, allowing multiple buffer IDs to remain in this dedicated allocation state simultaneously within the same processing phase, with each buffer ID occupying a corresponding cache quota.

12. An apparatus for dynamic cache control, the apparatus comprising: At least one processing circuit, the processing circuit comprising: The processing unit is configured to perform at least one processing operation; and A cache unit is configured to perform at least one cache operation; Wherein: the processing unit is configured to generate a first instruction for the first processing stage, wherein the first instruction is configured to assign a buffer identifier ID to target data of a special allocation type; The cache unit is configured to set the state of the buffer ID to a dedicated allocation state, so that the target data can be stored in the cache space corresponding to the buffer ID; The processing unit is configured to generate a second instruction for the second processing stage, wherein the second instruction is configured to change the state of the buffer ID to a released state; and The cache unit is configured to set the state of the buffer ID to the released state, so that the target data stored in the cache space corresponding to the buffer ID can be replaced.

13. The apparatus of claim 1, wherein, When the state of the buffer ID is set to the dedicated allocation state, the target data stored in the cache space corresponding to the buffer ID should not be replaced by subsequent data.

14. The apparatus of claim 12, wherein, The replaced target data will not be written back to external memory or dynamic random access memory (DRAM).

15. The apparatus of claim 12, wherein, This processing unit is a graphics processing unit (GPU).

16. The apparatus of claim 12, wherein, The first processing stage and the second processing stage are different rendering channels, and the target data of the dedicated allocation type includes a depth image.

17. The apparatus of claim 12, wherein, The target data for this special allocation type includes all data accessed by one or a combination of block buffers and texture buffers.

18. The apparatus of claim 12, wherein, The target data for this dedicated allocation type is associated with a predefined access channel type.

19. The apparatus of claim 12, wherein, The processing unit is a neural processing unit (NPU), and the first processing stage and the second processing stage correspond to the operations of different layers of the neural network.

20. The apparatus of claim 12, wherein, The cache unit is located outside or inside the processing unit.

Images