Task processing method applied to GPU, GPU, device and storage medium

Abstract

Embodiments of the present application provide a task processing method applied to a GPU, a GPU, a device and a storage medium, and relate to the technical field of GPU task scheduling. The GPU comprises a front-end scheduling processor, a graphics subsystem, a resource allocator and a computing unit. The method comprises: the front-end scheduling processor sending a first CCS to the graphics subsystem, the first CCS comprising a graphics processing task; the graphics subsystem filtering a first type of computing task from the graphics processing task and sending the first type of computing task to the resource allocator, the first type of computing task being a computing task related to the graphics processing task; and the resource allocator sending the first type of computing task to the computing unit. The first type of computing task related to the graphics processing task is processed by the graphics subsystem, the graphics subsystem can independently execute a graphics rendering process, the switching out and switching in of the graphics subsystem is avoided, the power consumption of the GPU is reduced, and the efficiency of task processing is improved.

Description

Technical Field

[0001] This application relates to the field of GPU (Graphics Processing Unit) task scheduling technology, and in particular to a task processing method, GPU, device and storage medium applied to GPU. Background Technology

[0002] The GPU, as a major component of a computer device, includes a computing subsystem and a graphics subsystem. In graphics rendering scenarios, the front-end scheduling processor in the GPU schedules the computing tasks that are continuously generated during the graphics rendering process.

[0003] For example, the front-end scheduling processor breaks down the relevant tasks in the graphics rendering process, assigning graphics tasks to the graphics subsystem and computation tasks to the computation subsystem. If the graphics subsystem generates a new computation task during processing, the processing result of the graphics subsystem is paused and saved, the new computation task is assigned to the computation subsystem for processing, and then the result obtained by the computation subsystem is returned to the graphics subsystem to resume processing.

[0004] However, in the above method, the graphics subsystem needs to wait for the execution of the computation task, so scene switching is required, which incurs a large GPU overhead and affects the GPU's execution efficiency. Summary of the Invention

[0005] This application provides a task processing method, a GPU, a device, and a storage medium applied to a GPU. The technical solution is as follows:

[0006] According to one aspect of the embodiments of this application, a task processing method applied to a GPU is provided, the GPU comprising: a front-end scheduling processor, a graphics subsystem, a resource allocator, and a computing unit; the method comprising:

[0007] The front-end scheduling processor sends a first CCS (Command Control Stream) to the graphics subsystem, the first CCS including a graphics processing task;

[0008] The graphics subsystem selects a first type of computing task from the graphics processing task and sends it to the resource allocator. The first type of computing task is a computing task related to the graphics processing task.

[0009] The resource allocator sends the first type of computing tasks to the computing unit.

[0010] According to one aspect of the embodiments of this application, a GPU is provided, the GPU comprising: a front-end scheduling processor, a graphics subsystem, a resource allocator, and a computing unit;

[0011] The front-end scheduling processor is used to send a first CCS to the graphics subsystem, the first CCS including a graphics processing task;

[0012] The graphics subsystem is used to filter out a first type of computing tasks from the graphics processing tasks and send them to the resource allocator. The first type of computing tasks are computing tasks related to the graphics processing tasks.

[0013] The resource allocator is used to send the first type of computing tasks to the computing unit.

[0014] According to one aspect of the embodiments of this application, a task processing apparatus for a GPU is provided, the GPU comprising: a front-end scheduling processor, a graphics subsystem, a resource allocator, and a computing unit; the apparatus comprising:

[0015] The first sending module is used to send a first command control flow (CCS) to the graphics subsystem through the front-end scheduling processor, wherein the first CCS includes a graphics processing task.

[0016] The filtering module is used to filter out a first type of computing tasks from the graphics processing tasks through the graphics subsystem and send them to the resource allocator. The first type of computing tasks are computing tasks related to the graphics processing tasks.

[0017] The second sending module is used to send the first type of computing tasks to the computing unit through the resource allocator.

[0018] According to one aspect of the present application, a computer device is provided, the computer device including a processor and a memory, the memory storing a computer program, the computer program being loaded and executed by the processor to implement the above-described task processing method applied to a GPU.

[0019] According to one aspect of the embodiments of this application, a computer-readable storage medium is provided, wherein a computer program is stored in the computer-readable storage medium, the computer program being loaded and executed by a processor to implement the above-described task processing method applied to a GPU.

[0020] According to one aspect of the present application, a computer program product is provided, the computer program product including a computer program that is loaded and executed by a processor to implement the above-described task processing method applied to a GPU.

[0021] The technical solutions provided in this application embodiment may have the following beneficial effects:

[0022] The first type of computational tasks related to graphics processing are handled by the graphics subsystem. The graphics subsystem can execute the graphics rendering process independently without relying on the computation subsystem to perform related computational tasks. This avoids switching in and out of the graphics subsystem, which reduces the power consumption of the GPU and improves the efficiency of task processing. On the other hand, it eliminates the need to consider the synchronization between the graphics subsystem and the computation subsystem, thus avoiding synchronization problems between the two. Attached Figure Description

[0023] Figure 1 This is a structural diagram of a graphics processor provided in one embodiment of this application;

[0024] Figure 2 This is a structural diagram of the GPU core computing subsystem and graphics subsystem provided in one embodiment of this application;

[0025] Figure 3 This is a flowchart of the processing of a graphics subtask provided in one embodiment of this application;

[0026] Figure 4 This is a flowchart of a task processing method applied to a GPU according to an embodiment of this application;

[0027] Figure 5 This is a structural diagram of the GPU core computing subsystem and graphics subsystem provided in another embodiment of this application;

[0028] Figure 6 This is a schematic diagram of a graphics control engine provided in one embodiment of this application;

[0029] Figure 7 This is a schematic diagram of a computing task arbitrator provided in one embodiment of this application;

[0030] Figure 8 This is a flowchart illustrating a task processing method applied to a GPU according to an embodiment of this application;

[0031] Figure 9 This is a block diagram of a task processing device applied to a GPU according to one embodiment of this application;

[0032] Figure 10 This is a block diagram of a computer device provided in one embodiment of this application. Detailed Implementation

[0033] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0034] Please refer to Figure 1This diagram illustrates the structure of a symmetric multi-core full-featured graphics processor provided in an embodiment of this application. The GPU system includes multiple GPU cores (such as GPU cores #0 to #7) connected to the main processor 101 via a configuration bus 102.

[0035] Figure 1 The image shows a multi-core GPU system. A full-featured GPU is a complex multi-core system, and each GPU core contains the same functional modules. Each GPU core includes a front-end scheduling processor, as well as a graphics subsystem, a computing subsystem, and other subsystems such as 2D (Two-Dimensional), video, and DMA-engine (Direct Memory Access engine).

[0036] A typical computing subsystem and graphics subsystem within a GPU core are as follows: Figure 2 As shown, the pipeline processing differs significantly. The graphics subsystem 201 obtains graphics tasks from the graphics control engine and performs vertex processing, surface processing, domain processing, and rasterization, etc.; while the computation subsystem 202 obtains computation tasks from the computation control engine, splits the computation tasks, and finally combines them into computation threads before sending them to the downstream resource allocator 203. The resource allocator 203 and the computation unit (shader) 204 (including cache / local memory / registers, etc.) are shared by the graphics subsystem and the computation subsystem.

[0037] The GPU system operates by having the host processor 101's program send tasks to the front-end scheduling processor 205 within the GPU core via the operating system driver. For example... Figure 3 As shown, taking a graphics rendering task as an example, the front-end scheduling processor 205 breaks down the task. The graphics rendering part generates graphics subtasks and sends them to the graphics subsystem 201, while the calculation part generates calculation subtasks and sends them to the calculation subsystem 202. After the graphics subsystem 201 and the calculation subsystem 202 complete their individual execution, the front-end scheduling processor 205 generates a task completion event and sends it to the main processor. Figure 3 The solid lines in the diagram represent the processing flow of the graphics subtask, while the dashed lines represent the processing flow of the computation subtask.

[0038] Modern GPUs divide their submodules into computing subsystems and graphics subsystems, satisfying the two major application scenarios of AI (Artificial Intelligence) computing and graphics rendering. However, with the promotion of increasingly complex applications, graphics rendering scenarios involve many small-granular calculations (such as matrix operations on vertices during graphics processing). The front-end scheduling processor of current GPU systems needs to continuously generate computing tasks during the graphics rendering process.

[0039] According to the above processing flow, if the graphics subsystem 201 needs to execute a computational subtask while processing graphics subtasks, such as performing matrix operations on vertices during graphics processing, the current processing flow of the graphics subsystem 201 needs to be paused. The front-end scheduling processor 205 then schedules the computational subsystem 202 to execute the computational subtask. After the computational subtask is completed, the processing result is returned to the graphics subsystem 201, and the graphics subsystem 201 then resumes executing the graphics subtask under the scheduling of the front-end scheduling processor 205. This results in a situation where the current graphics subsystem needs to wait for the computational subsystem to perform scene switching and swapping. Scene switching requires the GPU to save some scene information, such as private registers and the current GPU pipeline execution position, to memory. Swapping in is the opposite of switching out. Both switching out and swapping in are very costly for the GPU, affecting its execution efficiency.

[0040] In addition, the characteristics of computational tasks such as vertex calculations in graphics processing are different from those of general AI computational tasks. Implementing specific splitting algorithms in the graphics subsystem can improve the performance of graphics computation. On the other hand, since the computational subsystem and the graphics subsystem belong to two independent systems, synchronization problems between the two may occur, and the priority arrangement of tasks will also affect performance.

[0041] To address this issue, this application proposes a task processing method for GPUs to solve the performance problem caused by the independent task scheduling of the graphics subsystem and the computing subsystem. This application provides an improved system design that incorporates a lightweight computing scheduling pipeline into the GPU graphics pipeline, merging computing tasks within the graphics subsystem, reducing task splitting by the front-end scheduling processor, reducing task ingress and egress, and improving the efficiency of GPU in processing application tasks.

[0042] Please refer to Figure 4 This document illustrates a flowchart of a task processing method applied to a GPU according to an embodiment of this application. The GPU includes a front-end scheduling processor, a graphics subsystem, a resource allocator, and computing units. The method includes at least one of the following steps 410-430.

[0043] Step 410: The front-end scheduling processor sends the first CCS to the graphics subsystem. The first CCS includes a graphics processing task.

[0044] In some embodiments, a graphics rendering process consists of at least one CCS, where the first CCS can be any one of the at least one CCS. For example, an image processing task includes at least one of the following CCSs: shader-update, draw, prediction, and fence. Shader-update involves hot loading of the shader program, such as Just-in-Time (JIT) compilation to translate the High Level Shading Language (HLSL / GLSL) into machine code, and dynamic resource binding under a unified shader architecture. Draw includes the activation of the complete rendering pipeline, such as vertex fetch, primitive assembly, rasterization, and pixel shading. Prediction is used to implement hardware-level branch optimization, such as early termination of invalid pixel calculations (through Early-Z testing) and speculative execution to reduce shader latency. The waiting synchronization fence is used to ensure the timing of cross-engine operations, such as data consistency between graphics and computing engines, and timing control of multi-frame rendering (such as triple buffer synchronization).

[0045] In some embodiments, the front-end scheduling processor is used to schedule the graphics subsystem to perform drawing tasks. Unlike related technologies, in this embodiment, the front-end scheduling processor divides both the first type of computational tasks and drawing tasks into graphics subsystem processing.

[0046] In some embodiments, the front-end scheduling processor sends a first CCS to the graphics subsystem, the first CCS including a graphics processing task. In some embodiments, the first CCS includes a graphics processing task. In some embodiments, the graphics processing task includes a drawing task and a first type of computational task, the first type of computational task being a computational task related to the drawing task. In some embodiments, the front-end scheduling processor determines at least one drawing task and at least one first type of computational task, and then sends the at least one drawing task and at least one first type of computational task to the graphics subsystem.

[0047] In some embodiments, the drawing tasks and the first type of computational tasks in the first CCS are distinguished and marked by the front-end scheduler. Exemplarily, the front-end scheduler adds a first parameter to the first CCS, which is used to distinguish the first type of computational tasks. Exemplarily, the first CCS includes 10 subtasks, where the 2nd and 5th are first type of computational tasks, and the rest are drawing tasks; the first parameter is used to indicate that the 2nd and 5th are first type of computational tasks.

[0048] In some embodiments, the first parameter may be carried in the first CSS to indicate a first type of computing task, and the graphics subsystem determines the first type of computing task based on the first parameter. In some embodiments, the first parameter may also be carried in the first type of computing task to identify the current task as a first type of computing task.

[0049] In some embodiments, the first type of computational task is a computational task related to graphics processing. In some embodiments, the first type of computational task is a computational task that needs to be processed during graphics rendering. For example, the first type of computational task is performing matrix operations on vertices during image processing.

[0050] Step 420: The graphics subsystem selects the first type of computing tasks from the graphics processing tasks and sends them to the resource allocator. The first type of computing tasks are computing tasks related to the graphics processing tasks.

[0051] In some embodiments, drawing tasks are handled by the graphics subsystem, while first-type computational tasks need to be sent to the resource allocator, which allocates appropriate computing units for processing.

[0052] In some embodiments, the graphics subsystem filters out a first type of computational tasks from graphics processing tasks and sends the first type of computational tasks to the resource allocator. Exemplarily, the graphics subsystem sends the first type of computational tasks to the resource allocator via sideband signals.

[0053] Step 430: The resource allocator sends the first type of computing task to the computing unit.

[0054] In some embodiments, the resource allocator sends the first type of computing tasks to the computing unit, which then processes them. In some embodiments, the resource allocator sends the first type of computing tasks to the computing unit in sequence. In some embodiments, the first type of computing tasks are ordered according to task priority.

[0055] The technical solution provided in this application embodiment assigns the first type of computing tasks related to graphics processing tasks to the graphics subsystem. The graphics subsystem can independently execute the graphics rendering process without relying on the computing subsystem to execute related computing tasks. This avoids switching in and out of the graphics subsystem, which reduces the power consumption of the GPU and improves the efficiency of task processing. On the other hand, it eliminates the need to consider the synchronization between the graphics subsystem and the computing subsystem, thus avoiding synchronization problems between the two.

[0056] The following section will provide a detailed explanation of the process by which the graphics subsystem handles the first type of computational task. Please refer to [link / reference]. Figure 5 It shows a structural diagram of the GPU core computing subsystem and graphics subsystem provided in one embodiment of this application.

[0057] In some embodiments, the graphics subsystem includes a graphics control engine 501 and a drawing task processing unit. In some embodiments, the graphics control engine 501 selects a first type of computing tasks from graphics processing tasks and sends them to a resource allocator. For example, as... Figure 5 As shown, the graphics control engine 501 includes a computational task splitting unit and a computational thread distribution unit. The computational task splitting unit and the computational thread distribution unit are used to process the first type of computational tasks.

[0058] In some embodiments, the graphics control engine includes a painting processing module, which comprises a computational task splitting unit and a computational thread distribution unit. In some embodiments, the painting processing module further includes a painting unit for processing drawing tasks, such as distributing drawing tasks to corresponding drawing task processing units. The drawing task processing units are used to perform drawing tasks such as vertex data reassembly, primitive assembly, and geometry shader processing.

[0059] For example, such as Figure 6 As shown, the graphics control engine 501 consists of a command parsing module 601, a drawing processing module 602, a status update module 603, a serialization controller 604, and a memory management unit.

[0060] The command parsing module 601 is responsible for receiving and decoding front-end scheduling instructions, converting high-level graphics API (Application Programming Interface) instructions (such as OpenGL / Vulkan commands) into micro-operations recognizable by the underlying hardware. This process involves mapping the instruction set architecture, similar to the instruction decoding stage of a CPU. The painting processing module 602 executes the core operations of the graphics pipeline, including: vertex data reassembly (such as coordinate transformation and normal calculation), primitive assembly (triangle / quadrilateral generation), and geometry shader processing (advanced operations such as tessellation). The state update module 603 dynamically maintains the rendering pipeline state machine, including: shader program switching (such as switching from vertex shader to fragment shader), texture binding state updates, and blending mode configuration (Alhpa blending, depth testing, etc.). The serialization controller 604 ensures that the command execution order conforms to the semantics of the graphics API, resolving resource contention issues in parallel computing. An ordered processing of asynchronous tasks is achieved through a command queue. The memory management unit adopts a hierarchical memory architecture, including: memory allocation strategy (such as block memory pool management), data prefetching mechanism (predicting texture loading requirements based on drawing commands), and cache consistency maintenance (synchronizing L1 / L2 cache with memory).

[0061] The graphics control engine primarily receives scheduling tasks from the front-end scheduler, retrieves the core processing unit (CCS) from memory for internal processing, such as reordering data streams, serialization, conditional waiting, and inter-core synchronization. Finally, it updates the data in memory and transmits commands to the next module via sideband signals for subsequent processing. Reordering data streams refers to topologically sorting out-of-order drawing commands, for example, reordering drawing commands for transparent objects by depth value and merging drawing batches in the same state to reduce state switching overhead. Conditional waiting refers to implementing the GPU's internal fence mechanism, such as cross-computation unit data dependency detection (e.g., output-input dependencies between shaders) and hardware semaphore management (ensuring that the rendering target is written before subsequent operations). Inter-core synchronization refers to using a ring buffer to distribute tasks among multiple computation units, dynamically load balancing (allocating tasks based on SM unit utilization), and supporting atomic operations (for counter synchronization).

[0062] Sideband signals refer to control signals transmitted via dedicated physical channels (not memory buses), such as interrupt notifications (e.g., rendering completion interrupts) and hardware event counters (used for performance analysis).

[0063] In some embodiments, the computing task splitting unit splits a first type of computing task into at least one first subtask group, the first subtask group including at least one first subtask; the computing thread distribution unit sends at least one first subtask group to the resource allocator.

[0064] In one example, the first subtask group is divided at the thread level. In another example, the first subtask group is divided at the thread group level.

[0065] For example, the first type of computing task includes M1 thread groups, and each thread group includes N1 threads. The computing task splitting unit splits the first type of computing task into M2 first subtask groups, and each first subtask group includes N2 first subtasks, where M1, M2, N1, and N2 are positive integers.

[0066] If the first subtask group is divided at the thread level, then each first subtask group includes N2 threads. In this case, the product of M2 and N2 is equal to the product of M1 and N1. If the first subtask group is divided at the thread group level, that is, each first subtask group includes N3 thread groups, and each thread group includes N4 threads, then the product of M2, N3, and N4 is equal to the product of M1 and N1. Here, N3 and N4 are positive integers.

[0067] In some embodiments, N1 and N4 can be the same or different. If N1 = N4, then the product of M2 and N3 equals M1. In some embodiments, the value of N4 can be a fixed value or dynamically adjusted based on the actual task.

[0068] It should be noted that the above description is only illustrative, assuming that each thread group and each subtask group contains the same number of threads and the same number of subtasks. In reality, the number of threads in different thread groups within the M1 thread groups may be the same or different. Similarly, the number of subtasks in different first subtask groups within the M2 first subtask groups may be the same or different.

[0069] In some embodiments, the graphics control engine selects drawing tasks from graphics processing tasks and sends them to the drawing task processing unit, which executes the drawing tasks. In some embodiments, the drawing task processing unit includes one or more of the following: a surface processing unit, a vertex processing unit, a domain processing unit, a rasterization processing unit, a domain shader, and a geometry shader.

[0070] In some embodiments, the graphics control engine further includes a state update module, which updates computation shader information to assist in performing the first type of computation task. For example, as... Figure 6As shown, the graphics control engine 501 includes a status update module 603.

[0071] In some embodiments, the state update module sends computation shader information to the resource allocator; the resource allocator sends the computation shader information to the computation unit; and the computation unit performs a first type of computation task based on the computation shader information.

[0072] In some embodiments, the computation shader information includes: the program location of the computation shader and register information; the computation unit determines the address of the computation shader based on the program location of the computation shader; the address of the computation shader is stored in a register according to the register information; and a first type of computation task is executed based on the register.

[0073] For example, when a compute shader is started, the operating system loads the program from the hard drive into memory. This process typically includes the following steps:

[0074] Reading program files: The operating system reads the executable file (such as an .exe file) of a program from the hard drive. This process is accomplished by the computing unit determining the storage address of the computing shader on the hard drive based on the program location of the computing shader, thereby reading the executable file of the computing shader.

[0075] Memory allocation: The operating system allocates memory space for programs, which typically includes code segment, data segment and stack segment.

[0076] Loading into memory: The program's code and data are copied into the allocated memory space.

[0077] After the computation shader is loaded into memory, the computation unit executes the first type of computation task. During the execution, the relevant data of the computation shader is loaded into the register based on the register information. After the first type of computation task is executed, the execution result is stored in the register or written back to memory.

[0078] Registers are fast storage units inside the CPU used for temporary storage of data and instructions. Accessing registers is much faster than accessing main memory; therefore, the CPU frequently uses registers to store and manipulate data during program execution.

[0079] Using the above method, a computational task splitting unit can be added to the graphics control engine of the graphics subsystem, based on the characteristics of graphics processing, to implement different splitting algorithms. Computational tasks are then issued by modifying parameters within the current graphics command control flow, directly sending the computational tasks required for graphics processing to the graphics subsystem. This allows for the processing of the first type of computational task without requiring switching in or out of the graphics subsystem.

[0080] In some embodiments, the GPU further includes a computing subsystem. In this embodiment, the computing subsystem only needs to handle the second type of computing tasks, and the processing flow of the computing subsystem will be described exemplarily below.

[0081] In some embodiments, the front-end scheduling processor sends a second CCS to the computing subsystem. The second CCS includes a second type of computing task, which is a computing task unrelated to the graphics processing task.

[0082] In some embodiments, the second CCS includes one or more second-type computational tasks. In some embodiments, the second-type computational tasks are computational tasks unrelated to the drawing task. Exemplarily, the second-type computational tasks are computational tasks related to program execution.

[0083] In some embodiments, the computing subsystem splits a second type of computing task into at least one second subtask group, the second type of computing task including at least one second subtask; the computing subsystem sends at least one second subtask group to the resource allocator.

[0084] In one example, the second subtask group is divided at the thread level. In another example, the second subtask group is divided at the thread group level.

[0085] For example, the second type of computing task includes M1 thread groups, and each thread group includes N1 threads. The computing task splitting unit splits the second type of computing task into M2 second subtask groups, and each second subtask group includes N2 second subtasks, where M1, M2, N1, and N2 are positive integers.

[0086] If the second subtask group is divided at the thread level, then each second subtask group includes N2 threads. In this case, the product of M2 and N2 is equal to the product of M1 and N1. If the second subtask group is divided at the thread group level, that is, each second subtask group includes N3 thread groups, and each thread group includes N4 threads, then the product of M2, N3, and N4 is equal to the product of M1 and N1. Here, N3 and N4 are positive integers.

[0087] In some embodiments, N1 and N4 can be the same or different. If N1 = N4, then the product of M2 and N3 equals M1. In some embodiments, the value of N4 can be a fixed value or dynamically adjusted based on the actual task.

[0088] It should be noted that the above description is only illustrative, assuming that each thread group and each subtask group contains the same number of threads and the same number of subtasks. In reality, the number of threads in different thread groups within the M1 thread groups may be the same or different. Similarly, the number of subtasks in different second subtask groups within the M2 second subtask groups may be the same or different.

[0089] In some embodiments, the resource allocator sends at least one second subtask group to a computing unit among a plurality of computing units.

[0090] In some embodiments, the resource allocator sends the second type of computing tasks to a computing unit among multiple computing units, which then processes the second type of computing tasks. In some embodiments, the resource allocator sends the second type of computing tasks to the computing units sequentially. In some embodiments, the order of the second type of computing tasks is based on task priority.

[0091] In some embodiments, the resource allocator needs to handle both the first type of computing tasks and the second type of computing tasks. Since the available physical computing resources are fixed, the resource manager needs to prioritize the first and second types of computing tasks to determine the computing units corresponding to the first and second types of computing tasks, respectively.

[0092] In some embodiments, the resource allocator sequentially sends at least one first-type computing task and at least one second-type computing task to corresponding computing units among a plurality of computing units. In some embodiments, the resource allocator includes a computing task arbitrator for determining the execution order of the first-type computing tasks and the second-type computing tasks.

[0093] In some embodiments, the computing task arbitrator arbitrates the first type of computing tasks and the second type of computing tasks based on a task arbitration algorithm to determine the execution order of the first type of computing tasks and the second type of computing tasks, as well as the corresponding computing units.

[0094] For example, such as Figure 7 As shown, the computation task arbitrator 700 receives a first type of computation task from the graphics subsystem and a second type of computation task from the computation subsystem. Then, based on a task arbitration algorithm, it arbitrates the first and second types of computation tasks and schedules them based on the arbitration result. The arbitration result determines the execution order of the first and second types of computation tasks.

[0095] In some embodiments, arbitrating the first type of computing task and the second type of computing task means determining the execution order of the first type of computing task and the second type of computing task, as well as the corresponding computing units, based on the priority ranking of the first type of computing task and the second type of computing task.

[0096] In some embodiments, the task arbitration algorithm relates to at least one of the following: the priority relationship between the first type of computing tasks and the second type of computing tasks, and the load status of the computing units. In some embodiments, the priority relationship between the first type of computing tasks and the second type of computing tasks is determined by the front-end scheduler. Exemplarily, the priorities of the first type of computing tasks and the second type of computing tasks are determined according to the identifier for the first type of computing tasks in the first CCS and the identifier for the second type of computing tasks in the second CCS.

[0097] In some embodiments, the priority relationship between the first type of computing tasks and the second type of computing tasks is determined by the order in which the resource allocator receives the first type of computing tasks and the second type of computing tasks. For example, if the resource allocator receives the first type of computing tasks first, then the first type of computing tasks are considered to have a higher priority than the second type of computing tasks.

[0098] In some embodiments, the priority relationship between the first type of computing tasks and the second type of computing tasks is predefined, for example, the priority of the first type of computing tasks is predefined to be higher than that of the second type of computing tasks.

[0099] In some embodiments, the GPU includes at least one computing unit. When the GPU includes multiple computing units, the resource allocator can divide the multiple computing units into at least one computing unit group based on the arbitration result of the computing task arbitrator for a first type of computing task and a second type of computing task. Each computing unit group is used to execute a computing task (including a first type of computing task and a second type of computing task), and each computing unit group includes at least one computing unit. The order in which the at least one computing unit group executes the computing tasks is determined according to the execution order of the first type of computing task and the second type of computing task, thus achieving load balancing among the multiple computing units.

[0100] In some embodiments, the resource allocator obtains at least one first sub-task group as a first type of computational task from the graphics subsystem, and at least one second sub-task group as a second type of computational task from the computation subsystem. The resource allocator sequentially sends the at least one first sub-task group and the at least one second sub-task group to corresponding computation units among a plurality of computation units. Exemplarily, a computational task arbitrator arbitrates the at least one first sub-task group and the at least one second sub-task group. Exemplarily, after arbitrating the at least one first sub-task group and the at least one second sub-task group, the computational task arbitrator schedules the at least one first sub-task group and the at least one second sub-task group based on the arbitration result.

[0101] Using the above method, on the one hand, the computational subsystem no longer needs to handle additional computational tasks from the graphics rendering process. On the other hand, it achieves dynamic arbitration between the first type of computational tasks of the graphics subsystem and the second type of computational tasks of the computational subsystem, enabling optimization of system performance based on the application scenario.

[0102] Compared with related technologies, the embodiments of this application differ in at least the following aspects.

[0103] On the one hand, a new CCS was defined, and the definition of the computation task was added by modifying the plotting command parameters. For example, the size of the requested computational resource work-group x, y, z was defined (here x, y, z is a three-dimensional quantitative representation).

[0104] On the other hand, a state update command has been added, which adds information about the compute shader related to the computation task in the drawing command. For example, it defines the location of the shader program and the required registers.

[0105] On the other hand, the graphics control engine was modified, and a computation task splitting unit and a computation thread distribution unit were added, implementing a splitting algorithm in the graphics subsystem that meets the characteristics of graphics processing.

[0106] On the other hand, a computing task arbitrator has been added to the resource allocator to schedule and arbitrate the defined computing tasks of the graphics subsystem and the second type of computing tasks of the computing subsystem, so as to realize an arbitration algorithm that can be set according to different scenarios.

[0107] Please refer to Figure 8 The diagram illustrates a flowchart of a task processing method for GPUs provided in one embodiment of this application.

[0108] For example, the front-end scheduler sends graphics processing tasks (drawing tasks (shown by the thin solid line in the figure) and the first type of computing tasks used in the drawing tasks (shown by the dashed line in the figure)) to the graphics control engine through the same first CCS. The graphics control engine parses and splits the tasks. If it is a first type of computing task, it is split by the computing task splitting unit and the computing thread distribution unit and sent to the computing task arbitrator.

[0109] The front-end scheduling processor sends the second type of computing tasks of the computing program (shown by the thick solid line in the figure) to the computing control engine, which then parses and splits them before sending them to the computing task arbitrator.

[0110] The computing task arbitrator queues up the computing tasks according to the set computing task arbitration algorithm and sends the first type of computing tasks and the second type of computing tasks to the computing unit according to priority.

[0111] Of course, the actual system is more complex than the diagram shows. For example, the arbitration of computational tasks needs to return the processing progress (such as the number of work groups / work threads processed) to the graphics subsystem and the computational subsystem.

[0112] This application also provides a GPU. In some embodiments, the GPU includes: a front-end scheduling processor, a graphics subsystem, a resource allocator, and a computing unit.

[0113] The front-end scheduling processor is used to send the first command control flow (CCS) to the graphics subsystem, and the first CCS includes graphics processing tasks.

[0114] In some embodiments, graphics processing tasks include drawing tasks and a first type of computational task.

[0115] In some embodiments, the graphics subsystem includes a graphics control engine and a drawing task processing unit. The graphics control engine is used to filter out a first type of computational tasks from the graphics processing tasks and send them to the resource allocator; the graphics control engine is also used to filter out drawing tasks from the graphics processing tasks and send them to the drawing task processing unit, which is used to execute the drawing tasks.

[0116] In some embodiments, the graphics control engine includes a drawing processing module, which includes a computational task splitting unit and a computational thread distribution unit. The computational task splitting unit is used to split a first type of computational task to obtain at least one first subtask group, wherein the first type of computational task includes at least one first subtask; the computational thread distribution unit is used to send at least one first subtask group to the resource allocator.

[0117] In some embodiments, the first subtask group is divided at the thread level; or, the first subtask group is divided at the thread group level.

[0118] In some embodiments, the graphics control engine includes a state update module, which updates computation shader information to assist in the execution of a first type of computation task. The state update module sends the computation shader information to a resource allocator; the resource allocator sends the computation shader information to a computation unit; and the computation unit executes the first type of computation task based on the computation shader information.

[0119] In some embodiments, the computation shader information includes: the program location of the computation shader and register information. The computation unit is configured to determine the address of the computation shader based on the program location of the computation shader; the computation unit is also configured to store the address of the computation shader in a register according to the register information; the computation unit is also configured to perform a first type of computation task based on the register.

[0120] The graphics subsystem is used to filter out the first type of computing tasks from the graphics processing tasks and send them to the resource allocator. The first type of computing tasks are computing tasks related to the graphics processing tasks.

[0121] The resource allocator is used to send the first type of computing tasks to the computing unit.

[0122] In some embodiments, the GPU further includes a computing subsystem. This computing subsystem is used to process a second type of computing task, which is a computing task unrelated to graphics processing tasks.

[0123] For example, the front-end scheduling processor is further configured to send a second CCS to the computing subsystem, the second CCS including a second type of computing task, which is a computing task unrelated to the drawing task; the computing subsystem is configured to split the second type of computing task to obtain at least one second subtask group, the second type of computing task including at least one second subtask; the computing subsystem is further configured to send at least one second subtask group to the resource allocator; the resource allocator is further configured to send at least one second subtask group to the computing unit.

[0124] In some embodiments, the GPU further includes a computing subsystem. The computing subsystem is used to send at least one second type of computing task to a resource allocator, the second type of computing task being a computing task unrelated to the drawing task; the resource allocator is also used to sequentially send at least one first type of computing task and at least one second type of computing task to the corresponding computing unit.

[0125] In some embodiments, the resource allocator includes a computing task arbitrator. The computing task arbitrator determines the execution order of a first type of computing task and a second type of computing task. In some embodiments, the computing task arbitrator arbitrates the first and second types of computing tasks based on a task arbitration algorithm to determine the execution order of the first and second types of computing tasks, as well as the corresponding computing units.

[0126] In some embodiments, the task arbitration algorithm relates to at least one of the following: the priority relationship between the first type of computing tasks and the second type of computing tasks, and the load of the computing unit.

[0127] For the functions of the various modules and units of the GPU, please refer to the description in the above embodiments of the task processing method applied to the GPU, which will not be repeated here.

[0128] Please refer to Figure 9 This diagram illustrates a block diagram of a task processing apparatus applied to a GPU according to an embodiment of this application. The apparatus has the functionality to implement the method examples described above; this functionality can be implemented in hardware or by hardware executing corresponding software. The apparatus can be the GPU described above, or it can be located within a GPU. The GPU includes: a front-end scheduling processor, a graphics subsystem, a resource allocator, and computing units. Figure 9 As shown, the device 900 includes: a first transmitting module 910, a filtering module 920, and a second transmitting module 930.

[0129] The first sending module 910 is used to send a first command control flow (CCS) to the graphics subsystem through the front-end scheduling processor, wherein the first CCS includes a graphics processing task.

[0130] The filtering module 920 is used to filter out a first type of computing tasks from the graphics processing tasks through the graphics subsystem and send them to the resource allocator. The first type of computing tasks are computing tasks related to the graphics processing tasks.

[0131] The second sending module 930 is used to send the first type of computing task to the computing unit through the resource allocator.

[0132] In some embodiments, the graphics subsystem includes: a graphics control engine and a drawing task processing unit; the filtering module 920 is configured to filter out the first type of computing tasks from the graphics processing tasks through the graphics control engine and send them to the resource allocator; the filtering module 920 is further configured to filter out the drawing tasks from the graphics processing tasks through the graphics control engine and send them to the drawing task processing unit, the drawing task processing unit being configured to execute the drawing tasks.

[0133] In some embodiments, the graphics control engine includes a drawing processing module, which includes a computational task splitting unit and a computational thread distribution unit; the filtering module 920 is used to split the first type of computational task by the computational task splitting unit to obtain at least one first sub-task group, the first type of computational task including at least one first sub-task; the computational thread distribution unit sends the at least one first sub-task group to the resource allocator.

[0134] In some embodiments, the first subtask group is divided at the thread level; or, the first subtask group is divided at the thread group level.

[0135] In some embodiments, the graphics control engine includes a state update module for updating computation shader information, which assists in executing the first type of computation task. The apparatus further includes a third sending module 940. The third sending module 940 is used to send the computation shader information to the resource allocator through the state update module; the resource allocator sends the computation shader information to the computing unit; and the computing unit executes the first type of computation task based on the computation shader information.

[0136] In some embodiments, the computation shader information includes: the program location of the computation shader and register information; the apparatus 900 further includes a computation module 950. The computation module 950 is configured to determine the address of the computation shader based on the program location of the computation shader using the computation unit; store the address of the computation shader in a register according to the register information; and execute the first type of computation task based on the register.

[0137] In some embodiments, the GPU further includes a computing subsystem, and the resource allocator includes a computing task arbitrator; the computing subsystem is used to process a second type of computing task, which is a computing task unrelated to the graphics processing task; the computing task arbitrator is used to determine the execution order of the first type of computing task and the second type of computing task.

[0138] In some embodiments, the apparatus further includes an arbitration module 960. The arbitration module 960 is configured to arbitrate the first type of computing tasks and the second type of computing tasks based on a task arbitration algorithm using the computing task arbitrator, to determine the execution order of the first type of computing tasks and the second type of computing tasks, and the corresponding computing units; wherein the task arbitration algorithm is related to at least one of the following: the priority relationship between the first type of computing tasks and the second type of computing tasks, and the load condition of the computing units.

[0139] The technical solution provided in this application embodiment assigns the first type of computing tasks related to graphics processing tasks to the graphics subsystem. The graphics subsystem can independently execute the graphics rendering process without relying on the computing subsystem to execute related computing tasks. This avoids switching in and out of the graphics subsystem, which reduces the power consumption of the GPU and improves the efficiency of task processing. On the other hand, it eliminates the need to consider the synchronization between the graphics subsystem and the computing subsystem, thus avoiding synchronization problems between the two.

[0140] It should be noted that the above embodiments only illustrate the division of the above functional modules when implementing the device. In actual applications, the above functions can be assigned to different functional modules according to actual needs, that is, the content structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0141] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0142] Please refer to Figure 10 This illustration shows a schematic diagram of a computer device according to an embodiment of this application. This computer device can be used to implement the task processing method applied to a GPU provided in the above embodiments. The computer device 1000 is a computer device. Specifically:

[0143] The computer device 1000 includes a processor (such as a CPU (Central Processing Unit), GPU (Graphics Processing Unit), and FPGA (Field Programmable Gate Array)) 1001, a memory 1002 including RAM (Random-Access Memory) and ROM (Read-Only Memory), and a system bus connecting the memory 1002 and the processor 1001. The computer device 1000 also includes a basic input / output system (I / O system) to facilitate information transfer between various devices within the server, and a large-capacity storage device for storing the operating system, system memory, and other program modules.

[0144] In some embodiments, the basic input / output system includes a display for showing information and an input device such as a mouse or keyboard for user input. Both the display and the input device are connected to a central processing unit via an input / output controller connected to a system bus. The basic input / output system may also include an input / output controller for receiving and processing input from multiple other devices such as a keyboard, mouse, or electronic stylus. Similarly, the input / output controller also provides output to a display screen, printer, or other types of output devices.

[0145] The mass storage device is connected to the processor 1001 via a mass storage controller (not shown) connected to the system bus. The mass storage device and its associated computer-readable media provide non-volatile storage for the computer device. That is, the mass storage device may include computer-readable media (not shown) such as a hard disk or a CD-ROM (Compact Disc Read-Only Memory) drive.

[0146] Without loss of generality, the computer-readable medium may include computer storage media and communication media. Computer storage media include volatile and non-volatile, removable and non-removable media implemented using any method or technology for storing information such as computer-readable instructions, data structures, program modules, or other data. Computer storage media include RAM, ROM, EPROM (Erasable Programmable Read-Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), flash memory or other solid-state storage technologies, CD-ROM, DVD (Digital Video Disc) or other optical storage, magnetic tape cassettes, magnetic tape, disk storage, or other magnetic storage devices. Of course, those skilled in the art will recognize that the computer storage medium is not limited to the above-mentioned types. The aforementioned memory 1002 and mass storage devices can be collectively referred to as memory.

[0147] According to an embodiment of this application, the computer device 1000 can also be connected to a remote computer on a network, such as the Internet, for operation. That is, the computer device 1000 can be connected to a network via a network interface unit connected to the system bus, or it can use the network interface unit to connect to other types of networks or remote computer systems (not shown).

[0148] The memory also includes a computer program stored in the memory and configured to be executed by one or more processors to implement the task processing method described above for the GPU.

[0149] In an exemplary embodiment, a computer-readable storage medium is also provided, wherein a computer program is stored therein, and the computer program, when executed by a processor of a computer device, implements the above-described task processing method applied to a GPU.

[0150] Optionally, the computer-readable storage medium may include: ROM (Read-Only Memory), RAM (Random-Access Memory), SSD (Solid State Drives), or optical disc, etc. The random access memory may include ReRAM (Resistance Random Access Memory) and DRAM (Dynamic Random Access Memory).

[0151] In an exemplary embodiment, a computer program product is also provided, the computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the task processing method applied to the GPU described above.

[0152] It should be understood that "multiple" as used in this article refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0153] The above description is merely an exemplary embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A task processing method applied to a graphics processing unit (GPU), characterized in that, The GPU includes: a front-end scheduling processor, a graphics subsystem, a computing subsystem, a resource allocator, and computing units; the method includes: The front-end scheduling processor sends a first command control flow (CCS) to the graphics subsystem, the first CCS including graphics processing tasks; The graphics subsystem selects a first type of computing task from the graphics processing task and sends it to the resource allocator. The first type of computing task is a computing task related to the graphics processing task. The computing subsystem is used to process a second type of computing task, which is a computing task unrelated to the graphics processing task. The resource allocator sends the first type of computing tasks and the second type of computing tasks to the computing unit.

2. The method according to claim 1, characterized in that, The graphics subsystem includes: a graphics control engine and a drawing task processing unit; The graphics control engine selects the first type of computing tasks from the graphics processing tasks and sends them to the resource allocator; The graphics control engine selects drawing tasks from the graphics processing tasks and sends them to the drawing task processing unit, which then executes the drawing tasks.

3. The method according to claim 2, characterized in that, The graphics control engine includes a painting processing module, which includes a computational task splitting unit and a computational thread distribution unit. The computing task splitting unit splits the first type of computing task into at least one first subtask group, and the first subtask group includes at least one first subtask. The computing thread distribution unit sends the at least one first subtask group to the resource allocator.

4. The method according to claim 3, characterized in that, The first subtask group is divided at the thread level; or, The first subtask group is divided according to the granularity of thread groups.

5. The method according to claim 2, characterized in that, The graphics control engine includes a state update module, which is used to update computation shader information, and the computation shader information is used to assist in the execution of the first type of computation task. The state update module sends the compute shader information to the resource allocator; The resource allocator sends the computation shader information to the computation unit; The computing unit performs the first type of computing task based on the computing shader information.

6. The method according to claim 5, characterized in that, The compute shader information includes: the compute shader's program location and register information; The computing unit determines the address of the computing shader based on the program location of the computing shader; The computing unit stores the address of the computing shader in the register according to the register information; The computing unit performs the first type of computing task based on the register.

7. The method according to any one of claims 1 to 6, characterized in that, The resource allocator includes a computing task arbitrator; The computing task arbitrator is used to determine the execution order of the first type of computing task and the second type of computing task.

8. The method according to claim 7, characterized in that, The method further includes: The computing task arbitrator arbitrates the first type of computing task and the second type of computing task based on the task arbitration algorithm to determine the execution order of the first type of computing task and the second type of computing task, as well as the corresponding computing unit; The task arbitration algorithm is related to at least one of the following: the priority relationship between the first type of computing tasks and the second type of computing tasks, and the load status of the computing unit.

9. A graphics processing unit (GPU), characterized in that, The GPU includes: a front-end scheduling processor, a graphics subsystem, a computing subsystem, a resource allocator, and computing units; The front-end scheduling processor is used to send a first command control flow (CCS) to the graphics subsystem, wherein the first CCS includes graphics processing tasks. The graphics subsystem is used to filter out a first type of computing tasks from the graphics processing tasks and send them to the resource allocator. The first type of computing tasks are computing tasks related to the graphics processing tasks. The computing subsystem is used to process a second type of computing task, which is a computing task unrelated to the graphics processing task. The resource allocator is used to send the first type of computing tasks and the second type of computing tasks to the computing unit.

10. The GPU according to claim 9, characterized in that, The graphics subsystem includes: a graphics control engine and a drawing task processing unit; The graphics control engine is used to filter out the first type of computing tasks from the graphics processing tasks and send them to the resource allocator; The graphics control engine is also used to filter out drawing tasks from the graphics processing tasks and send them to the drawing task processing unit, which is used to execute the drawing tasks.

11. The GPU according to claim 10, characterized in that, The graphics control engine includes a painting processing module, which includes a computational task splitting unit and a computational thread distribution unit. The computing task splitting unit is used to split the first type of computing task to obtain at least one first subtask group, and the first subtask group includes at least one first subtask. The computing thread distribution unit is used to send the at least one first subtask group to the resource allocator.

12. The GPU according to claim 11, characterized in that, The first subtask group is divided at the thread level; or, The first subtask group is divided according to the granularity of thread groups.

13. The GPU according to claim 10, characterized in that, The graphics control engine includes a state update module, which is used to update computation shader information, and the computation shader information is used to assist in the execution of the first type of computation task. The status update module is used to send the compute shader information to the resource allocator; The resource allocator is used to send the computation shader information to the computation unit; The computing unit is used to perform the first type of computing task based on the computing shader information.

14. The GPU according to claim 13, characterized in that, The compute shader information includes: the compute shader's program location and register information; The computing unit is used to determine the address of the computing shader based on the program location of the computing shader; The computing unit is also configured to store the address of the computing shader in the register according to the register information; The computing unit is also used to perform the first type of computing task based on the register.

15. The GPU according to any one of claims 9 to 14, characterized in that, The resource allocator includes a computing task arbitrator; The computing task arbitrator is used to determine the execution order of the first type of computing task and the second type of computing task.

16. The GPU according to claim 15, characterized in that, The computing task arbitrator is used to arbitrate the first type of computing task and the second type of computing task based on the task arbitration algorithm, and to determine the execution order of the first type of computing task and the second type of computing task, as well as the corresponding computing unit. The task arbitration algorithm is related to at least one of the following: the priority relationship between the first type of computing tasks and the second type of computing tasks, and the load status of the computing unit.

17. A task processing device applied to a graphics processing unit (GPU), characterized in that, The GPU includes: a front-end scheduling processor, a graphics subsystem, a computing subsystem, a resource allocator, and a computing unit; the device includes: The first sending module is used to send a first command control flow (CCS) to the graphics subsystem through the front-end scheduling processor, wherein the first CCS includes a graphics processing task. The filtering module is used to filter out a first type of computing tasks from the graphics processing tasks through the graphics subsystem and send them to the resource allocator. The first type of computing tasks are computing tasks related to the graphics processing tasks. The second sending module is used to send the first type of computing tasks and the second type of computing tasks to the computing unit through the resource allocator, wherein the second type of computing tasks are computing tasks unrelated to the graphics processing tasks. The computing subsystem is used to process the second type of computing task.

18. A computer device, characterized in that, The computer device includes a processor and a memory, the memory storing a computer program that is loaded and executed by the processor to implement the task processing method applied to a graphics processing unit (GPU) as described in any one of claims 1 to 8.

19. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which is loaded and executed by a processor to implement the task processing method applied to a graphics processing unit (GPU) as described in any one of claims 1 to 8.

20. A computer program product, characterized in that, The computer program product includes a computer program that is loaded and executed by a processor to implement the task processing method applied to a graphics processing unit (GPU) as described in any one of claims 1 to 8.

Images