A multi-task parallel GPU graphics workstation task queuing and scheduling system

Abstract

The application discloses a kind of multitask parallel GPU graphics workstation task queuing scheduling systems, it is related to computer resource scheduling technical field.The application is parsed to each GPU task by obtaining GPU task and corresponding user service level, according to the fine-grained analysis result, it is decomposed into subtask with progressive dependency relationship or parallel execution relationship, and then constructs task directed acyclic graph, according to user service level division service area and allocation GPU resource, the GPU resource in service area is divided into computing subunit, and according to the spatial distribution management of computing subunit to task directed acyclic graph according to subunit spatial distribution position in service area, and the connection relationship of subtask in task directed acyclic graph constructs task transmission path, executes task transmission path, and dynamically adjusts GPU resource between each service area by resource sharing protocol in task transmission path process.

Description

Technical Field

[0001] This invention relates to the field of computer resource scheduling technology, specifically to a multi-task parallel GPU graphics workstation task queuing and scheduling system. Background Technology

[0002] With the widespread application of graphics workstations in fields such as 3D rendering, deep learning training, scientific computing, and video encoding / decoding, multi-user, multi-task parallel execution has become the norm for GPU resource scheduling. Existing GPU task scheduling methods mostly adopt whole-card granularity resource allocation and first-in-first-out queue scheduling mechanisms, which are difficult to meet the differentiated needs of users with different service levels for response latency and throughput.

[0003] On the one hand, whole-card granular allocation leads to low GPU resource utilization. Individual tasks often fail to fully utilize the GPU's computing power, while other tasks have to wait in queues. This results in fragmented waste of hardware resources such as streaming multiprocessor clusters, memory bandwidth, tensor cores, and L2 cache. In typical multi-tenant scenarios, the average GPU computing power utilization is less than 60%, and the utilization of memory bandwidth and tensor cores is even lower. On the other hand, existing scheduling methods lack fine-grained analysis of the computational dependencies within tasks, treating GPU tasks as indivisible wholes. This fails to leverage the parallelism between subtasks to improve execution efficiency or to perform precise resource matching based on the resource profiles of subtasks. Furthermore, service level isolation in multi-tenant scenarios mainly relies on priority sorting at the queue level, failing to implement tiered services at the spatial distribution level of physical resources. High-priority tasks may still experience latency jitter due to low-priority tasks occupying resources during resource contention. Elastic resource allocation mechanisms across service regions are also lacking, and there is a lack of dynamic resource migration capabilities based on real-time load, making it difficult to effectively address local resource bottlenecks caused by sudden loads. Therefore, there is an urgent need for a multi-task parallel GPU graphics workstation task queuing and scheduling system that can achieve fine-grained task parsing, spatial resource management, and cross-regional elastic allocation. Summary of the Invention

[0004] The purpose of this invention is to provide a multi-task parallel GPU graphics workstation task queuing and scheduling system to address the shortcomings in the prior art.

[0005] To achieve the above objectives, the present invention provides the following technical solution: A multi-task parallel GPU graphics workstation task queuing and scheduling system includes a task profile construction module, a GPU resource management and allocation module, a task delivery path planning module, and a GPU resource regulation module. The task profiling module is used to obtain GPU tasks and corresponding user service levels, perform fine-grained analysis on each GPU task, decompose it into subtasks with progressive dependencies or parallel execution relationships based on the fine-grained analysis results, and then construct a directed acyclic graph of tasks. The graphics task resource management module is used to divide service areas and allocate GPU resources according to user service levels, divide GPU resources within the service area into computing sub-units, and manage the spatial distribution of computing sub-units according to the task directed acyclic graph, while establishing a resource sharing protocol between service areas. The task delivery path planning module is used to construct a task delivery path based on the spatial distribution of sub-units within the service area and the connection relationship of sub-tasks in the directed acyclic graph of tasks. The GPU resource control module is used to execute the task delivery path and dynamically allocate GPU resources between service areas through the resource sharing protocol during the task delivery path process.

[0006] Furthermore, the process of obtaining GPU tasks and corresponding user service levels includes: Obtain the GPU task and the corresponding user service level identifier. The user service level identifier is pre-bound by the service level agreement of the user account and includes three types: high-priority interactive level, normal business level, and background batch processing level.

[0007] Furthermore, the process of decomposing the fine-grained parsing results into subtasks with progressive dependencies or parallel execution relationships includes: The task description of GPU tasks is statically analyzed and dynamically decomposed through fine-grained analysis, which includes decomposing the computation graph structure, operator call sequence, memory access mode and communication topology of the task layer by layer. The fine-grained parsing process first identifies the top-level execution stage of the GPU task, and then further decomposes each top-level execution stage into several subtasks with independent computational semantics. There are progressive dependencies and parallel execution relationships between the subtasks. The progressive dependency relationship means that the output of the previous subtask is used as the input of the next subtask, and the next subtask must wait for the previous subtask to complete before it can start. The parallel execution relationship means that there are no data dependencies between the subtasks, and they can be executed simultaneously on different computing resources.

[0008] Furthermore, the process of constructing the task-directed acyclic graph includes: The directed acyclic graph of the task consists of several directed task nodes, each of which corresponds to a subtask. The edges between the directed task nodes represent the data dependencies and result transmission relationships between the subtasks. The construction of the directed acyclic graph of tasks follows the acyclic constraint. If a circular dependency is detected, dependency resolution is triggered, and the common task content in the cycle is extracted into independent subtasks to break the cycle.

[0009] Furthermore, the process of dividing GPU resources within the service area into computing sub-units includes: Based on the user's service level, the GPU physical resources of the graphics workstation are divided into three service areas, corresponding to high-priority interactive level, normal business level and background batch processing level respectively. Within each service area, GPU resources are further divided into a guaranteed resource pool and a resource sharing pool. The guaranteed resource pool is used to meet the basic service quality commitments of users in the corresponding service area. It is configured with a minimum number of computing sub-units, memory quotas, and bandwidth limits. The computing sub-units in the guaranteed resource pool are only used by GPU tasks within this service area and do not participate in cross-region allocation. The computing subunits are constructed by refining the physical resources of the GPU in the graphics workstation from the whole card level, while the resource sharing pool consists of cross-regional, elastically deployable computing subunits used to absorb sudden loads and achieve resource complementarity.

[0010] Furthermore, the process of managing the spatial distribution of computational sub-units based on the task's directed acyclic graph includes: The computing sub-units within each service area are spatially distributed in a three-dimensional chessboard pattern. The X-axis of the three-dimensional chessboard represents streaming multiprocessor clusters or computing unit groups, the Y-axis represents memory channels and bandwidth partitions, and the Z-axis represents time slices or execution phases. Each chessboard square corresponds to a computing sub-unit. The X-axis coordinates reflect the computing power level and on-chip interconnect distance of the computing sub-units, the Y-axis coordinates reflect the memory channel ownership and L2 cache sharing relationship, and the Z-axis coordinates represent the logical division of the same hardware resource in different time slices.

[0011] Furthermore, the process of establishing resource-sharing agreements between service areas includes: The service area sharing protocol includes provisions on the usage rights, borrowing rules, priority of access, and settlement method of the resource sharing pool.

[0012] Furthermore, the process of constructing the task delivery path includes: Each subtask is treated as a chess piece, and an initial placement position is assigned on a three-dimensional chessboard. The assignment of the initial placement position follows the principles of service area affinity and resource matching. The service area affinity principle requires that chess pieces be placed in the logical blocks of the service area corresponding to their respective user service level. The resource matching principle requires that the resource capability profile of the chessboard square where the chess piece is placed meets the resource profile information constraints carried by the chess piece. Using the directed task nodes of the directed acyclic graph of tasks as vertices and the propagation cost between chessboard squares on a 3D chessboard as edge weights, an extended spatiotemporal graph is constructed, in which each directed task node generates a spatiotemporal vertex on each available chessboard square and each available time slice. Simultaneously, a constrained shortest path search is adopted to select the path with the smallest total propagation cost as the task delivery path. If the propagation costs are the same, the path that crosses the service area boundary the fewest times is selected as the task delivery path.

[0013] Furthermore, after the task delivery path is determined, a necessary time slice is reserved for each piece on the path. The length of the time slice is determined by adding a preset buffer time to the expected execution time of the piece. The buffer time ratio is generally 10%-20% of the expected execution time to absorb the prediction error of the execution time. Time slice reservations are made on the Z-axis of the 3D chessboard. Only one piece can occupy the same chessboard square in the same time slice. If multiple pieces request the same time slice for the same chessboard square, they are sorted from highest to lowest according to their service level. Pieces with higher service levels are given priority for reservation, while pieces with lower service levels are deferred to subsequent idle time slices.

[0014] Furthermore, the process of dynamically allocating GPU resources among different service regions based on the execution results of the task delivery path includes: The propagation of subtask results is simulated by the movement of chess pieces. After a subtask completes its calculation in a certain computing subunit, it passes the intermediate result to the next computing subunit on the path through shared video memory, NVLink, PCIe or network, while releasing the computing resources originally occupied. The method of passing intermediate results is automatically selected according to the hardware topology of the chessboard square where the upstream and downstream chess pieces are located. The status of each piece is continuously monitored, and the GPU resources of each computing subunit are adjusted in real time. The adjustment process begins with each service area to count the GPU resources required at present, determine whether the guaranteed resource pool of the corresponding service area meets the task requirements, and further determine whether there is resource redundancy based on the judgment result. The criterion for resource redundancy is the number of computing sub-units in the resource pool whose continuous idle time exceeds the redundancy threshold. Based on the judgment result, redundant computing sub-units are temporarily separated through the resource sharing pool and marked with the start and end times of their availability, so that they can be borrowed by other service areas. Then, the GPU resources between service areas are completed according to the resource sharing protocol between service areas.

[0015] The technical effects and advantages provided by the present invention in the above technical solution are as follows: 1. This invention decomposes GPU tasks into subtasks with progressive dependencies or parallel execution relationships by performing fine-grained analysis of GPU tasks and constructing a directed acyclic graph of tasks. Each subtask is encapsulated as a schedulable execution unit carrying resource profile information, thereby breaking the granularity limitations of traditional whole-card scheduling. It can fully explore the parallelism between subtasks, achieve precise matching of multi-dimensional resources such as computing power, memory, and bandwidth, effectively shorten task queuing time, and improve GPU resource utilization and overall execution efficiency in multi-task parallel scenarios to a certain extent.

[0016] 2. This invention implements multi-tenant isolation and hierarchical services from the queue level to the physical resource spatial distribution level through a three-dimensional chessboard-shaped spatial distribution management of computing sub-units and a resource sharing protocol between service areas. This ensures that the hierarchical design of the resource pool and the resource sharing pool not only meets the service quality commitments of different service levels, but also absorbs sudden loads and achieves resource complementarity through cross-regional elastic allocation, reducing latency jitter of high-priority tasks and improving scheduling stability in multi-task parallel scenarios of graphics workstations. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this invention. For those skilled in the art, other drawings can be obtained based on these drawings.

[0018] Figure 1 This is a system block diagram of the present invention.

[0019] Figure 2 This is a flowchart of the present invention. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0021] Please see Figure 1 and Figure 2 As shown, a multi-task parallel GPU graphics workstation task queuing and scheduling system includes a task profile construction module, a GPU resource management and allocation module, a task delivery path planning module, and a GPU resource regulation module. The task profiling module is used to obtain GPU tasks and corresponding user service levels, perform fine-grained analysis on each GPU task, decompose it into subtasks with progressive dependencies or parallel execution relationships based on the fine-grained analysis results, and then construct a directed acyclic graph of tasks. The graphics task resource management module is used to divide service areas and allocate GPU resources according to user service levels, divide GPU resources within the service area into computing sub-units, and manage the spatial distribution of computing sub-units according to the task directed acyclic graph, while establishing a resource sharing protocol between service areas. The task delivery path planning module is used to construct a task delivery path based on the spatial distribution of sub-units within the service area and the connection relationship of sub-tasks in the directed acyclic graph of tasks. The GPU resource control module is used to execute the task delivery path and dynamically allocate GPU resources between service areas through the resource sharing protocol during the task delivery path process.

[0022] The working principle of the present invention is illustrated below through examples: The task profile building module receives GPU tasks submitted from multiple user terminals in real time through the task access interface of the graphics workstation. Each GPU task carries a corresponding user service level identifier when it is submitted. The user service level identifier is pre-bound by the user account's service level agreement, including three types: high-priority interactive level, normal business level, and background batch processing level; The task profiling module performs format verification and integrity checks on the received GPU tasks. GPU tasks that pass the verification are entered into the parsing queue, while GPU tasks that fail the verification are returned with an error code. At the same time, the task access interface assigns a globally unique task identifier to each GPU task and records its submission time, submitting user identifier, and corresponding user service level to form a task registration record. For each GPU task in the parsing queue, a static analysis and dynamic subdivision of its task description are performed through fine-grained parsing. The fine-grained parsing includes decomposing the task's computation graph structure, operator call sequence, memory access mode, and communication topology layer by layer. The fine-grained parsing process first identifies the top-level execution stage of the GPU task, and then further decomposes each top-level execution stage into several subtasks with independent computational semantics. There are progressive dependencies and parallel execution relationships between the subtasks. The progressive dependency relationship means that the output of the previous subtask is used as the input of the next subtask, and the next subtask must wait for the previous subtask to be completed before it can start. The parallel execution relationship means that there are no data dependencies between the subtasks, and they can be executed simultaneously on different computing resources. The task profiling module determines the relationship between each pair of subtasks based on operator type, data flow direction, and control flow branch, and assigns a subtask identifier and its associated GPU task identifier to each subtask. For example, for an interactive 3D rendering task, it can be decomposed into stages such as viewport rendering, lighting calculation, and post-processing compositing. There is a progressive dependency between viewport rendering and lighting calculation, and viewport rendering subtasks of multiple viewports can be executed in parallel. For a deep learning training task, it can be decomposed into stages such as forward propagation, backward propagation, and gradient aggregation. There is a progressive dependency between forward propagation and backward propagation, and forward propagation subtasks of multiple data slices can be executed in parallel.

[0023] Furthermore, the task profiling module constructs a directed acyclic graph of tasks based on the subtasks and their interrelationships obtained from fine-grained parsing. The directed acyclic graph of the task consists of several directed task nodes, each of which corresponds to a subtask. The edges between the directed task nodes represent the data dependencies and result transmission relationships between the subtasks. The construction of the directed acyclic graph (DAG) follows the acyclic constraint. If a circular dependency is detected, dependency resolution is triggered, extracting the common task content in the loop into independent subtasks to break the loop. Each subtask is encapsulated as a schedulable execution unit, which carries resource profile information, including floating-point computation volume, video memory usage, video memory bandwidth requirements, PCIe / NVLink communication volume, expected execution time, deadline, user service level, and fault tolerance requirements. Each resource profile information is obtained from the number of multiply-accumulate operations of the subtask, the peak video memory usage of tensors, the video memory read / write rate, the upstream and downstream data transmission volume, historical execution log regression prediction, the overall deadline reverse amortization, and the statistics of checkpoints and re-execution strategies.

[0024] Furthermore, the GPU resource management module obtains the corresponding user service level identifier carried by the GPU task when it is submitted from the task profile building module in real time. Then, the GPU resource management module divides the GPU physical resources of the graphics workstation into three service areas according to the user service level, which correspond to high priority interactive level, ordinary business level and background batch processing level respectively. Each service area corresponds to an independent logical resource domain. The GPU resource allocation ratio of the service area is determined by the global resource configuration strategy. By default, the high-priority interactive level, the normal business level, and the background batch processing level are allocated no less than 40%, no less than 35%, and no more than 25% of the total GPU resources, respectively. The allocation ratio can be dynamically adjusted according to the peak business period. Within each service area, GPU resources are further divided into a guaranteed resource pool and a resource sharing pool. The guaranteed resource pool is used to meet the basic service quality commitments of users in the corresponding service area. It is configured with a minimum number of computing sub-units, memory quotas, and bandwidth limits. The computing sub-units in the guaranteed resource pool are only used by GPU tasks within this service area and do not participate in cross-region allocation. The computing subunit is constructed by refining the physical resources of the GPU in the graphics workstation from the whole card level. The computing subunit is divided according to the GPU hardware capabilities, including streaming multiprocessor clusters, memory channel partitions, L2 cache slices, NVENC / NVDEC encoding and decoding units, tensor core groups and time slice quotas, etc. The resource sharing pool consists of cross-regional, elastically deployable computing sub-units used to absorb sudden loads and achieve resource complementarity. When there is no load in the local service area, the computing sub-units in the resource sharing pool can be temporarily borrowed by other service areas. Each service area is equipped with a regional resource manager, which is responsible for ensuring the quota maintenance and usage statistics of the resource pool and the resource sharing pool. The computing sub-units within each service area are spatially distributed in a three-dimensional chessboard pattern. The X-axis of the three-dimensional chessboard represents streaming multiprocessor clusters or computing unit groups, the Y-axis represents memory channels and bandwidth partitions, and the Z-axis represents time slices or execution phases. Each chessboard square corresponds to a computing sub-unit, and its size and shape are dynamically adjusted according to computing, storage, and communication capabilities. The X-axis coordinates reflect the computing power level and on-chip interconnect distance of the computing sub-units, the Y-axis coordinates reflect the memory channel ownership and L2 cache sharing relationships, and the Z-axis coordinates represent the logical partitioning of the same hardware resource in different time slices. The service area forms continuous or discrete logical blocks on the chessboard. The block boundary corresponds to the quota isolation boundary, ensuring that the computing sub-units in the resource pool are located in the core of the block, and the computing sub-units in the resource sharing pool are located at the edge of the block, which facilitates cross-regional allocation. The shape of the logical block is dynamically shaped according to the resource quota and task load characteristics of the service area. When there are many computing-intensive tasks, the block expands in the X-axis direction, and when there are many memory-intensive tasks, the block expands in the Y-axis direction.

[0025] Furthermore, after the GPU resource management module completes the spatial distribution of service areas, it establishes a sharing protocol between service areas, which specifies the usage permissions, borrowing rules, preemption priority, and settlement method of the resource sharing pool. Regarding usage permissions, each service area has the right to use all computing sub-units in its own resource sharing pool, and to temporarily use idle computing sub-units in the resource sharing pools of other service areas when the borrowing conditions are met. Regarding the borrowing rules, the borrower must send a resource sharing request to the global resource sharing pool in advance. The request must include the service level, the type and quantity of the required computing sub-units, the expected start and end time of use, and the priority. The borrowing time must not exceed the preset maximum borrowing period, which is generally 500 milliseconds to 2 seconds. Regarding priority preemption, resource requests from high-priority interactive service areas can preempt computing sub-units that have been borrowed but not yet started in ordinary business-level and background batch processing-level service areas. Resource requests from ordinary business-level service areas can preempt computing sub-units that have been borrowed but not yet started in background batch processing-level service areas. When preemption occurs, the preempted party receives a priority compensation mark and will receive priority in subsequent resource allocation. Regarding the settlement method, the entire resource sharing pool records the amount of resources borrowed and borrowed in each service area, and dynamically balances the resource quota according to the settlement cycle. For service areas where the borrowing is consistently greater than the borrowing, the guaranteed resource pool quota will be appropriately increased in the next allocation cycle.

[0026] Furthermore, the GPU resource allocation module shares the service area network with the task delivery path planning module, which then treats each subtask as a chess piece and assigns an initial placement position on the three-dimensional chessboard. The allocation of the initial placement position follows the principles of service area affinity and resource matching. The service area affinity principle requires that chess pieces be placed in the logical blocks of the service areas corresponding to their respective user service levels. High-priority interaction-level chess pieces are placed in the guaranteed resource pool of high-priority interaction-level service areas, ordinary business-level chess pieces are placed in the guaranteed resource pool of ordinary business-level service areas, and background batch processing-level chess pieces are placed in the resource sharing pool of background batch processing-level service areas. The resource matching principle requires that the resource capability profile of the chessboard grid where the chess piece is placed meets the constraints of the resource profile information carried by the chess piece. That is, the available computing power, available video memory, and available video memory bandwidth of the chessboard grid are not less than the floating-point calculation amount required by the chess piece divided by the expected execution time, video memory usage, and video memory bandwidth requirements, respectively. For parallel subtasks, they can be distributed across different computing sub-units within the same service area and executed simultaneously. The initial placement position is preferably selected on adjacent checkerboard grids on the X-axis or Y-axis to reduce the synchronization overhead between parallel subtasks. For progressive subtasks, subsequent computing sub-units are planned sequentially according to the edges of the directed acyclic graph of the task. The initial placement position of upstream subtasks is preferably selected near checkerboard grids with higher interconnect bandwidth and shorter NUMA topology distance to facilitate result transfer.

[0027] Furthermore, considering the real-time load of the computing subunit, remaining video memory, interconnect bandwidth, NUMA topology distance, and service area boundaries, a task propagation path that satisfies resource constraints and has the lowest propagation cost is selected for each GPU task's directed acyclic graph. Using the directed task nodes in the directed acyclic graph as vertices, the propagation cost between grid squares on a 3D chessboard is considered. The calculation is obtained by weighted summation of computation cost, communication cost, and waiting cost, corresponding to the expected execution time of a piece on the chessboard, the estimated time for upstream and downstream pieces to transmit intermediate results via shared memory, NVLink, PCIe, or network, and the time waiting for the upstream piece to complete or for the chessboard to become idle. The calculation process includes:

[0028] in , , For the weighting coefficients, satisfying Default value , , It can be dynamically adjusted according to the type of GPU task (increase for computationally intensive tasks). The number of communication-intensive tasks is increasing. ); computational cost ,in The floating-point computation cost of the subtask. Calculate the effective computing power of sub-units for the target chessboard grid (calculated from real-time utilization); communication cost ,in This refers to the amount of intermediate result data between upstream and downstream subtasks (i.e., PCIe / NVLink communication volume). Effective bandwidth of the corresponding interconnect link (shared video memory > NVLink > PCIe > network); waiting cost That is, the larger of the remaining execution time of the upstream subtask and the idle waiting time of the target chessboard square; Simultaneously, a constrained shortest path search is adopted. During the search process, the resource capability profile of each spatiotemporal vertex is verified in real time to ensure that it meets the resource profile information constraints of the chess piece. The verification content includes whether the real-time utilization rate, remaining video memory, video memory bandwidth usage, temperature, and fault status are within their respective threshold ranges. For GPU tasks with multiple feasible paths, the path with the smallest total propagation cost is selected as the task delivery path. If the propagation costs are the same, the path that crosses the service area boundary the fewest times is selected as the task delivery path.

[0029] Furthermore, after the task delivery path is determined, a necessary time slice is reserved for each piece on the path. The length of the time slice is determined by adding a preset buffer time to the expected execution time of the piece. The buffer time ratio is generally 10%-20% of the expected execution time to absorb the prediction error of the execution time. Time slice reservations are made on the Z-axis of the 3D chessboard. Only one piece can occupy the same chessboard square in the same time slice. If multiple pieces request the same time slice for the same chessboard square, they are sorted according to their service level from highest to lowest. Pieces with higher service levels are given priority for reservation, while pieces with lower service levels are deferred to subsequent idle time slices. After all time slices for all pieces are reserved, a path execution plan is generated. The path execution plan includes the subtask identifier of each piece, the coordinates of the allocated chessboard square, the start and end times of the reserved time slice, the identifiers of upstream and downstream pieces, and the intermediate result transmission method. The path execution plan is sent to the corresponding area resource manager and computation subunit executor, which are used to lock the time slice of the corresponding chessboard square and prepare the input data and execution environment for the subtask, respectively.

[0030] Furthermore, after completing the task delivery path planning, the task delivery path planning module sends the task delivery path to the GPU resource control module. Furthermore, during the execution of the task delivery path, the GPU resource control module uses chess piece movement to simulate the propagation of subtask results. After a subtask completes its calculation in a certain computing subunit, it passes the intermediate result to the next computing subunit on the path through shared memory, NVLink, PCIe, or network, while releasing the computing resources originally occupied. The method of passing intermediate results is automatically selected according to the hardware topology of the chessboard grid where the upstream and downstream chess pieces are located. Within the same GPU, it is passed through shared memory; between GPUs with NVLink on the same node, it is passed through NVLink; between GPUs without NVLink on the same node, it is passed through PCIe; and across nodes, it is passed through network. Continuously monitor the status of each piece, record the start time, end time, actual resource consumption and propagation time of each subtask, and form a spatiotemporal execution sequence of the task transmission path. Actual resource consumption includes actual floating-point calculation, actual peak video memory, actual video memory bandwidth usage and actual PCIe / NVLink communication volume. Propagation time is the actual time taken for intermediate results to be transmitted from upstream pieces to downstream pieces. When multiple task delivery paths encounter spatiotemporal conflicts on a 3D chessboard, conflict resolution is performed. A spatiotemporal conflict occurs when two or more pieces request the same chessboard square in the same time slice, or when a piece's intermediate result delivery request consumes the interconnect bandwidth reserved for other pieces. Conflict resolution first identifies critical and non-critical paths in each task delivery path. The critical path is the path with the highest propagation cost in the directed acyclic graph of tasks; its delay will directly cause the GPU task deadline to be missed. Non-critical paths have a certain amount of slack time. For pieces on non-critical paths, methods such as delaying, detouring around adjacent empty sub-units, or backfilling are used. The strategies correspond to deferring the execution time slice, migrating to an adjacent empty chessboard square, and inserting execution using the empty time slices released in advance by other pieces. For pieces on the critical path, priority is given to ensuring their resource occupation. If a critical path piece conflicts with a non-critical path piece, the non-critical path piece executes the above-mentioned resolution strategy. If a conflict occurs between critical path pieces, they are sorted according to user service level from high to low. Critical path pieces with high service level occupy resources first, while critical path pieces with low service level trigger resource sharing requests and obtain additional computing sub-units from other service areas through the resource sharing protocol.

[0031] Furthermore, the GPU resource control module adjusts the GPU resources of each computing subunit in real time. The adjustment process begins with each service area compiling statistics on the currently required GPU resources. The statistics include the total resource profile information of the queued tasks in the area, the remaining resource requirements of the tasks currently being executed, and the pre-allocated resource requirements of the tasks that are about to arrive. Determine whether the guaranteed resource pool of the corresponding service area meets the task requirements. If the guaranteed resource pool meets the resources required by the current task, that is, the total computing power, total video memory and total bandwidth of the available computing subunits in the guaranteed resource pool are not lower than those required by the current task, then further check whether there is resource redundancy. The criterion for resource redundancy is the number of computing sub-units in the resource pool whose continuous idle time exceeds the redundancy threshold. The redundancy threshold is generally between 100 milliseconds and 300 milliseconds. If redundancy exists, the redundant computing sub-units are temporarily separated through the resource sharing pool and marked with the start and end times of their availability, so that they can be borrowed by other service areas. The block boundary marker of the separated redundant computing sub-units on the three-dimensional chessboard is changed from the core position of the resource pool to the edge position of the resource sharing pool, and its resource ownership marker is temporarily changed to the global resource sharing pool's reconfigurable state, but the service area retains the priority right to reclaim them when needed. If the resource pool cannot meet the current task requirements (e.g., all computing sub-units in the pool are occupied but there are still queued tasks in the region), the service region sends a resource sharing request to the global resource sharing pool. The request includes the service level, the type and quantity of the required computing sub-units, the estimated start and end times, and the priority. Upon receiving resource requests from multiple service regions, the resource sharing pool allocates resources based on each service region's service level, request time window, and current resource pool availability. The allocation process first sorts the requests by service level, with requests from higher service level regions receiving priority. For multiple requests of the same service level, priority is given to requests based on their priority. Priority is determined by the urgency of the request, with requests for GPU tasks approaching their deadline having higher priority. If time windows overlap, meaning multiple requests request the same computing subunit within the same time period, a preemption, partial allocation, or delayed fulfillment strategy is adopted. This corresponds to a high-service-level request preempting an allocated but unused computing subunit from a low-service-level request, allocating available computing subunits proportionally to the number of requests, and deferring the request to the next available time window, respectively. After successful allocation, the right to use the corresponding computing subunit is temporarily transferred to the request area, and the block boundaries and resource ownership markers are updated on the 3D chessboard. The borrowed area must complete its GPU task execution before the expected start and end time of use. After use, the borrowed area must return the resources on time, ensuring that all intermediate results on the computing sub-unit have been transferred to downstream pieces or written to persistent storage, and that the computing sub-unit's memory has been cleared. Upon receiving the returned computing sub-unit, the resource sharing pool will re-mark it as available, update its resource capability profile with real-time utilization, temperature, power consumption, memory bandwidth usage, and fault status, and restore its block boundary and resource ownership marker on the 3D chessboard to the edge position of the original service area's resource sharing pool. If the borrowed area cannot return the resource on time due to task execution timeout, it must send a renewal request to the resource sharing pool in advance, including the renewal duration and reason. The resource sharing pool will decide whether to approve the renewal based on the current availability of the resource pool and other waiting requests. If the renewal is approved, the start and end time of use will be updated. If not approved, the borrowed area must release the computing sub-unit before the original deadline, and any unfinished subtasks will be migrated to other available computing sub-units within the service area or placed in a queue.

[0032] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A multi-task parallel GPU graphics workstation task queuing and scheduling system, characterized in that, It includes a task profile building module, a GPU resource management and allocation module, a task delivery path planning module, and a GPU resource regulation module; The task profiling module is used to obtain GPU tasks and corresponding user service levels, perform fine-grained analysis on each GPU task, decompose it into subtasks with progressive dependencies or parallel execution relationships based on the fine-grained analysis results, and then construct a directed acyclic graph of tasks. The graphics task resource management module is used to divide service areas and allocate GPU resources according to user service levels, divide GPU resources within the service area into computing sub-units, and manage the spatial distribution of computing sub-units according to the task directed acyclic graph, while establishing a resource sharing protocol between service areas. The task delivery path planning module is used to construct a task delivery path based on the spatial distribution of sub-units within the service area and the connection relationship of sub-tasks in the directed acyclic graph of tasks. The GPU resource control module is used to execute the task delivery path and dynamically allocate GPU resources between service areas through the resource sharing protocol during the task delivery path process.

2. The multi-task parallel GPU graphics workstation task queuing and scheduling system according to claim 1, characterized in that, The process of obtaining GPU tasks and corresponding user service levels includes: Obtain the GPU task and the corresponding user service level identifier. The user service level identifier is pre-bound by the service level agreement of the user account and includes three types: high-priority interactive level, normal business level, and background batch processing level.

3. The multi-task parallel GPU graphics workstation task queuing and scheduling system according to claim 2, characterized in that, The process of decomposing the results of fine-grained parsing into subtasks with progressive dependencies or parallel execution relationships includes: The task description of GPU tasks is statically analyzed and dynamically decomposed through fine-grained analysis, which includes decomposing the computation graph structure, operator call sequence, memory access mode and communication topology of the task layer by layer. The fine-grained parsing process first identifies the top-level execution stage of the GPU task, and then further decomposes each top-level execution stage into several subtasks with independent computational semantics. There are progressive dependencies and parallel execution relationships between the subtasks. The progressive dependency relationship means that the output of the previous subtask is used as the input of the next subtask, and the next subtask must wait for the previous subtask to complete before it can start. The parallel execution relationship means that there are no data dependencies between the subtasks, and they can be executed simultaneously on different computing resources.

4. The multi-task parallel GPU graphics workstation task queuing and scheduling system according to claim 3, characterized in that, The process of constructing a directed acyclic graph for a task includes: The directed acyclic graph of the task consists of several directed task nodes, each of which corresponds to a subtask. The edges between the directed task nodes represent the data dependencies and result transmission relationships between the subtasks. The construction of the directed acyclic graph of tasks follows the acyclic constraint. If a circular dependency is detected, dependency resolution is triggered, and the common task content in the cycle is extracted into independent subtasks to break the cycle.

5. A multi-task parallel GPU graphics workstation task queuing and scheduling system according to claim 4, characterized in that, The process of dividing GPU resources within a service area into computing sub-units includes: Based on the user's service level, the GPU physical resources of the graphics workstation are divided into three service areas, corresponding to high-priority interactive level, normal business level and background batch processing level respectively. Within each service area, GPU resources are further divided into a guaranteed resource pool and a resource sharing pool. The guaranteed resource pool is used to meet the basic service quality commitments of users in the corresponding service area. It is configured with a minimum number of computing sub-units, memory quotas, and bandwidth limits. The computing sub-units in the guaranteed resource pool are only used by GPU tasks within this service area and do not participate in cross-region allocation. The computing subunits are constructed by refining the physical resources of the GPU in the graphics workstation from the whole card level, while the resource sharing pool consists of cross-regional, elastically deployable computing subunits used to absorb sudden loads and achieve resource complementarity.

6. A multi-task parallel GPU graphics workstation task queuing and scheduling system according to claim 5, characterized in that, The process of managing the spatial distribution of computational sub-units based on the task's directed acyclic graph includes: The computing sub-units within each service area are spatially distributed in a three-dimensional chessboard pattern. The X-axis of the three-dimensional chessboard represents streaming multiprocessor clusters or computing unit groups, the Y-axis represents memory channels and bandwidth partitions, and the Z-axis represents time slices or execution phases. Each chessboard square corresponds to a computing sub-unit. The X-axis coordinates reflect the computing power level and on-chip interconnect distance of the computing sub-units, the Y-axis coordinates reflect the memory channel ownership and L2 cache sharing relationship, and the Z-axis coordinates represent the logical division of the same hardware resource in different time slices.

7. A multi-task parallel GPU graphics workstation task queuing and scheduling system according to claim 6, characterized in that, The process of establishing a resource-sharing agreement between service areas includes: The service area sharing protocol includes provisions on the usage rights, borrowing rules, priority of access, and settlement method of the resource sharing pool.

8. A multi-task parallel GPU graphics workstation task queuing and scheduling system according to claim 7, characterized in that, The process of constructing the task delivery path includes: Each subtask is treated as a chess piece, and an initial placement position is assigned on a three-dimensional chessboard. The assignment of the initial placement position follows the principles of service area affinity and resource matching. The service area affinity principle requires that chess pieces be placed in the logical blocks of the service area corresponding to their respective user service level. The resource matching principle requires that the resource capability profile of the chessboard square where the chess piece is placed meets the resource profile information constraints carried by the chess piece. Using the directed task nodes of the directed acyclic graph of tasks as vertices and the propagation cost between chessboard squares on a 3D chessboard as edge weights, an extended spatiotemporal graph is constructed, in which each directed task node generates a spatiotemporal vertex on each available chessboard square and each available time slice. Simultaneously, a constrained shortest path search is adopted to select the path with the smallest total propagation cost as the task delivery path. If the propagation costs are the same, the path that crosses the service area boundary the fewest times is selected as the task delivery path.

9. A multi-task parallel GPU graphics workstation task queuing and scheduling system according to claim 8, characterized in that, Once the task delivery path is determined, a necessary time slice is reserved for each piece on the path. The length of the time slice is determined by adding a preset buffer time to the expected execution time of the piece. The buffer time ratio is generally 10%-20% of the expected execution time to absorb the prediction error of the execution time. Time slice reservations are made on the Z-axis of the 3D chessboard. Only one piece can occupy the same chessboard square in the same time slice. If multiple pieces request the same time slice for the same chessboard square, they are sorted from highest to lowest according to their service level. Pieces with higher service levels are given priority for reservation, while pieces with lower service levels are deferred to subsequent idle time slices.

10. A multi-task parallel GPU graphics workstation task queuing and scheduling system according to claim 9, characterized in that, The process of dynamically allocating GPU resources among different service regions based on the execution results of the task delivery path includes: The propagation of subtask results is simulated by the movement of chess pieces. After a subtask completes its calculation in a certain computing subunit, it passes the intermediate result to the next computing subunit on the path through shared video memory, NVLink, PCIe or network, while releasing the computing resources originally occupied. The method of passing intermediate results is automatically selected according to the hardware topology of the chessboard square where the upstream and downstream chess pieces are located. The status of each piece is continuously monitored, and the GPU resources of each computing subunit are adjusted in real time. The adjustment process begins with each service area to count the GPU resources required at present, determine whether the guaranteed resource pool of the corresponding service area meets the task requirements, and further determine whether there is resource redundancy based on the judgment result. The criterion for resource redundancy is the number of computing sub-units in the resource pool whose continuous idle time exceeds the redundancy threshold. Based on the judgment result, redundant computing sub-units are temporarily separated through the resource sharing pool and marked with the start and end times of their availability, so that they can be borrowed by other service areas. Then, the GPU resources between service areas are completed according to the resource sharing protocol between service areas.

Images