Task scheduling method and device, equipment, medium and product

Abstract

The application relates to the technical field of computers, and provides a task scheduling method, device, equipment, medium and product. The method comprises the following steps: acquiring an instance mode and a processing sequence of each to-be-processed task in a plurality of to-be-processed tasks of a GPU; determining a first task allocation scheme according to the instance mode and the processing sequence of each to-be-processed task, wherein the first task allocation scheme is used for indicating a mapping relationship between an instance created on the GPU and the to-be-processed tasks; performing an optimization operation on the first task allocation scheme to obtain a second task allocation scheme, wherein a completion time of the second task allocation scheme is less than a completion time of the first task allocation scheme; and the optimization operation comprises at least one of a first optimization operation and a second optimization operation. According to the scheme, the task scheduling efficiency for the GPU can be improved.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a task scheduling method, apparatus, device, medium, and product. Background Technology

[0002] With the widespread adoption of cloud computing technology, cloud server clusters often utilize GPU virtualization technology to divide physical hardware into multiple instances with different computing specifications to handle computing tasks. In real-world business scenarios, user-submitted tasks are often diverse, with varying demands on computing resources. This necessitates the rational allocation of tasks to appropriate instances. However, existing task scheduling strategies often struggle to balance task execution order with resource consumption when matching multiple instances with heterogeneous tasks, easily leading to resource imbalances. This imbalance not only reduces hardware resource utilization but also prolongs the overall task execution cycle, making it difficult to meet time-sensitive business requirements. Therefore, improving GPU task scheduling efficiency has become a pressing technical challenge. Summary of the Invention

[0003] This application provides a task scheduling method, apparatus, device, medium, and product to address the technical problem of how to improve task scheduling efficiency for GPUs.

[0004] In a first aspect, embodiments of this application provide a task scheduling method, including: Obtain the instance mode and processing order of each of the multiple pending tasks on the GPU; The first task allocation scheme is determined based on the instance mode and processing order of each task to be processed. The first task allocation scheme is used to indicate the mapping relationship between the instances created on the GPU and the tasks to be processed. An optimization operation is performed on the first task allocation scheme to obtain a second task allocation scheme. The completion time of the second task allocation scheme is less than that of the first task allocation scheme. The optimization operation includes at least one of a first optimization operation and a second optimization operation; the first optimization operation is used to migrate a task to be processed specified in one instance to another instance between two instances with the same instance pattern; the second optimization operation is used to exchange a task to be processed specified in one instance with a task to be processed specified in another instance between two instances with the same instance pattern.

[0005] In conjunction with the first aspect, in some possible implementations, the instance mode and processing order of each of the multiple pending tasks on the GPU are obtained, including: The instance mode of each pending task is determined based on the task attributes of each pending task in the multiple pending tasks of the GPU. Get the processing time for each pending task; Multiple task lists are created based on the multiple instance modes supported by the GPU, with each instance mode corresponding to a task list. Based on the instance mode and processing time of each pending task, add each pending task to the corresponding task list in sequence; The processing order of each task is determined based on the order of the tasks to be processed in multiple task lists.

[0006] Combining the first aspect and the above implementation methods, in some possible implementation methods, the processing time of each task to be processed is obtained, including: Get the execution time, creation time, and destruction time of each pending task; The processing time for each pending task is determined based on its execution time, creation time, and destruction time.

[0007] Combining the first aspect and the above implementation methods, in some possible implementations, each pending task is added sequentially to the corresponding task list based on its instance mode and processing time, including: Based on the processing time of each task, sort the tasks in descending order of processing time to obtain the total task sequence. Based on the instance pattern of each pending task in the overall task sequence, each pending task is added to the corresponding task list in sequence.

[0008] Combining the first aspect and the above implementation methods, in some possible implementations, each pending task is added sequentially to the corresponding task list based on its instance mode and processing time, including: Based on the instance pattern of each pending task, multiple pending tasks are divided into multiple task sets, and each task set corresponds one-to-one with an instance pattern. For each of the multiple task sets, the tasks to be processed within each task set are added to the corresponding task list in descending order of their processing time.

[0009] Combining the first aspect and the above implementation methods, in some possible implementation methods, the first task allocation scheme is determined based on the instance mode and processing order of each task to be processed, including: Based on the order of computing resources consumed by the multiple instance modes supported by the GPU from most to least, traverse multiple instance modes. For the first instance pattern encountered during traversal, determine the first computing resource occupied by the first instance pattern, and determine the first instance based on the first computing resource; The unassigned tasks corresponding to the first instance mode are assigned to the first instance in order of processing. In the case where the GPU's computing resources include a second computing resource in addition to the first computing resource, a second instance is determined based on the second computing resource, and at least a portion of the unallocated pending tasks corresponding to the second instance mode are allocated to the second instance; the second instance mode is an instance mode other than the first instance mode among multiple instance modes, and the duration of the first instance is greater than or equal to the duration of the second instance. If multiple pending tasks have been assigned to the corresponding instances, the traversal ends to obtain the first task assignment scheme.

[0010] Combining the first aspect and the above implementation methods, in some possible implementation methods, at least a portion of the unassigned pending tasks corresponding to the second instance mode are assigned to the second instance, including: During the lifetime of the first instance, with the goal of maximizing the total processing time of the pending tasks allocated to the second instance, at least a portion of the pending tasks are selected from the unallocated pending tasks corresponding to the second instance mode. At least some of the pending tasks will be assigned to the second instance.

[0011] Combining the first aspect and the above implementation methods, in some possible implementation methods, when the optimization operation includes the first optimization operation, performing an optimization operation on the first task allocation scheme to obtain the second task allocation scheme includes: Obtain the first task chain of the first task allocation scheme, and the processing time of the first task chain; Iterate through the instances to be migrated out associated with the first task chain; For each instance to be migrated out, determine the corresponding instance to be migrated in; the instance to be migrated in and the instance to be migrated out have the same instance schema. If, among the pending tasks associated with the instance to be migrated out, there is a first target task whose processing time is less than or equal to the idle time of the instance to be migrated in, then the first target task will be migrated from the instance to be migrated out to the instance to be migrated in; the idle time of the instance to be migrated in is the difference between the completion time of the instance to be migrated out and the completion time of the instance to be migrated in. Update the first task chain and its processing time, and return to the step of traversing the instances to be migrated associated with the first task chain until the processing time of the first task chain no longer decreases, then end the traversal to obtain the second task allocation scheme.

[0012] Combining the first aspect and the above implementation methods, in some possible implementation methods, when the optimization operation includes a second optimization operation, performing an optimization operation on the first task allocation scheme to obtain a second task allocation scheme includes: Obtain the first task chain of the first task allocation scheme, and the processing time of the first task chain; Iterate through the first instance to be swapped associated with the first task chain; For each first instance to be swapped encountered, determine the corresponding second instance to be swapped; the second instance to be swapped has the same instance mode as the first instance to be swapped, and the completion time of the second instance to be swapped is less than the completion time of the first instance to be swapped. If a second target task exists among the pending tasks associated with the first instance to be swapped, and a third target task exists among the pending tasks associated with the second instance to be swapped, and the processing time of the second target task is greater than the processing time of the third target task, and the difference between the processing times of the second and third target tasks is less than the idle time of the second instance to be swapped, then the second target task and the third target task will be swapped. The idle time of the second instance to be swapped is the difference between the completion time of the first instance to be swapped and the completion time of the second instance to be swapped. Update the first task chain and its processing time, and return to the step of traversing the first instance to be swapped associated with the first task chain until the processing time of the first task chain no longer decreases, then end the traversal to obtain the second task allocation scheme.

[0013] In combination with the first aspect and the above implementation methods, in some possible implementation methods, when the optimization operation includes a first optimization operation and a second optimization operation, performing an optimization operation on the first task allocation scheme to obtain a second task allocation scheme includes: Perform the first optimization operation on the first task allocation scheme to obtain the intermediate task allocation scheme; Perform a second optimization operation on the intermediate task allocation scheme to obtain a second task allocation scheme.

[0014] Combining the first aspect and the above implementation methods, in some possible implementation methods, performing a first optimization operation on the first task allocation scheme to obtain an intermediate task allocation scheme includes: Obtain the first task chain of the first task allocation scheme, and the processing time of the first task chain; Iterate through the instances to be migrated out associated with the first task chain; For each instance to be migrated out, determine the corresponding instance to be migrated in; the instance to be migrated in and the instance to be migrated out have the same instance schema. If, among the pending tasks associated with the instance to be migrated out, there is a first target task whose processing time is less than or equal to the idle time of the instance to be migrated in, then the first target task will be migrated from the instance to be migrated out to the instance to be migrated in; the idle time of the instance to be migrated in is the difference between the completion time of the instance to be migrated out and the completion time of the instance to be migrated in. Update the first task chain and its processing time, and return to the step of traversing the instances to be migrated associated with the first task chain until the processing time of the first task chain no longer decreases, then end the traversal to obtain the intermediate task allocation scheme.

[0015] Combining the first aspect and the above implementation methods, in some possible implementation methods, a second optimization operation is performed on the intermediate task allocation scheme to obtain a second task allocation scheme, including: Obtain the second task chain of the intermediate task allocation scheme, and the processing time of the second task chain; Iterate through the first instance to be swapped associated with the second task chain; For each first instance to be swapped encountered, determine the corresponding second instance to be swapped; the second instance to be swapped has the same instance mode as the first instance to be swapped, and the completion time of the second instance to be swapped is less than the completion time of the first instance to be swapped. If a second target task exists among the pending tasks associated with the first instance to be swapped, and a third target task exists among the pending tasks associated with the second instance to be swapped, and the processing time of the second target task is greater than the processing time of the third target task, and the difference between the processing times of the second and third target tasks is less than the idle time of the second instance to be swapped, then the second target task and the third target task will be swapped. The idle time of the second instance to be swapped is the difference between the completion time of the first instance to be swapped and the completion time of the second instance to be swapped. Update the second task chain and its processing time, and return to the step of traversing the first instance to be swapped associated with the second task chain until the processing time of the second task chain no longer decreases, then end the traversal to obtain the second task allocation scheme.

[0016] Secondly, embodiments of this application provide a task scheduling device, comprising: The acquisition module is used to acquire the instance mode and processing order of each of the multiple pending tasks in the GPU; The determination module is used to determine the first task allocation scheme based on the instance mode and processing order of each task to be processed. The first task allocation scheme is used to indicate the mapping relationship between the instances created on the GPU and the tasks to be processed. The optimization module is used to perform optimization operations on the first task allocation scheme to obtain a second task allocation scheme, the completion time of the second task allocation scheme is less than the completion time of the first task allocation scheme; The optimization operation includes at least one of a first optimization operation and a second optimization operation; the first optimization operation is used to migrate a task to be processed specified in one instance to another instance between two instances with the same instance pattern; the second optimization operation is used to exchange a task to be processed specified in one instance with a task to be processed specified in another instance between two instances with the same instance pattern.

[0017] Thirdly, embodiments of this application provide an electronic device, including a processor and a memory storing a computer program, wherein the processor executes the program to implement the steps of the task scheduling method of the first aspect.

[0018] Fourthly, embodiments of this application provide a non-transitory computer-readable storage medium storing a computer program thereon, wherein the computer program, when executed by a processor, implements the steps of the task scheduling method of the first aspect.

[0019] Fifthly, embodiments of this application provide a computer program product, including a computer program, which, when executed by a processor, implements the steps of the task scheduling method of the first aspect.

[0020] The task scheduling method, apparatus, device, medium, and product provided in this application first obtain the instance mode and processing order of each of the multiple pending tasks on the GPU. Then, based on the instance mode and processing order of each pending task, a first task allocation scheme is determined to indicate the mapping relationship between instances created on the GPU and pending tasks. Subsequently, an optimization operation is performed on the first task allocation scheme to obtain a second task allocation scheme. The optimization operation includes at least one of the first optimization operation and the second optimization operation. Through the above steps, the mapping relationship established based on the instance mode and processing order realizes the matching of tasks and resources. By adjusting the first task allocation scheme through the first optimization operation and / or the second optimization operation, the load between instances is balanced and resource fragmentation is reduced, thereby shortening the overall task completion time and improving the task scheduling efficiency for the GPU. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a schematic diagram of the GPU architecture provided in an embodiment of this application; Figure 2 This is a flowchart illustrating the task scheduling method provided in an embodiment of this application; Figure 3 This is a schematic diagram of GPU instance mode resource configuration provided in an embodiment of this application; Figure 4 This is a schematic diagram of task scheduling and resource allocation in a multi-instance mode provided in an embodiment of this application; Figure 5 This is a schematic diagram comparing the task scheduling timing before and after the first optimization operation provided in the embodiments of this application; Figure 6 This is a schematic diagram comparing the task scheduling timing before and after the second optimization operation, provided as an embodiment of this application. Figure 7 This is a schematic diagram of the structure of the task scheduling device provided in the embodiments of this application; Figure 8 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0024] With the popularization of cloud computing technology, cloud server clusters often utilize GPU virtualization technology to divide physical hardware into multiple instances with different computing specifications to carry out computing tasks. In actual business scenarios, user-submitted tasks are often diverse, and different tasks have different requirements for computing resources. This necessitates the rational allocation of tasks to appropriate instances.

[0025] Please see Figure 1 , Figure 1This is a schematic diagram of the GPU architecture provided in an embodiment of this application. The graphics processing unit (GPU) integrates multiple streaming multiprocessors (SMs), each of which forms a data path with the underlying video memory via a unified L2 cache. Each streaming multiprocessor is further divided into several sub-partitions, each configured with a warp scheduler, paired computation pipelines, register files, and an L0 instruction cache. Data interaction between sub-partitions is achieved through a shared L1 cache / shared memory. At the instruction execution level, the GPU driver decomposes computational tasks into large-scale, fine-grained threads, forming warps of 32 threads each as the basic scheduling unit. The warp scheduler maintains the context switching of multiple warps, immediately scheduling ready warps for execution when a current warp is waiting due to data dependencies, thereby masking memory access latency and ensuring the computation pipeline remains active. Based on these hardware characteristics, the physical GPU is virtualized into several instances to handle different workloads. Each instance, as a virtual execution unit with specific computational specifications, consists of several streaming multiprocessor slices, and the number of slices directly determines the computational throughput of the instance. Different instance modes correspond to differentiated streaming multiprocessor slice configurations, and the actual runtime of a task is closely related to the instance mode of the allocated instance.

[0026] For example, in some related technologies, a first-in-first-out (FIFO) strategy is typically used to allocate tasks to instances, or allocation is based on the remaining computing resources of the instances. Specifically, when a task is received, the system allocates it to an instance that is currently idle or has sufficient computing resources for execution, without considering the differences in processing time of the tasks or the coordination relationship between instance modes. Furthermore, during the idle phase after an instance completes its task processing, related technologies often lack a dynamic adjustment mechanism for instance resources, failing to adjust the instance mode in a timely manner according to the needs of subsequent tasks.

[0027] It is evident that the aforementioned technologies have shortcomings: the lack of optimization in matching the processing time of tasks to be processed with the instance mode of instances leads to an imbalance in the allocation of computing resources. Delaying the execution of tasks with long processing times will cause tail blocking in the overall task processing cycle; while static allocation strategies tend to generate fragmented idle periods in the GPU's computing resources, reducing the utilization of hardware resources and thus extending the overall execution cycle of tasks.

[0028] Therefore, improving the efficiency of task scheduling for GPUs has become an urgent technical problem to be solved.

[0029] To address the aforementioned issues, the solution provided in this application mainly includes: firstly, obtaining the instance mode and processing order of each of the multiple pending tasks on the GPU; then, determining a first task allocation scheme based on the instance mode and processing order of each pending task to indicate the mapping relationship between instances created on the GPU and pending tasks; subsequently, performing an optimization operation on the first task allocation scheme to obtain a second task allocation scheme, wherein the optimization operation includes at least one of a first optimization operation and a second optimization operation; through the above steps, the mapping relationship established based on the instance mode and processing order realizes the matching of tasks and resources, and the adjustment of the first task allocation scheme through the first optimization operation and / or the second optimization operation balances the load among instances and reduces resource fragmentation, thereby shortening the overall task completion time and improving the task scheduling efficiency for the GPU.

[0030] The task scheduling method provided in the embodiments of this application will be described in detail below.

[0031] Please see Figure 2 , Figure 2 This is a flowchart illustrating a task scheduling method provided in an embodiment of this application. Figure 2 As shown, the method in this application embodiment may include the following steps S101~S103: S101, obtain the instance mode and processing order of each of the multiple pending tasks in the GPU; S102, determine the first task allocation scheme according to the instance mode and processing order of each task to be processed. The first task allocation scheme is used to indicate the mapping relationship between the instances created on the GPU and the tasks to be processed. S103, perform an optimization operation on the first task allocation scheme to obtain a second task allocation scheme, the completion time of the second task allocation scheme is less than the completion time of the first task allocation scheme; the optimization operation includes at least one of a first optimization operation and a second optimization operation; the first optimization operation is used to migrate a task to be processed specified by one instance to another instance between two instances with the same instance mode; the second optimization operation is used to exchange a task to be processed specified by one instance with a task to be processed specified by another instance between two instances with the same instance mode.

[0032] Specifically, the first step is to obtain the instance mode and processing order of each of the multiple pending tasks on the GPU. Here, GPU refers to the Graphics Processing Unit, used to execute computationally intensive tasks; pending tasks refer to jobs or processes that require computation by the GPU, such as image rendering tasks, deep learning model training tasks, or scientific computing tasks; the instance mode of a pending task refers to the computing resource configuration information of the instance used to run that pending task, which represents the number of streaming multiprocessors or the slice size contained in the instance; and the processing order of pending tasks refers to the order in which each pending task is scheduled for execution.

[0033] Regarding this step, in some possible implementations, multiple tasks to be processed by the GPU can be obtained through relevant data reading interfaces, and the instance mode and processing order of each task can be extracted from the task attribute information of the multiple tasks. Alternatively, the multiple tasks to be processed by the GPU can be parsed and processed using relevant parsing algorithms to obtain the instance mode and processing order of each task.

[0034] It should be noted that the GPU can be a GPU in a cloud server cluster. A cloud server cluster is a computing system composed of multiple cloud servers interconnected through a network, which can provide elastic computing power. In cloud service scenarios, the method provided in this embodiment can be applied to application scenarios such as cloud video processing, large-scale data analysis, and artificial intelligence inference, and is not limited here.

[0035] Furthermore, a first task allocation scheme is determined based on the instance mode and processing order of each task to be processed. This first task allocation scheme indicates the mapping relationship between instances created on the GPU and tasks to be processed. Specifically, the first task allocation scheme refers to the initially generated allocation plan used to guide the task scheduling process; the mapping relationship between instances and tasks to be processed refers to the correspondence between the instance's identification information and the task's identification information, as well as the expected execution time information of the task on the instance. It is understood that the first task allocation scheme can be implemented based on tables, linked lists, or other forms; this embodiment does not limit the specific data structure or representation of the first task allocation scheme.

[0036] Regarding this step, in some possible implementations, a mapping relationship between instances created on the GPU and tasks to be processed can be determined using a relevant matching algorithm based on the instance mode and processing order of each task to be processed, thus obtaining a first task allocation scheme. Alternatively, a mapping relationship between instances created on the GPU and tasks to be processed can be determined using relevant rules based on the instance mode and processing order of each task to be processed, thus obtaining a first task allocation scheme.

[0037] Furthermore, an optimization operation is performed on the first task allocation scheme to obtain a second task allocation scheme. The optimization operation refers to the process of adjusting the mapping relationships in the first task allocation scheme. This adjustment process may consist of a single adjustment phase or multiple sequentially executed adjustment phases. The completion time of the second task allocation scheme is shorter than that of the first task allocation scheme. This means that after the optimization operation, the load on each instance is more balanced, resource fragmentation is reduced, and thus the total time required to process all pending tasks is shortened.

[0038] The optimization operation includes at least one of the first optimization operation and the second optimization operation. That is, the optimization operation may include only the first optimization operation, only the second optimization operation, or both the first optimization operation and the second optimization operation.

[0039] The first optimization operation is used to migrate a specified pending task from one instance to another between two instances with the same instance pattern. The specified pending task is any selected pending task that meets the migration criteria. It can be understood that the first optimization operation fills idle time periods of the instances by migrating tasks from a heavily loaded or later-completed instance to a less loaded or earlier-completed instance of the same pattern, thereby balancing the load.

[0040] The second optimization operation is used to swap a task specified by one instance with a task specified by another instance between two instances with the same instance pattern. The specified task is any selected task that satisfies the swap condition. It can be understood that by swapping tasks on different instances, the second optimization operation can change the final idle time of each instance, thereby shortening the length of the task chain that determines the overall completion time.

[0041] Regarding this step, in some possible implementations, only the first optimization operation can be performed, i.e., adjusting the first task allocation scheme using a relevant transfer algorithm to obtain the second task allocation scheme. Alternatively, only the second optimization operation can be performed, i.e., adjusting the first task allocation scheme using a relevant exchange algorithm to obtain the second task allocation scheme. Alternatively, the first and second optimization operations can be performed sequentially, i.e., first adjusting the first task allocation scheme using a relevant transfer algorithm, and then further adjusting it using a relevant exchange algorithm to obtain the second task allocation scheme.

[0042] Understandably, after determining the second task allocation scheme, the system will create corresponding instances sequentially according to the mapping relationship indicated by the second task allocation scheme, and distribute the tasks to be processed to the corresponding instances for execution. During execution, the GPU will allocate the corresponding streaming multiprocessor resources according to the instance mode, and control the execution flow of the tasks according to the start time and duration in the task record, until all tasks to be processed are completed.

[0043] In this embodiment, the instance mode and processing order of each of the multiple pending tasks on the GPU are first obtained. Then, a first task allocation scheme is determined based on the instance mode and processing order of each pending task to indicate the mapping relationship between instances created on the GPU and pending tasks. Subsequently, an optimization operation is performed on the first task allocation scheme to obtain a second task allocation scheme. The optimization operation includes at least one of the first optimization operation and the second optimization operation. Through the above steps, the mapping relationship established based on the instance mode and processing order realizes the matching of tasks and resources. By adjusting the first task allocation scheme through the first optimization operation and / or the second optimization operation, the load between instances is balanced and resource fragmentation is reduced, thereby shortening the overall task completion time and improving the task scheduling efficiency for the GPU.

[0044] In one embodiment, the step of "obtaining the instance mode and processing order of each of the multiple tasks to be processed in the GPU" can be further refined and may include the following steps: The instance mode of each pending task is determined based on the task attributes of each pending task in the multiple pending tasks of the GPU. Get the processing time for each pending task; Multiple task lists are created based on the multiple instance modes supported by the GPU, with each instance mode corresponding to a task list. Based on the instance mode and processing time of each pending task, add each pending task to the corresponding task list in sequence; The processing order of each task is determined based on the order of the tasks to be processed in multiple task lists.

[0045] Specifically, considering the diversity of tasks to be processed and the different computing resource requirements of different instance modes, this embodiment proposes a grouping and sorting mechanism based on task attributes and processing time.

[0046] First, the instance mode of each task needs to be determined based on its task attributes among the multiple tasks to be processed on the GPU. Task attributes refer to a set of information describing the characteristics of the task, including its computational resource requirements, such as the required number of streaming multiprocessors or the size of video memory.

[0047] Regarding this step, in some possible implementations, the resource requirement parameters of each task to be processed can be parsed, and the resource requirement parameters can be matched with the specifications of the instance modes supported by the GPU. The instance mode that matches successfully can be determined as the instance mode of the task to be processed.

[0048] Furthermore, the processing time for each pending task is obtained. The processing time for a pending task refers to the time interval required for the task to run on the instance from start to finish.

[0049] Regarding this step, some possible implementations include calculating the execution time of the task in the corresponding instance mode based on the historical execution records of the task or a pre-defined estimation model, and combining this with the time consumed by instance creation and destruction to obtain the processing time for each task. Alternatively, the timestamp carried when the task is submitted can be read directly, and the duration indicated by the timestamp can be used as the processing time for the task.

[0050] Furthermore, multiple task lists are created based on the various instance modes supported by the GPU, with a one-to-one correspondence between instance modes and task lists. Here, a task list refers to a data structure used to store the identifiers of tasks to be processed; a one-to-one correspondence means that each instance mode is uniquely associated with one task list.

[0051] Regarding this step, in some possible implementations, several empty container objects can be initialized, with the number of containers matching the number of instance modes supported by the GPU. Each container object is then labeled with a corresponding instance mode index, thereby establishing a mapping relationship between the task list and the instance modes.

[0052] Furthermore, based on the instance mode and processing time of each pending task, each pending task is added to the corresponding task list in sequence.

[0053] Regarding this step, in some possible implementations, all pending tasks can be traversed, the corresponding task list can be located based on the instance pattern of the current pending task, and the current pending task can be inserted into the corresponding position in the task list according to the processing time.

[0054] Furthermore, the processing order of each task is determined based on the order of the tasks to be processed in the multiple task lists.

[0055] Regarding this step, in some possible implementations, the tasks to be processed in each task list can be extracted sequentially according to a preset reading rule, and the order number of the tasks to be processed in the task list can be mapped to a global scheduling number, thereby determining the processing order of each task to be processed.

[0056] In this embodiment, by grouping multiple tasks into corresponding task lists according to the instance pattern of each task, the system achieves categorized management of tasks with different computing resource requirements. By adjusting the order of tasks in the task list based on processing time, tasks with longer processing times can be prioritized, avoiding tail blocking caused by long tasks in the later stages of scheduling. This embodiment's combination of classification and sorting standardizes the processing order of tasks, reduces the fragmentation of computing resources during subsequent allocation, and provides an ordered data foundation for determining the first task allocation scheme, thereby helping to shorten the overall task completion time.

[0057] In one embodiment, the step of "obtaining the processing time of each task to be processed" can be further refined and may include the following steps: Get the execution time, creation time, and destruction time of each pending task; The processing time for each pending task is determined based on its execution time, creation time, and destruction time.

[0058] Specifically, considering that the running cycle of the task to be processed on the instance includes the calculation phase and the resource management phase, this embodiment proposes a processing time calculation method that combines the execution time and the instance management time.

[0059] First, we need to obtain the execution time, creation time, and destruction time of each pending task. The execution time refers to the duration required for the pending task to perform computational operations on the instance; the creation time refers to the duration required to build the instance and initialize the environment for running the pending task; and the destruction time refers to the duration required to release instance resources and clean up the environment after the pending task is completed.

[0060] Regarding this step, some possible implementations include querying historical execution records to obtain the reference execution time, reference creation time, and reference destruction time corresponding to the task to be processed; or reading the configuration parameters of the task to be processed and using the time values ​​contained in the configuration parameters as the execution time, creation time, and destruction time, respectively.

[0061] Furthermore, the processing time for each pending task is determined based on its execution time, creation time, and destruction time.

[0062] Regarding this step, in some possible implementations, the obtained execution time, creation time, and destruction time can be summed up, and the result can be used to determine the processing time for each task to be processed.

[0063] It's important to note that in GPU task scheduling scenarios, instance creation and destruction times are typically in the millisecond range, while execution times are usually in the minutes or hours range. Creation and destruction times are numerically much smaller than execution times. In specific scheduling scenarios where error is acceptable, the execution time can be directly determined as the processing time of the task, or creation and destruction times can be set as preset constants. However, in high-concurrency or short-task scenarios, incorporating creation and destruction times into the calculation can improve the accuracy of processing time, thereby enhancing scheduling precision.

[0064] In this embodiment, the execution time, creation time, and destruction time of each task to be processed are obtained, and the processing time is determined by combining these three factors, thus quantifying the resource usage time of the tasks to be processed. This approach avoids scheduling bias caused by ignoring instance management overhead, ensuring that the determined processing time reflects the actual GPU resource usage of the tasks to be processed, thereby improving the accuracy of the task allocation scheme.

[0065] In one embodiment, the step of "adding each pending task to the corresponding task list sequentially according to the instance mode and processing time of each pending task" can be further refined and may include the following steps: Based on the processing time of each task, sort the tasks in descending order of processing time to obtain the total task sequence. Based on the instance pattern of each pending task in the overall task sequence, each pending task is added to the corresponding task list in sequence.

[0066] Specifically, considering that if sorting is only performed within groups of different instance modes, it may be difficult to prioritize scheduling the longest pending tasks from a global perspective, which may easily lead to long task waiting or tail blocking of idle resources in the later stages of the overall scheduling scheme, thereby affecting the overall task completion time, this embodiment proposes a task grouping strategy based on global timing optimization.

[0067] First, the tasks to be processed need to be sorted in descending order of their processing time to obtain the total task sequence. The total task sequence refers to the ordered set formed by treating all tasks as a whole and arranging them in descending order of their processing time. The first task in this ordered set has the longest processing time.

[0068] Regarding this step, in some possible implementations, multiple tasks to be processed can be traversed, the processing time of each task can be extracted, and the multiple tasks to be processed can be sorted in descending order using a preset sorting algorithm to obtain the total task sequence.

[0069] Furthermore, based on the instance pattern of each pending task in the overall task sequence, each pending task is added to the corresponding task list in sequence.

[0070] Regarding this step, in some possible implementations, each pending task in the overall task sequence can be read sequentially. For the currently pending task, the instance pattern recorded in its task attributes can be identified, and the current pending task can be added to the task list that corresponds to the instance pattern. Since the overall task sequence is already arranged in descending order of processing time, the pending tasks stored in each task list will naturally form an order from longest to shortest processing time.

[0071] In this embodiment, by first globally sorting all pending tasks and then distributing them to the corresponding task lists, it is ensured that tasks with longer processing times are prioritized at the beginning of the task list. Based on this, during subsequent allocation according to the task list, tasks with longer processing times will be scheduled for execution first, avoiding tail blocking problems caused by long tasks being ranked later and only starting execution later in the scheduling process. This reduces fragmented idle time of computing resources, thereby shortening the overall task completion time.

[0072] In one embodiment, the step of "adding each pending task to the corresponding task list sequentially according to the instance mode and processing time of each pending task" can be further refined and may include the following steps: Based on the instance pattern of each pending task, multiple pending tasks are divided into multiple task sets, and each task set corresponds one-to-one with an instance pattern. For each of the multiple task sets, the tasks to be processed within each task set are added to the corresponding task list in descending order of their processing time.

[0073] Specifically, considering that directly sorting all tasks to be processed globally would result in high computational complexity when dealing with a massive number of tasks, and that it would be difficult to isolate tasks of different instance modes for independent scheduling and management, this embodiment proposes a task list construction strategy of grouping first and then sorting.

[0074] First, based on the instance pattern of each task to be processed, multiple tasks need to be divided into multiple task sets, with each task set corresponding one-to-one with an instance pattern. A task set refers to a logical grouping unit containing tasks to be processed that share the same instance pattern; a one-to-one correspondence means that each task set is uniquely associated with one instance pattern, and each instance pattern corresponds to only one task set.

[0075] Regarding this step, in some possible implementations, multiple tasks to be processed can be traversed, the instance pattern of each task to be processed can be read, and tasks to be processed with the same instance pattern can be grouped into the same set, thereby constructing multiple task sets.

[0076] Furthermore, for each of the multiple task sets, the tasks to be processed contained within each task set are added to the corresponding task list in descending order of their processing times.

[0077] Regarding this step, in some possible implementations, for each of the multiple task sets, the processing time of the tasks to be processed contained in that task set can be read, the tasks to be processed in that task set can be sorted in descending order of processing time, and the sorted tasks to be processed can be added to the task list corresponding to that task set in order.

[0078] In this embodiment, by first dividing the tasks to be processed into task sets according to instance modes, and then independently sorting within each task set, the classification and management of tasks with different resource requirements is achieved. This approach transforms global sorting into multiple local sortings, reducing the computational complexity of the sorting algorithm, improving the efficiency of task list construction, and ensuring that the order of tasks in each task list is determined only by the processing time of tasks in the same group, thus avoiding sorting interference between tasks in different instance modes.

[0079] In one embodiment, the step of "determining the first task allocation scheme based on the instance mode and processing order of each task to be processed" can be further refined and may include the following steps: Based on the order of computing resources consumed by the multiple instance modes supported by the GPU from most to least, traverse multiple instance modes. For the first instance pattern encountered during traversal, determine the first computing resource occupied by the first instance pattern, and determine the first instance based on the first computing resource; The unassigned tasks corresponding to the first instance mode are assigned to the first instance in order of processing. In the case where the GPU's computing resources include a second computing resource in addition to the first computing resource, a second instance is determined based on the second computing resource, and at least a portion of the unallocated pending tasks corresponding to the second instance mode are allocated to the second instance; the second instance mode is an instance mode other than the first instance mode among multiple instance modes, and the duration of the first instance is greater than or equal to the duration of the second instance. If multiple pending tasks have been assigned to the corresponding instances, the traversal ends to obtain the first task assignment scheme.

[0080] Specifically, considering that the different instance modes of GPUs have different usage of computing resources, direct allocation may lead to resource fragmentation. This embodiment proposes an allocation strategy based on the relationship between computing resource partitioning and instance hierarchy.

[0081] First, based on the order of computing resources occupied by the multiple instance modes supported by the GPU from most to least, the multiple instance modes are traversed.

[0082] Regarding this step, in some possible implementations, the computational resource usage of each of the multiple instance modes supported by the GPU can be obtained, the multiple instance modes can be sorted in descending order of computational resource usage, and the multiple instance modes can be traversed in the sorted order as the first instance mode.

[0083] Furthermore, for the first instance pattern encountered during traversal, the first computing resource occupied by the first instance pattern is determined, and the first instance is determined based on the first computing resource. Here, the first instance pattern refers to the instance pattern currently encountered that corresponds to the maximum computing resource consumption; the first computing resource refers to the portion of computing resources required by the first instance pattern; and the first instance refers to a virtual computing unit partitioned based on the first computing resource.

[0084] Regarding this step, in some possible implementations, the configuration information of the first instance mode currently being traversed can be parsed, the computing resource occupancy indicated in the configuration information can be used as the first computing resource occupied by the first instance mode, and virtual computing units of corresponding specifications can be divided according to the first computing resource, and the virtual computing unit can be used as the first instance determined according to the first computing resource.

[0085] Furthermore, the unassigned tasks corresponding to the first instance mode are sequentially assigned to the first instance according to the processing order.

[0086] Regarding this step, in some possible implementations, unassigned pending tasks belonging to the first instance mode can be obtained, and the unassigned pending tasks can be read sequentially according to the processing order. The identification information of the read unassigned pending tasks can be mapped to the identification information of the first instance. The mapping relationship can be used to assign the unassigned pending tasks corresponding to the first instance mode to the first instance.

[0087] Furthermore, when the GPU's computing resources include second computing resources in addition to the first computing resources, a second instance is determined based on the second computing resources, and at least a portion of the unallocated pending tasks corresponding to the second instance mode are allocated to the second instance; the second instance mode is an instance mode other than the first instance mode among multiple instance modes, and the duration of the first instance is greater than or equal to the duration of the second instance.

[0088] In this context, GPU computing resources refer to the total set of hardware resources available for tasks; secondary computing resources refer to the computing resources remaining after deducting primary computing resources from the GPU's total computing resources. It can be understood that if the total amount of GPU computing resources equals the primary computing resources, then secondary computing resources do not exist; if the total amount of GPU computing resources is greater than the primary computing resources, then secondary computing resources exist.

[0089] The second instance refers to a virtual computing unit partitioned based on the second computing resource; the second instance mode refers to an instance mode among multiple instance modes whose computing resource consumption is less than that of the first instance mode; the duration of an instance refers to the time period spanned by an instance from the start of executing the assigned pending tasks to the completion of the last assigned pending task; if the duration of the first instance is greater than or equal to the duration of the second instance, it means that all pending tasks assigned on the second instance must be completed within the runtime of the first instance.

[0090] It should be noted that since the size of the second computing resource can support multiple instance modes with different computing resource occupancy, the number of second instance modes is at least one, and correspondingly, the number of second instances is also at least one.

[0091] Regarding this step, in some possible implementations, it can be determined whether the total amount of GPU computing resources is greater than the first computing resource. If it is greater, the difference between the total amount of GPU computing resources and the first computing resource is used as the second computing resource. Based on the capacity of the second computing resource, the second instance mode is determined from multiple instance modes other than the first instance mode. Based on the second computing resource, the second instance corresponding to the second instance mode is determined, and at least a portion of the unallocated pending tasks corresponding to the second instance mode are allocated to the second instance, so that the duration of the first instance is greater than or equal to the duration of the second instance.

[0092] Furthermore, if multiple pending tasks have been assigned to corresponding instances, the traversal ends to obtain the first task allocation scheme.

[0093] The fact that multiple pending tasks are assigned to corresponding instances means that all the pending tasks have been mapped to the corresponding instances.

[0094] Regarding this step, in some possible implementations, it is possible to detect whether there are any pending tasks that have not yet been mapped to any instance. If there are no pending tasks that have not yet been mapped to any instance, the set of mapping relationships between all current instances and pending tasks is used as the first task allocation scheme, and the traversal ends.

[0095] It should be noted that if multiple pending tasks are not all assigned to the corresponding instance, it indicates that the first instance pattern currently being traversed and its derived second instance pattern cannot completely consume the remaining pending tasks. The traversal needs to be performed to assign the next instance pattern as the new first instance pattern.

[0096] Optionally, before the step "traverse multiple instance modes according to the order of the computing resources occupied by the multiple instance modes supported by the GPU from most to least", it may also include "determining the parent instance corresponding to the computing resources of the GPU".

[0097] Specifically, a parent instance refers to a virtual computing unit that occupies the entire set of computing resources on the GPU. As the top-level virtual computing unit, the parent instance can be broken down into child instances that occupy a portion of the computing resources, and the child instances are logically subordinate to the parent instance.

[0098] Regarding this step, some possible implementations involve obtaining the GPU's hardware attribute information, treating all the GPU's streaming multiprocessors as the GPU's computing resources, and identifying the virtual computing units occupying all of these streaming multiprocessors as the parent instance corresponding to the GPU's computing resources. It's understandable that, based on the definition of a parent instance, the first instance refers to a child instance partitioned from the parent instance based on the first computing resource, and the second instance refers to a child instance partitioned from the parent instance based on the second computing resource. Logically, the first and second instances are subordinate to the parent instance.

[0099] In this embodiment, multiple instance modes supported by the GPU are traversed in descending order of the amount of computing resources they occupy. A first instance and a second instance are determined based on all or part of the GPU's computing resources. Tasks to be processed are then assigned to the corresponding instances, achieving a hierarchical partitioning of GPU computing resources. Since the lifespan of the first instance is greater than or equal to that of the second instance, this partitioning method ensures that tasks to be processed on the second instance, which occupies fewer computing resources, can be completed within the runtime of the first instance, which occupies more computing resources. This reduces scheduling wait caused by computing resource fragmentation during the planning phase and provides resource-constrained allocation logic for obtaining the first task allocation scheme.

[0100] In one embodiment, the step of "assigning at least a portion of the unassigned pending tasks corresponding to the second instance mode to the second instance" can be further refined and may include the following steps: During the lifetime of the first instance, with the goal of maximizing the total processing time of the pending tasks allocated to the second instance, at least a portion of the pending tasks are selected from the unallocated pending tasks corresponding to the second instance mode. At least some of the pending tasks will be assigned to the second instance.

[0101] Specifically, considering that the lifespan of the second instance is limited by the lifespan of the first instance, if the amount of tasks allocated is not limited, the tasks may exceed the time constraints or cause idle computing resources. Therefore, this embodiment proposes a task selection strategy based on time boundary constraints.

[0102] First, within the lifespan of the first instance, with the goal of maximizing the total processing time of the tasks assigned to the second instance, at least a portion of the unassigned tasks corresponding to the second instance mode need to be selected.

[0103] Regarding this step, in some possible implementations, the duration of the first instance can be used as the time limit. The unassigned pending tasks corresponding to the second instance mode are traversed, and the total processing time after allocating each pending task to the second instance is calculated. Under the condition that the total processing time does not exceed the time limit, with the goal of maximizing the total processing time of the pending tasks allocated to the second instance, at least a portion of the pending tasks that meet the conditions are selected from the unassigned pending tasks corresponding to the second instance mode.

[0104] It should be noted that the goal of maximizing the total processing time of the tasks assigned to the second instance is to improve the utilization of computing resources as much as possible within the time boundary of the first instance's lifespan, reduce the idle time of the second instance during operation, and thus avoid wasting computing resources.

[0105] Furthermore, at least a portion of the pending tasks are assigned to the second instance.

[0106] Regarding this step, in some possible implementations, the identification information of at least some of the tasks to be processed can be associated with the identification information of the second instance to generate a task record that instructs the second instance to execute at least some of the tasks to be processed, and the generated task record is used to assign at least some of the tasks to be processed to the second instance.

[0107] In this embodiment, by maximizing the total processing time of the tasks allocated to the second instance within the lifespan of the first instance, at least a portion of the unallocated tasks corresponding to the second instance mode are selected and assigned to the second instance, thus achieving full utilization of computing resources under time constraints. This selection method ensures that the tasks allocated to the second instance are compactly arranged, reducing the resource idle time of the second instance during its runtime, thereby improving the overall utilization rate of computing resources in the first task allocation scheme while meeting the lifespan limit of the first instance.

[0108] In one embodiment, the step of "performing an optimization operation on the first task allocation scheme to obtain a second task allocation scheme" can be further refined and may include the following steps: Obtain the first task chain of the first task allocation scheme, and the processing time of the first task chain; Iterate through the instances to be migrated out associated with the first task chain; For each instance to be migrated out, determine the corresponding instance to be migrated in; the instance to be migrated in and the instance to be migrated out have the same instance schema. If, among the pending tasks associated with the instance to be migrated out, there is a first target task whose processing time is less than or equal to the idle time of the instance to be migrated in, then the first target task will be migrated from the instance to be migrated out to the instance to be migrated in; the idle time of the instance to be migrated in is the difference between the completion time of the instance to be migrated out and the completion time of the instance to be migrated in. Update the first task chain and its processing time, and return to the step of traversing the instances to be migrated associated with the first task chain until the processing time of the first task chain no longer decreases, then end the traversal to obtain the second task allocation scheme.

[0109] Specifically, considering that the load of different instances in the first task allocation scheme may be unbalanced, some instances may have a late completion time due to task backlog, while other instances with the same instance mode may have idle periods. This load difference will lead to an extension of the processing time of the first task chain, thereby increasing the overall completion time of the first task allocation scheme. This embodiment proposes a load balancing optimization mechanism based on task migration. By migrating tasks from heavily loaded instances to lightly loaded instances of the same mode, idle periods are filled, load distribution is balanced, and the processing time of the first task chain is shortened.

[0110] First, it's necessary to obtain the first task chain of the first task allocation scheme and its processing time. A task chain refers to a sequence of related instances and their assigned tasks within the task allocation scheme, constituting a critical sequence that determines the overall completion time of the scheme. The first task chain is the critical sequence that determines the overall completion time of the first task allocation scheme; the sum of the processing times of the instances and their assigned tasks within this sequence determines the completion time of the first task allocation scheme. The processing time of the first task chain refers to the total time required to execute all tasks within it, i.e., the maximum time required for each instance in the first task chain to complete its assigned task. It's important to note that the completion time of the first task allocation scheme is determined by the processing time of the first task chain. As the critical path in the scheme, the processing time of the first task chain directly determines the final time required for all tasks to complete. Therefore, shortening the processing time of the first task chain shortens the completion time of the first task allocation scheme.

[0111] Regarding this step, in some possible implementations, the mapping relationship between each instance and the task to be processed in the first task allocation scheme can be parsed, the processing time of the task to be processed assigned to each instance can be identified, the critical path in the first task allocation scheme can be determined according to the completion time of each instance, the sequence of instances constituting the critical path and its associated task to be processed can be determined as the first task chain, and the processing time of the first task chain can be determined according to the processing time of the task to be processed associated with each instance on the first task chain.

[0112] Furthermore, iterate through the instances to be migrated out associated with the first task chain. Here, instances to be migrated out refer to those in the first task chain that have the potential to migrate and assign pending tasks, specifically instances in the first task chain that have a later completion time and have pending tasks that can be migrated.

[0113] Regarding this step, in some possible implementations, the instance identification information contained in the first task chain can be read sequentially, and each instance in the first task chain can be traversed as an instance to be migrated out. For each instance to be migrated out that is traversed, the task to be processed associated with that instance can be obtained, so as to determine whether there is a first target task that meets the migration conditions.

[0114] Furthermore, for each instance to be migrated out, a corresponding instance to be migrated in is determined; the instance to be migrated in has the same instance schema as the instance to be migrated out. Here, the instance to be migrated in refers to an instance that has the same instance schema as the instance to be migrated out, and whose completion time is earlier than that of the instance to be migrated out, and this instance has time slack to receive migration tasks.

[0115] Regarding this step, in some possible implementations, the instance mode identifier information of the instance to be migrated out can be obtained, other instances with the same instance mode identifier information in the first task allocation scheme can be traversed, and instances whose completion time is earlier than that of the instance to be migrated out can be identified as instances to be migrated in. The difference between the completion time of the instance to be migrated in and the completion time of the instance to be migrated out can be calculated as the idle time of the instance to be migrated in.

[0116] Furthermore, if among the pending tasks associated with the instance to be migrated out, there exists a first target task whose processing time is less than or equal to the idle time of the instance to be migrated in, then the first target task will be migrated from the instance to be migrated out to the instance to be migrated in. The idle time of the instance to be migrated in is the difference between the completion time of the instance to be migrated out and the completion time of the instance to be migrated in. Here, the first target task refers to the pending tasks associated with the instance to be migrated out, whose processing time is less than or equal to the idle time of the instance to be migrated in, and which is suitable for migration to the instance to be migrated in for execution; the idle time refers to the difference between the completion time of the instance to be migrated in and the completion time of the instance to be migrated out, representing the time margin by which the instance to be migrated in completes earlier than the instance to be migrated out, and this time margin can be used to accommodate the first target task migrated from the instance to be migrated out.

[0117] Regarding this step, in some possible implementations, the pending tasks associated with the instance to be migrated out can be traversed, the processing time of each pending task can be extracted, and the pending tasks whose processing time is less than or equal to the idle time of the instance to be migrated in can be identified as the first target task. The mapping association between the first target task and the instance to be migrated out can be removed, and the task code, start time and duration of the first target task can be added to the task record of the instance to be migrated in, so as to migrate the first target task from the instance to be migrated out to the instance to be migrated in.

[0118] It should be noted that by migrating the first target task, whose processing time is less than or equal to the idle time, from the outgoing instance with a later completion time to the incoming instance with an earlier completion time, the idle time of the incoming instance can be filled, reducing the idle computing resources and alleviating the load pressure on the outgoing instance. This makes the completion time of the two instances more balanced, thereby shortening the maximum completion time of each instance in the first task chain and achieving the goal of optimizing the overall completion time of the first task allocation scheme.

[0119] Understandably, if there is no first target task whose processing time is less than or equal to the idle time of the instance to be migrated out among the pending tasks associated with the instance to be migrated out, then the mapping relationship between the current instance to be migrated out and the pending tasks remains unchanged, and the process continues to traverse the next instance to be migrated out associated with the first task chain, or to determine other instances to be migrated in for the current instance to be migrated out, until the traversal is completed or a first target task that meets the migration conditions is found.

[0120] Further, update the first task chain and its processing time, and return to the step of traversing the instances to be migrated associated with the first task chain until the processing time of the first task chain no longer decreases, then end the traversal to obtain the second task allocation scheme.

[0121] Regarding this step, in some possible implementations, after the first target task migration is completed, the completion times of the instances to be migrated out and the instances to be migrated in are recalculated. Based on the updated completion times of each instance, the critical path in the first task allocation scheme is re-determined, and the updated critical path is determined as the updated first task chain. The maximum completion time of each instance on the updated first task chain is determined as the processing time of the updated first task chain. It is then determined whether the processing time of the updated first task chain is shorter than the processing time of the first task chain before the update. If it is indeed shorter, the step of traversing the instances to be migrated out associated with the first task chain is returned, and the search for migrateable tasks continues. If the processing time of the first task chain is no longer shortened, the current task allocation scheme is determined as the second task allocation scheme, and the optimization process ends.

[0122] In this embodiment, by obtaining the first task chain and its processing time of the first task allocation scheme, traversing the instances to be migrated out associated with the first task chain, determining the instances to be migrated in with the same instance mode for each instance to be migrated out, and migrating the first target task whose processing time is less than or equal to the idle time of the instance to be migrated in from the instance to be migrated out to the instance to be migrated in, and iteratively updating the first task chain and its processing time until it no longer shortens, dynamic adjustment of task allocation relationship is realized based on the first task allocation scheme; this adjustment shortens the processing time of the first task chain by balancing the load distribution among instances with the same instance mode, filling idle periods, and reducing idle computing resources, thereby obtaining a second task allocation scheme with a completion time less than the first task allocation scheme, and thus improving the task scheduling efficiency for GPU.

[0123] In one embodiment, the step of "performing an optimization operation on the first task allocation scheme to obtain a second task allocation scheme" can be further refined and may include the following steps: Obtain the first task chain of the first task allocation scheme, and the processing time of the first task chain; Iterate through the first instance to be swapped associated with the first task chain; For each first instance to be swapped encountered, determine the corresponding second instance to be swapped; the second instance to be swapped has the same instance mode as the first instance to be swapped, and the completion time of the second instance to be swapped is less than the completion time of the first instance to be swapped. If a second target task exists among the pending tasks associated with the first instance to be swapped, and a third target task exists among the pending tasks associated with the second instance to be swapped, and the processing time of the second target task is greater than the processing time of the third target task, and the difference between the processing times of the second and third target tasks is less than the idle time of the second instance to be swapped, then the second target task and the third target task will be swapped. The idle time of the second instance to be swapped is the difference between the completion time of the first instance to be swapped and the completion time of the second instance to be swapped. Update the first task chain and its processing time, and return to the step of traversing the first instance to be swapped associated with the first task chain until the processing time of the first task chain no longer decreases, then end the traversal to obtain the second task allocation scheme.

[0124] Specifically, considering that in the first task allocation scheme, there may be a misalignment in the task processing time among different instances with the same instance mode, that is, some instances are assigned tasks with longer processing times, resulting in later completion times, while other instances are assigned tasks with shorter processing times and have spare time, this uneven distribution of tasks will prolong the processing time of the first task chain. This embodiment proposes a chain length compression optimization mechanism based on task exchange. By exchanging the tasks to be processed on different instances, the cumulative processing time of each instance is changed, thereby shortening the processing time of the first task chain.

[0125] First, it is necessary to obtain the first task chain of the first task allocation scheme and the processing time of the first task chain. The implementation process of this step and the definition of the first task chain have been described in the above embodiments, and will not be repeated here.

[0126] Furthermore, iterate through the first instance to be exchanged associated with the first task chain. Here, the first instance to be exchanged refers to the instance contained in the first task chain that has the potential to exchange and distribute pending tasks. Specifically, it is an instance in the first task chain that has a later completion time and whose associated pending tasks include exchangeable tasks.

[0127] Regarding this step, in some possible implementations, the instance identification information contained in the first task chain can be read sequentially, and each instance in the first task chain can be traversed as the first instance to be exchanged. For each first instance to be exchanged that is traversed, the task to be processed associated with the first instance to be exchanged can be obtained, so as to determine whether there is a second target task that meets the exchange conditions.

[0128] Furthermore, for each first instance to be swapped encountered, a corresponding second instance to be swapped is determined. The second instance to be swapped has the same instance pattern as the first instance to be swapped, and the completion time of the second instance to be swapped is less than that of the first instance to be swapped. Here, the second instance to be swapped refers to an instance that has the same instance pattern as the first instance to be swapped and whose completion time is earlier than that of the first instance to be swapped. This instance has the time basis to swap tasks with the first instance to balance the load.

[0129] Regarding this step, in some possible implementations, the instance mode identifier information of the first instance to be swapped can be obtained, other instances in the first task allocation scheme that have the same instance mode identifier information can be traversed, and instances whose completion time is earlier than that of the first instance to be swapped can be identified as the second instance to be swapped. The difference between the completion time of the first instance to be swapped and the completion time of the second instance to be swapped can be calculated as the idle time of the second instance to be swapped.

[0130] Furthermore, if a second target task exists among the pending tasks associated with the first instance to be swapped, and a third target task exists among the pending tasks associated with the second instance to be swapped, and the processing time of the second target task is greater than the processing time of the third target task, and the difference between the processing times of the second and third target tasks is less than the idle time of the second instance to be swapped, then the second target task and the third target task will be swapped. The idle time of the second instance to be swapped is the difference between the completion time of the first instance to be swapped and the completion time of the second instance to be swapped. Here, the second target task refers to the pending task associated with the first instance to be swapped that has a relatively long processing time and is suitable for migration to the second instance to be swapped for execution; the third target task refers to the pending task associated with the second instance to be swapped that has a relatively short processing time and is suitable for migration to the first instance to be swapped for execution.

[0131] Regarding this step, in some possible implementations, the pending tasks associated with the first instance to be exchanged can be traversed, and the processing time of each pending task can be extracted as the processing time of the candidate second target task. The pending tasks associated with the second instance to be exchanged can be traversed, and the processing time of each pending task can be extracted as the processing time of the candidate third target task. The pending tasks whose processing time of the candidate second target task is greater than the processing time of the candidate third target task, and whose difference between the processing time of the candidate second target task and the processing time of the candidate third target task is less than the idle time of the second instance to be exchanged, are respectively determined as the second target task and the third target task. The mapping association between the second target task and the first instance to be exchanged, as well as the mapping association between the third target task and the second instance to be exchanged, are removed. The task code, start time, and duration of the third target task are added to the task record of the first instance to be exchanged, and the task code, start time, and duration of the second target task are added to the task record of the second instance to be exchanged, so as to exchange the second target task and the third target task.

[0132] It should be noted that by swapping the second target task with a longer processing time on the first swap instance with the third target task with a shorter processing time on the second swap instance, the second swap instance can continue executing the second target task swapped from the first swap instance after completing its original third target task, thus filling the idle time of the second swap instance. At the same time, after releasing the second target task, the first swap instance can switch to executing the third target task with a shorter processing time, thereby shortening the completion time of the first swap instance. When the first swap instance is a key instance on the first task chain that determines the overall completion time, this swap operation can shorten the processing time of the first task chain.

[0133] Understandably, if there is no second target task with a processing time greater than that of the third target task among the pending tasks associated with the first instance to be swapped, the mapping relationship between the current first instance to be swapped and the pending tasks remains unchanged; if there is no third target task with a processing time less than that of the second target task among the pending tasks associated with the second instance to be swapped, the mapping relationship between the current second instance to be swapped and the pending tasks remains unchanged; if the difference between the processing time of the second target task and the processing time of the third target task is greater than or equal to the idle time of the second instance to be swapped, the completion time of the second instance to be swapped after the swap will exceed the original completion time of the first instance to be swapped, and the processing time of the first task chain cannot be shortened, so the swap operation is not performed; if the processing time of the second target task is less than or equal to the processing time of the third target task, the completion time of the first instance to be swapped after the swap will be extended, and the processing time of the first task chain cannot be shortened, so the swap operation is not performed.

[0134] Further, update the first task chain and its processing time, and return to the step of traversing the first instance to be swapped associated with the first task chain until the processing time of the first task chain no longer decreases, then end the traversal to obtain the second task allocation scheme.

[0135] Regarding this step, in some possible implementations, after the second and third target tasks are swapped, the completion times of the first and second instances to be swapped are recalculated. Based on the updated completion times of each instance, the critical path in the first task allocation scheme is redefined, and the updated critical path is determined as the updated first task chain. The maximum completion time of each instance on the updated first task chain is determined as the processing time of the updated first task chain. It is then determined whether the processing time of the updated first task chain is shorter than the processing time of the first task chain before the update. If it is indeed shorter, the step of traversing the first instances to be swapped associated with the first task chain is returned, and the search for swappable tasks continues. If the processing time of the first task chain is no longer shortened, the current task allocation scheme is determined as the second task allocation scheme, and the optimization process ends.

[0136] In this embodiment, by obtaining the first task chain and its processing time of the first task allocation scheme, traversing the first instances to be swapped associated with the first task chain, determining the second instances to be swapped with the same instance mode and earlier completion time for each first instance to be swapped, and swapping the second target task with the third target task under the condition of satisfying the processing time difference constraint, and iteratively updating the first task chain and its processing time until it no longer shortens, dynamic adjustment of task allocation relationship is realized based on the first task allocation scheme. This adjustment reduces the cumulative processing time of the first instance to be swapped by swapping the second target task with the third target task with the shorter processing time on the second instance to be swapped, and fills the idle time of the second instance to be swapped, thereby shortening the maximum completion time of each instance in the first task chain, and obtaining a second task allocation scheme with a completion time less than the first task allocation scheme, thereby improving the task scheduling efficiency for the GPU.

[0137] In one embodiment, the step of "performing an optimization operation on the first task allocation scheme to obtain a second task allocation scheme" can be further refined and may include the following steps: Perform the first optimization operation on the first task allocation scheme to obtain the intermediate task allocation scheme; Perform a second optimization operation on the intermediate task allocation scheme to obtain a second task allocation scheme.

[0138] Specifically, considering that a single optimization operation has limited adjustment range for the task allocation scheme and is difficult to solve the problems of unbalanced load and excessively long task chains at the same time, this embodiment proposes a phased optimization mechanism. By first performing a migration operation to balance the instance load, and then performing a swap operation to compress the task chain length, a progressive optimization of the task allocation scheme is achieved.

[0139] First, a first optimization operation needs to be performed on the first task allocation scheme to obtain an intermediate task allocation scheme. The intermediate task allocation scheme refers to a transitional task allocation scheme formed after the first optimization operation, used to connect with the second optimization operation. In this scheme, the mapping relationship between each instance and the task to be processed has been adjusted according to the migration operation, and its overall completion time is less than that of the first task allocation scheme.

[0140] For details on the specific execution process of the first optimization operation, please refer to the following embodiment.

[0141] Furthermore, a second optimization operation is performed on the intermediate task allocation scheme to obtain a second task allocation scheme.

[0142] For details on the specific execution process of the second optimization operation, please refer to the following embodiment.

[0143] In this embodiment, by sequentially executing the first optimization operation and the second optimization operation, the pending tasks associated with instances with later completion times on the task chain are first migrated to instances with the same pattern and earlier completion times to fill the idle time period. Then, the pending tasks on different instances are swapped to compress the critical path of the task chain. This achieves a progressive optimization of the first task allocation scheme and results in a second task allocation scheme with further shortened completion time.

[0144] In one embodiment, the step of "performing a first optimization operation on the first task allocation scheme to obtain an intermediate task allocation scheme" can be further refined and may include the following steps: Obtain the first task chain of the first task allocation scheme, and the processing time of the first task chain; Iterate through the instances to be migrated out associated with the first task chain; For each instance to be migrated out, determine the corresponding instance to be migrated in; the instance to be migrated in and the instance to be migrated out have the same instance schema. If, among the pending tasks associated with the instance to be migrated out, there is a first target task whose processing time is less than or equal to the idle time of the instance to be migrated in, then the first target task will be migrated from the instance to be migrated out to the instance to be migrated in; the idle time of the instance to be migrated in is the difference between the completion time of the instance to be migrated out and the completion time of the instance to be migrated in. Update the first task chain and its processing time, and return to the step of traversing the instances to be migrated associated with the first task chain until the processing time of the first task chain no longer decreases, then end the traversal to obtain the intermediate task allocation scheme.

[0145] Specifically, the first step is to obtain the first task chain of the first task allocation scheme and its processing time. The first task chain refers to the sequence of critical tasks that determines the overall completion time of the first task allocation scheme. The sum of the processing times of the instances in this task chain and their assigned tasks determines the completion time of the first task allocation scheme. The processing time of the first task chain refers to the total time required to execute all tasks in the first task chain, i.e., the maximum time required for each instance in the first task chain to complete its assigned task.

[0146] Regarding this step, in some possible implementations, the mapping relationship between each instance and the task to be processed in the first task allocation scheme can be parsed, the processing time of the task to be processed assigned to each instance can be identified, the critical path in the first task allocation scheme can be determined according to the completion time of each instance, the sequence of instances constituting the critical path and its associated task to be processed can be determined as the first task chain, and the processing time of the first task chain can be determined according to the processing time of the task to be processed associated with each instance on the first task chain.

[0147] Furthermore, iterate through the instances to be migrated out associated with the first task chain. Here, instances to be migrated out refer to those in the first task chain that have the potential to migrate and assign pending tasks, specifically instances in the first task chain that have a later completion time and have pending tasks that can be migrated.

[0148] Regarding this step, in some possible implementations, the instance identification information contained in the first task chain can be read sequentially, and each instance in the first task chain can be traversed as an instance to be migrated out. For each instance to be migrated out that is traversed, the task to be processed associated with that instance can be obtained, so as to determine whether there is a first target task that meets the migration conditions.

[0149] Furthermore, for each instance to be migrated out, a corresponding instance to be migrated in is determined; the instance to be migrated in has the same instance schema as the instance to be migrated out. Here, the instance to be migrated in refers to an instance that has the same instance schema as the instance to be migrated out, and whose completion time is earlier than that of the instance to be migrated out, and this instance has time slack to receive migration tasks.

[0150] Regarding this step, in some possible implementations, the instance mode identifier information of the instance to be migrated out can be obtained, other instances with the same instance mode identifier information in the first task allocation scheme can be traversed, and instances whose completion time is earlier than that of the instance to be migrated out can be identified as instances to be migrated in. The difference between the completion time of the instance to be migrated in and the completion time of the instance to be migrated out can be calculated as the idle time of the instance to be migrated in.

[0151] Furthermore, if among the pending tasks associated with the instance to be migrated out, there exists a first target task whose processing time is less than or equal to the idle time of the instance to be migrated in, then the first target task will be migrated from the instance to be migrated out to the instance to be migrated in. The idle time of the instance to be migrated in is the difference between the completion time of the instance to be migrated out and the completion time of the instance to be migrated in. Here, the first target task refers to the pending tasks associated with the instance to be migrated out, whose processing time is less than or equal to the idle time of the instance to be migrated in, and which is suitable for migration to the instance to be migrated in for execution; the idle time refers to the difference between the completion time of the instance to be migrated in and the completion time of the instance to be migrated out, representing the time margin by which the instance to be migrated in completes earlier than the instance to be migrated out, and this time margin can be used to accommodate the first target task migrated from the instance to be migrated out.

[0152] Regarding this step, in some possible implementations, the pending tasks associated with the instance to be migrated out can be traversed, the processing time of each pending task can be extracted, and the pending tasks whose processing time is less than or equal to the idle time of the instance to be migrated in can be identified as the first target task. The mapping association between the first target task and the instance to be migrated out can be removed, and the task code, start time and duration of the first target task can be added to the task record of the instance to be migrated in, so as to migrate the first target task from the instance to be migrated out to the instance to be migrated in.

[0153] Further, update the first task chain and its processing time, and return to the step of traversing the instances to be migrated associated with the first task chain until the processing time of the first task chain no longer decreases, then end the traversal to obtain the intermediate task allocation scheme.

[0154] Regarding this step, in some possible implementations, after the first target task migration is completed, the completion times of the instances to be migrated out and the instances to be migrated in are recalculated. Based on the updated completion times of each instance, the critical path in the first task allocation scheme is re-determined, and the updated critical path is determined as the updated first task chain. The maximum completion time of each instance on the updated first task chain is determined as the processing time of the updated first task chain. It is then determined whether the processing time of the updated first task chain is shorter than the processing time of the first task chain before the update. If it is indeed shorter, the step of traversing the instances to be migrated out associated with the first task chain is returned, and the search for migrateable tasks continues. If the processing time of the first task chain is no longer shortened, the current task allocation scheme is determined as the intermediate task allocation scheme, and the optimization process ends.

[0155] In this embodiment, by obtaining the first task chain and its processing time of the first task allocation scheme, traversing the instances to be migrated out associated with the first task chain, determining the instances to be migrated in with the same instance mode for each instance to be migrated out, and migrating the first target task whose processing time is less than or equal to the idle time of the instance to be migrated in from the instance to be migrated out to the instance to be migrated in, and iteratively updating the first task chain and its processing time until it no longer shortens, dynamic adjustment of task allocation relationship is realized based on the first task allocation scheme. This adjustment shortens the processing time of the first task chain by balancing the load distribution among instances with the same instance mode, filling in idle periods, and reducing idle computing resources, thereby obtaining an intermediate task allocation scheme with a completion time less than the first task allocation scheme, providing an optimization basis for the subsequent second optimization operation, and thus improving the task scheduling efficiency for the GPU.

[0156] In one embodiment, the step of "performing a second optimization operation on the intermediate task allocation scheme to obtain a second task allocation scheme" can be further refined and may include the following steps: Obtain the second task chain of the intermediate task allocation scheme, and the processing time of the second task chain; Iterate through the first instance to be swapped associated with the second task chain; For each first instance to be swapped encountered, determine the corresponding second instance to be swapped; the second instance to be swapped has the same instance mode as the first instance to be swapped, and the completion time of the second instance to be swapped is less than the completion time of the first instance to be swapped. If a second target task exists among the pending tasks associated with the first instance to be swapped, and a third target task exists among the pending tasks associated with the second instance to be swapped, and the processing time of the second target task is greater than the processing time of the third target task, and the difference between the processing times of the second and third target tasks is less than the idle time of the second instance to be swapped, then the second target task and the third target task will be swapped. The idle time of the second instance to be swapped is the difference between the completion time of the first instance to be swapped and the completion time of the second instance to be swapped. Update the second task chain and its processing time, and return to the step of traversing the first instance to be swapped associated with the second task chain until the processing time of the second task chain no longer decreases, then end the traversal to obtain the second task allocation scheme.

[0157] Specifically, the first step is to obtain the second task chain of the intermediate task allocation scheme and its processing time. The second task chain refers to the sequence of critical tasks that determines the overall completion time of the intermediate task allocation scheme. The sum of the processing times of the instances in this task chain and their assigned tasks determines the completion time of the intermediate task allocation scheme. The processing time of the second task chain refers to the total time required to execute all tasks in the second task chain, i.e., the maximum time required for each instance in the second task chain to complete its assigned task.

[0158] Regarding this step, in some possible implementations, the mapping relationship between each instance and the task to be processed in the intermediate task allocation scheme can be parsed, the processing time of the task to be processed assigned to each instance can be identified, the critical path in the intermediate task allocation scheme can be determined according to the completion time of each instance, the sequence of instances constituting the critical path and its associated task to be processed can be determined as the second task chain, and the processing time of the second task chain can be determined according to the processing time of the task to be processed associated with each instance in the second task chain.

[0159] Furthermore, iterate through the first instance to be exchanged associated with the second task chain. Here, the first instance to be exchanged refers to the instance contained in the second task chain that has the potential to exchange and distribute pending tasks. Specifically, it is an instance in the second task chain that has a later completion time and whose associated pending tasks include an exchangeable task.

[0160] Regarding this step, in some possible implementations, the instance identification information contained in the second task chain can be read sequentially, and each instance in the second task chain can be traversed as the first instance to be exchanged. For each first instance to be exchanged that is traversed, the task to be processed associated with the first instance to be exchanged can be obtained, so as to determine whether there is a second target task that meets the exchange conditions.

[0161] Furthermore, for each first instance to be swapped encountered, a corresponding second instance to be swapped is determined. The second instance to be swapped has the same instance pattern as the first instance to be swapped, and the completion time of the second instance to be swapped is less than that of the first instance to be swapped. Here, the second instance to be swapped refers to an instance that has the same instance pattern as the first instance to be swapped and whose completion time is earlier than that of the first instance to be swapped. This instance has the time basis to swap tasks with the first instance to balance the load.

[0162] Regarding this step, in some possible implementations, the instance mode identifier information of the first instance to be swapped can be obtained, other instances with the same instance mode identifier information in the intermediate task allocation scheme can be traversed, and instances whose completion time is earlier than that of the first instance to be swapped can be identified as the second instance to be swapped. The difference between the completion time of the first instance to be swapped and the completion time of the second instance to be swapped can be calculated as the idle time of the second instance to be swapped.

[0163] Furthermore, if a second target task exists among the pending tasks associated with the first instance to be swapped, and a third target task exists among the pending tasks associated with the second instance to be swapped, and the processing time of the second target task is greater than the processing time of the third target task, and the difference between the processing times of the second and third target tasks is less than the idle time of the second instance to be swapped, then the second target task and the third target task will be swapped. The idle time of the second instance to be swapped is the difference between the completion time of the first instance to be swapped and the completion time of the second instance to be swapped. Here, the second target task refers to the pending task associated with the first instance to be swapped that has a relatively long processing time and is suitable for migration to the second instance to be swapped for execution; the third target task refers to the pending task associated with the second instance to be swapped that has a relatively short processing time and is suitable for migration to the first instance to be swapped for execution.

[0164] Regarding this step, in some possible implementations, the pending tasks associated with the first instance to be exchanged can be traversed, and the processing time of each pending task can be extracted as the processing time of the candidate second target task. The pending tasks associated with the second instance to be exchanged can be traversed, and the processing time of each pending task can be extracted as the processing time of the candidate third target task. The pending tasks whose processing time of the candidate second target task is greater than the processing time of the candidate third target task, and whose difference between the processing time of the candidate second target task and the processing time of the candidate third target task is less than the idle time of the second instance to be exchanged, are respectively determined as the second target task and the third target task. The mapping association between the second target task and the first instance to be exchanged, as well as the mapping association between the third target task and the second instance to be exchanged, are removed. The task code, start time, and duration of the third target task are added to the task record of the first instance to be exchanged, and the task code, start time, and duration of the second target task are added to the task record of the second instance to be exchanged, so as to exchange the second target task and the third target task.

[0165] Further, update the second task chain and its processing time, and return to the step of traversing the first instance to be swapped associated with the second task chain until the processing time of the second task chain no longer decreases, then end the traversal to obtain the second task allocation scheme.

[0166] Regarding this step, in some possible implementations, after the second and third target tasks are swapped, the completion times of the first and second instances to be swapped are recalculated. Based on the updated completion times of each instance, the critical path in the intermediate task allocation scheme is re-determined, and the updated critical path is determined as the updated second task chain. The maximum completion time of each instance on the updated second task chain is determined as the processing time of the updated second task chain. It is then determined whether the processing time of the updated second task chain is shorter than the processing time of the unupdated second task chain. If it is indeed shorter, the step of traversing the first instances to be swapped associated with the second task chain is returned, and the search for swappable tasks continues. If the processing time of the second task chain is no longer shortened, the current task allocation scheme is determined as the second task allocation scheme, and the optimization process ends.

[0167] In this embodiment, by obtaining the second task chain and its processing time of the intermediate task allocation scheme, the first swapped instances associated with the second task chain are traversed. For each first swapped instance, a second swapped instance with the same instance mode and an earlier completion time is determined. Under the condition of satisfying the processing time difference constraint, the second target task is swapped with the third target task. The second task chain and its processing time are iteratively updated until they no longer shorten, realizing the dynamic adjustment of the task allocation relationship based on the intermediate task allocation scheme. This adjustment reduces the cumulative processing time of the first swapped instance by swapping the second target task with the third target task with the shorter processing time on the second swapped instance, and fills the idle time of the second swapped instance. This shortens the maximum completion time of each instance in the second task chain, resulting in a second task allocation scheme with a completion time less than that of the intermediate task allocation scheme, thereby improving the task scheduling efficiency for the GPU.

[0168] In one embodiment, for easier understanding of the mapping relationship between graphics processor hardware resources and instance modes in this application, please refer to [link to relevant documentation]. Figure 3 , Figure 3 This is a schematic diagram of GPU instance mode resource configuration provided in an embodiment of this application.

[0169] Specifically, Figure 3 This diagram illustrates the various instance modes supported by a graphics processor with four streaming multiprocessors and their respective combinations of streaming multiprocessor usage. The units labeled SM1, SM2, SM3, and SM4 correspond to the first, second, third, and fourth streaming multiprocessors, respectively.

[0170] exist Figure 3In the hierarchical structure shown, from bottom to top, the first layer consists of four independent streaming multiprocessors (SM1, SM2, SM3, SM4); the second layer consists of four instance modes A, each occupying one streaming multiprocessor; the third layer has two instance modes A occupying SM1 and SM2 respectively on the left, and one instance mode B occupying SM3 and SM4 on the right; the fourth layer has one instance mode B occupying SM1 and SM2 on the left, and two instance modes A occupying SM3 and SM4 respectively on the right; the fifth layer consists of two instance modes B, one of which occupies SM1 and SM2. SM2, another instance mode B occupies SM3 and SM4; in the 6th layer, the left side is an instance mode A occupying SM1, the middle is an instance mode B occupying SM2 and SM3, and the right side is an instance mode A occupying SM4; in the 7th layer, the left side is an instance mode A occupying SM1, and the right side is an instance mode C occupying SM2, SM3, and SM4; in the 8th layer, the left side is an instance mode C occupying SM1, SM2, and SM3, and the right side is an instance mode A occupying SM4; in the 9th layer, an instance mode D occupies SM1, SM2, SM3, and SM4. Instance mode A corresponds to a slice of computing resources formed by one streaming multiprocessor, instance mode B corresponds to a slice of computing resources formed by two streaming multiprocessors, instance mode C corresponds to a slice of computing resources formed by three streaming multiprocessors, and instance mode D corresponds to a slice of computing resources formed by four streaming multiprocessors. The computing resources occupied by the upper-level instance modes are an aggregation of the computing resources occupied by one or more lower-level instance modes, and the streaming multiprocessors occupied by each instance mode are physically contiguous.

[0171] In this embodiment, the hierarchical resource allocation structure defines the mapping relationship between instance modes and streaming multiprocessors in the graphics processing unit (GPU). Instance modes A, B, C, and D are composed of one, two, three, and four streaming multiprocessors, respectively, providing a configuration basis for dynamically allocating computing resources according to the instance mode requirements of the task to be processed. By obtaining the hardware attributes of the GPU to determine the number of streaming multiprocessors it possesses and the instance modes it supports, the allocation and on-demand combination of GPU computing resources can be achieved.

[0172] In one embodiment, for easier understanding of the content of this application regarding the task migration mechanism and task chain compression in the first optimization operation, please refer to [link to relevant documentation]. Figure 4 , Figure 4 This is a schematic diagram of task scheduling and resource allocation in a multi-instance mode provided in the embodiments of this application.

[0173] Specifically, Figure 4It consists of two parts: a task list status diagram on the left and a task chain execution sequence diagram on the right. The left part shows the arrangement of each task to be processed in the corresponding task list after being sorted by processing time. The right part shows the resource allocation of the GPU's first streaming multiprocessor SM1, second streaming multiprocessor SM2, third streaming multiprocessor SM3 and fourth streaming multiprocessor SM4 and the task execution sequence relationship after migration optimization.

[0174] On the left side, from top to bottom, are the task lists for Instance Mode A, Instance Mode B, Instance Mode C, and Instance Mode D. Instance Mode A contains Task 2, Task 4, Task 5, Task 6, Task 13, and Task 14; Instance Mode B contains Task 1, Task 3, and Task 7; Instance Mode C contains Task 8, Task 9, and Task 10; and Instance Mode D contains Task 11 and Task 12. Within each of these task lists, the tasks are arranged from left to right in descending order of processing time.

[0175] In the task scheduling execution sequence section on the right, the four streaming multiprocessors of the GPU are identified by SM1, SM2, SM3, and SM4, with the time progression proceeding from bottom to top. In the initial stage, the GPU's computing resources are first allocated to the instance corresponding to instance mode D, so that tasks 11 and 12 use all the computing resources of SM1 to SM4 for execution. After the above tasks are completed, an instance destruction and creation operation node appears to release the computing resources occupied by the instance corresponding to instance mode D and reallocate computing resources.

[0176] The process then enters a mixed execution phase of instance mode C and instance mode A: the GPU's computing resources are divided into two parts. One part is used to create instances corresponding to instance mode C to execute tasks 8, 9, and 10, which occupy the computing resources of SM1 to SM3. The other part is used to create instances corresponding to instance mode A to execute tasks 2 and 5, which occupy the computing resources of SM4. After the above tasks are completed, another instance destruction and creation operation node appears.

[0177] Finally, the execution phase of mixed instance mode B and instance mode A begins: GPU computing resources are further divided. One part is used to create instances corresponding to instance mode B to execute task 1, which uses the computing resources of SM1 and SM2. The other part is used to create instances corresponding to instance mode B to execute tasks 3 and 7, which use the computing resources of SM3 and SM4. After task 7 is completed, an instance destruction and creation operation node appears, creating two instances corresponding to instance mode A to execute tasks 6, 13, and 14. Task 6 uses the computing resources of SM3, while tasks 13 and 14 use the computing resources of SM4.

[0178] In the above process, the order in which each task is arranged in the corresponding task list is consistent with its allocation order in the execution sequence on the right. This reflects the sorting result of sorting multiple tasks in descending order of their processing time and adding them to the corresponding task list in sequence.

[0179] In addition, the height of each rectangle in the figure represents the time required for the GPU to run the corresponding task as an instance in the corresponding instance mode, and the width represents the number of streaming multiprocessors used by the corresponding task during its execution.

[0180] In this embodiment, multiple tasks to be processed are stored in corresponding task lists according to their instance modes, and sorted in descending order of processing time within each task list. The tasks are then assigned to the corresponding instances of the GPU for execution based on the sorting results, thus establishing a mapping relationship between tasks and computing resources. At the same time, by destroying and recreating instances after the larger instance mode has been completed to reconstruct computing resources, and allocating the remaining computing resources to instances corresponding to smaller instance modes to execute other tasks, dynamic adjustment and efficient utilization of computing resources are achieved.

[0181] In one embodiment, for easier understanding of the content of this application regarding the task migration mechanism and the compression of the first task chain in the first optimization operation, please refer to [link to relevant documentation]. Figure 5 , Figure 5 This is a schematic diagram comparing the task scheduling timing before and after the first optimization operation, provided as an embodiment of this application.

[0182] Specifically, Figure 5 The timeline is presented in two parts: the left side represents the task allocation state before the first optimization operation, and the right side represents the task allocation state after the first optimization operation. The GPU's computing resource units are identified by horizontally arranged SM1 to SM4, with the vertical axis representing the time progression direction.

[0183] In the left-hand section, GPU computing resources are first allocated to the instance corresponding to instance mode D, and tasks 11 and 12 are executed sequentially. These tasks consume all computing resources from SM1 to SM4. After the tasks are completed, an instance destruction and creation operation is performed to release instance resources and reallocate computing resources.

[0184] The process then enters a mixed execution phase of instance mode C and instance mode A: the GPU's computing resources are divided into two parts. The first part is used to create instances corresponding to instance mode C to execute tasks 8, 9, and 10 sequentially, with these three tasks occupying computing resources from SM1 to SM3. The second part is used to create instances corresponding to instance mode A to execute tasks 2 and 5, with these tasks occupying computing resources from SM4. After the above tasks are completed, the instance destruction and creation operation is performed again.

[0185] The process then enters a mixed execution phase of instance mode B and instance mode A: the GPU's computing resources are re-divided into two parts. The first part corresponds to the instance to be migrated out formed in instance mode B, and tasks 1 and 7 are executed sequentially. These tasks occupy the computing resources of SM1 and SM2, with task 7 located above task 1. The completion time of the instance to be migrated out is determined by the end time of task 7. The second part corresponds to the instance formed in instance mode B and the instance formed in instance mode A. Task 3 occupies the computing resources of SM3 and SM4. After task 3 is completed, the instance destruction and creation operation is performed, followed by the sequential execution of tasks 6, 13, and 14. Task 6 occupies the computing resources of SM3, while tasks 13 and 14 occupy the computing resources of SM4.

[0186] In the right-hand section, through the first optimization operation, the first target task is migrated from the instance to be migrated out to the instance to be migrated in. Specifically, task 7 is migrated from the instance to be migrated out, which occupies SM1 and SM2, to the instance to be migrated in, which occupies SM3 and SM4 (the instance executing task 3), so that task 7 is positioned above task 3 and executed below the instance destruction and creation operation. The instance to be migrated in and the instance to be migrated out have the same instance mode, and the completion time of the instance to be migrated in is earlier than that of the instance to be migrated out. The processing time of task 7 is less than or equal to the idle time of the instance to be migrated in, where the idle time is the difference between the completion time of the instance to be migrated out and the completion time of the instance to be migrated in. It can be seen that the overall task processing time is shortened after the migration.

[0187] Through the above migration operation, task 7 no longer occupies the computing resources of SM1 and SM2 for execution, but is migrated to the computing resources of SM3 and SM4 for execution, which shortens the processing time of the first task chain, thus obtaining a task allocation state with a completion time of less than the first task allocation scheme.

[0188] In this embodiment, by migrating the first target task, whose processing time is less than or equal to the idle time, from the outgoing instance with a later completion time to the incoming instance with an earlier completion time and the same instance pattern, the idle time of the incoming instance is filled, the load distribution among different instances is balanced, the processing time of the first task chain is shortened, and the first task allocation scheme is optimized.

[0189] In one embodiment, for easier understanding of the content of this application regarding the task exchange mechanism and the second task chain compression in the second optimization operation, please refer to [link to relevant documentation]. Figure 6 , Figure 6 This is a schematic diagram comparing the task scheduling timing before and after the second optimization operation, provided as an embodiment of this application.

[0190] Specifically, Figure 6 The timeline is presented in two parts: the left side represents the task allocation state before the second optimization operation, and the right side represents the task allocation state after the second optimization operation. The figure uses SM1, SM2, SM3, and SM4 arranged horizontally to identify the multiple streaming multiprocessors of the GPU, and the vertical direction represents the time progression.

[0191] In the left-hand section, GPU computing resources are first allocated to the instance corresponding to instance mode D, and tasks 11 and 12 are executed sequentially. These tasks consume all computing resources from SM1 to SM4. After the tasks are completed, an instance destruction and creation operation is performed to release instance resources and reallocate computing resources.

[0192] The process then enters a mixed execution phase of instance mode C and instance mode A: the GPU's computing resources are divided into two parts. The first part is used to create instances corresponding to instance mode C to execute tasks 8, 9, and 10 sequentially. These three tasks occupy computing resources from SM1 to SM3, and tasks 8, 9, and 10 are all marked with the constraint that instance mode C is required. The second part is used to create instances corresponding to instance mode A to execute tasks 2 and 5. These tasks occupy computing resources from SM4. After the above tasks are completed, the instance destruction and creation operation is performed again.

[0193] The process then proceeds to a mixed execution phase of instance mode B and instance mode A: GPU computing resources are re-divided into two parts. The first part includes the first instance to be swapped formed by instance mode B and instances formed by instance mode A, executing tasks 1 and 14 sequentially. These tasks occupy the computing resources of SM1 and SM2, with task 14 positioned above task 1. The completion time of the first instance to be swapped is determined by the end time of task 14. After task 14 completes, an instance destruction and creation operation is performed, creating two instances to execute tasks 13 and 15 respectively, occupying SM1 and SM2. The second part includes the second instance to be swapped formed by instance mode B and instances formed by instance mode A, with tasks 3 and 7 occupying the computing resources of SM3 and SM4, and task 7 positioned above task 3. After task 7 completes, an instance destruction and creation operation is performed, creating instances corresponding to instance mode A to execute tasks 16 and 6, with task 16 occupying the computing resources of SM3 and task 6 occupying the computing resources of SM4. The first instance to be swapped (the instance executing tasks 1 and 14) and the second instance to be swapped (the instance executing tasks 3 and 7) have the same instance pattern, and the completion time of the second instance to be swapped is less than the completion time of the first instance to be swapped.

[0194] In the right-hand section, through a second optimization operation, the second target task (Task 14) is swapped with the third target task (Task 7). Specifically, Task 14 is swapped from the first instance to be swapped, occupying SM1 and SM2, to the second instance to be swapped, occupying SM3 and SM4, so that Task 14 is positioned above Task 3 and executed below the instance destruction / creation operation. Simultaneously, Task 7 is swapped from the second instance to the first instance to be swapped, so that Task 7 is positioned above Task 1 and executed below the instance destruction / creation operation. The processing time of Task 14 is greater than that of Task 7, and the difference between the processing times of Task 14 and Task 7 is less than the idle time of the second instance to be swapped, where the idle time is the difference between the completion time of the first instance to be swapped and the completion time of the second instance to be swapped. Therefore, the overall task processing time is shortened after the swap.

[0195] Through the above exchange operation, task 14 no longer occupies the computing resources of SM1 and SM2 for execution, but is exchanged for execution on the computing resources of SM3 and SM4. At the same time, task 7 is exchanged for execution on the computing resources of SM1 and SM2, thereby shortening the processing time of the second task chain.

[0196] In this embodiment, by swapping the second target task with a longer processing time on the first swapping instance with the third target task with a shorter processing time on the second swapping instance, the cumulative processing time of the task on the first swapping instance is reduced, and the idle time of the second swapping instance is filled, thereby shortening the maximum completion time of each instance on the second task chain and obtaining a second task allocation scheme with a shorter completion time, thereby improving the task scheduling efficiency for the GPU.

[0197] The following will combine Figure 7 The task scheduling device 800 provided in this application embodiment will be described in detail. The task scheduling device 800 and the task scheduling method described above can be referred to and correspond to each other. Specifically, the task scheduling device 800 may include an acquisition module 810, a determination module 820, and an optimization module 830, as detailed below: The acquisition module 810 is used to acquire the instance mode and processing order of each of the multiple pending tasks in the GPU; The determination module 820 is used to determine a first task allocation scheme based on the instance mode and processing order of each task to be processed. The first task allocation scheme is used to indicate the mapping relationship between the instances created on the GPU and the tasks to be processed. The optimization module 830 is used to perform optimization operations on the first task allocation scheme to obtain a second task allocation scheme, wherein the completion time of the second task allocation scheme is less than the completion time of the first task allocation scheme; the optimization operation includes at least one of the first optimization operation and the second optimization operation; the first optimization operation is used to migrate the pending task specified by one instance to another instance between two instances with the same instance mode; the second optimization operation is used to exchange the pending task specified by one instance with the pending task specified by another instance between two instances with the same instance mode.

[0198] Optionally, in some embodiments, the acquisition module 810 may be used to: The instance mode of each pending task is determined based on the task attributes of each pending task in the multiple pending tasks of the GPU. Get the processing time for each pending task; Multiple task lists are created based on the multiple instance modes supported by the GPU, with each instance mode corresponding to a task list. Based on the instance mode and processing time of each pending task, add each pending task to the corresponding task list in sequence; The processing order of each task is determined based on the order of the tasks to be processed in multiple task lists.

[0199] Optionally, in some embodiments, the acquisition module 810 may be used to: Get the execution time, creation time, and destruction time of each pending task; The processing time for each pending task is determined based on its execution time, creation time, and destruction time.

[0200] Optionally, in some embodiments, the acquisition module 810 may be used to: Based on the processing time of each task, sort the tasks in descending order of processing time to obtain the total task sequence. Based on the instance pattern of each pending task in the overall task sequence, each pending task is added to the corresponding task list in sequence.

[0201] Optionally, in some embodiments, the acquisition module 810 may be used to: Based on the instance pattern of each pending task, multiple pending tasks are divided into multiple task sets, and each task set corresponds one-to-one with an instance pattern. For each of the multiple task sets, the tasks to be processed within each task set are added to the corresponding task list in descending order of their processing time.

[0202] Optionally, in some embodiments, the determining module 820 may be used to: Based on the order of computing resources consumed by the multiple instance modes supported by the GPU from most to least, traverse multiple instance modes. For the first instance pattern encountered during traversal, determine the first computing resource occupied by the first instance pattern, and determine the first instance based on the first computing resource; The unassigned tasks corresponding to the first instance mode are assigned to the first instance in order of processing. In the case where the GPU's computing resources include a second computing resource in addition to the first computing resource, a second instance is determined based on the second computing resource, and at least a portion of the unallocated pending tasks corresponding to the second instance mode are allocated to the second instance; the second instance mode is an instance mode other than the first instance mode among multiple instance modes, and the duration of the first instance is greater than or equal to the duration of the second instance. If multiple pending tasks have been assigned to the corresponding instances, the traversal ends to obtain the first task assignment scheme.

[0203] Optionally, in some embodiments, the determining module 820 may be used to: During the lifetime of the first instance, with the goal of maximizing the total processing time of the pending tasks allocated to the second instance, at least a portion of the pending tasks are selected from the unallocated pending tasks corresponding to the second instance mode. At least some of the pending tasks will be assigned to the second instance.

[0204] Optionally, in some embodiments, the optimization module 830 may be used to: Obtain the first task chain of the first task allocation scheme, and the processing time of the first task chain; Iterate through the instances to be migrated out associated with the first task chain; For each instance to be migrated out, determine the corresponding instance to be migrated in; the instance to be migrated in and the instance to be migrated out have the same instance schema. If, among the pending tasks associated with the instance to be migrated out, there is a first target task whose processing time is less than or equal to the idle time of the instance to be migrated in, then the first target task will be migrated from the instance to be migrated out to the instance to be migrated in; the idle time of the instance to be migrated in is the difference between the completion time of the instance to be migrated out and the completion time of the instance to be migrated in. Update the first task chain and its processing time, and return to the step of traversing the instances to be migrated associated with the first task chain until the processing time of the first task chain no longer decreases, then end the traversal to obtain the second task allocation scheme.

[0205] Optionally, in some embodiments, the optimization module 830 may be used to: Obtain the first task chain of the first task allocation scheme, and the processing time of the first task chain; Iterate through the first instance to be swapped associated with the first task chain; For each first instance to be swapped encountered, determine the corresponding second instance to be swapped; the second instance to be swapped has the same instance mode as the first instance to be swapped, and the completion time of the second instance to be swapped is less than the completion time of the first instance to be swapped. If a second target task exists among the pending tasks associated with the first instance to be swapped, and a third target task exists among the pending tasks associated with the second instance to be swapped, and the processing time of the second target task is greater than the processing time of the third target task, and the difference between the processing times of the second and third target tasks is less than the idle time of the second instance to be swapped, then the second target task and the third target task will be swapped. The idle time of the second instance to be swapped is the difference between the completion time of the first instance to be swapped and the completion time of the second instance to be swapped. Update the first task chain and its processing time, and return to the step of traversing the first instance to be swapped associated with the first task chain until the processing time of the first task chain no longer decreases, then end the traversal to obtain the second task allocation scheme.

[0206] Optionally, in some embodiments, the optimization module 830 may be used to: Perform the first optimization operation on the first task allocation scheme to obtain the intermediate task allocation scheme; Perform a second optimization operation on the intermediate task allocation scheme to obtain a second task allocation scheme.

[0207] Optionally, in some embodiments, the optimization module 830 may be used to: Obtain the first task chain of the first task allocation scheme, and the processing time of the first task chain; Iterate through the instances to be migrated out associated with the first task chain; For each instance to be migrated out, determine the corresponding instance to be migrated in; the instance to be migrated in and the instance to be migrated out have the same instance schema. If, among the pending tasks associated with the instance to be migrated out, there is a first target task whose processing time is less than or equal to the idle time of the instance to be migrated in, then the first target task will be migrated from the instance to be migrated out to the instance to be migrated in; the idle time of the instance to be migrated in is the difference between the completion time of the instance to be migrated out and the completion time of the instance to be migrated in. Update the first task chain and its processing time, and return to the step of traversing the instances to be migrated associated with the first task chain until the processing time of the first task chain no longer decreases, then end the traversal to obtain the intermediate task allocation scheme.

[0208] Optionally, in some embodiments, the optimization module 830 may be used to: Obtain the second task chain of the intermediate task allocation scheme, and the processing time of the second task chain; Iterate through the first instance to be swapped associated with the second task chain; For each first instance to be swapped encountered, determine the corresponding second instance to be swapped; the second instance to be swapped has the same instance mode as the first instance to be swapped, and the completion time of the second instance to be swapped is less than the completion time of the first instance to be swapped. If a second target task exists among the pending tasks associated with the first instance to be swapped, and a third target task exists among the pending tasks associated with the second instance to be swapped, and the processing time of the second target task is greater than the processing time of the third target task, and the difference between the processing times of the second and third target tasks is less than the idle time of the second instance to be swapped, then the second target task and the third target task will be swapped. The idle time of the second instance to be swapped is the difference between the completion time of the first instance to be swapped and the completion time of the second instance to be swapped. Update the second task chain and its processing time, and return to the step of traversing the first instance to be swapped associated with the second task chain until the processing time of the second task chain no longer decreases, then end the traversal to obtain the second task allocation scheme.

[0209] The effects achievable in this embodiment can be found in the relevant embodiments of the task scheduling method described above, and will not be repeated here.

[0210] Figure 8 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 8 As shown, the electronic device may include: a processor 1301, a communication interface 1302, a memory 1303, and a communication bus 1304, wherein the processor 1301, the communication interface 1302, and the memory 1303 communicate with each other via the communication bus 1304. The processor 1301 can call a computer program in the memory 1303 to execute the steps of a task scheduling method, such as including: Obtain the instance mode and processing order of each of the multiple pending tasks on the GPU; The first task allocation scheme is determined based on the instance mode and processing order of each task to be processed. The first task allocation scheme is used to indicate the mapping relationship between the instances created on the GPU and the tasks to be processed. An optimization operation is performed on the first task allocation scheme to obtain a second task allocation scheme. The completion time of the second task allocation scheme is less than that of the first task allocation scheme. The optimization operation includes at least one of a first optimization operation and a second optimization operation; the first optimization operation is used to migrate a task to be processed specified in one instance to another instance between two instances with the same instance pattern; the second optimization operation is used to exchange a task to be processed specified in one instance with a task to be processed specified in another instance between two instances with the same instance pattern.

[0211] Furthermore, when the logical instructions in the aforementioned memory can be implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0212] On the other hand, embodiments of this application also provide a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can perform the steps of the task scheduling method provided in the above embodiments, such as including: Obtain the instance mode and processing order of each of the multiple pending tasks on the GPU; The first task allocation scheme is determined based on the instance mode and processing order of each task to be processed. The first task allocation scheme is used to indicate the mapping relationship between the instances created on the GPU and the tasks to be processed. An optimization operation is performed on the first task allocation scheme to obtain a second task allocation scheme. The completion time of the second task allocation scheme is less than that of the first task allocation scheme. The optimization operation includes at least one of a first optimization operation and a second optimization operation; the first optimization operation is used to migrate a task to be processed specified in one instance to another instance between two instances with the same instance pattern; the second optimization operation is used to exchange a task to be processed specified in one instance with a task to be processed specified in another instance between two instances with the same instance pattern.

[0213] On the other hand, embodiments of this application also provide a non-transitory computer-readable storage medium storing a computer program. The computer program is used to cause a processor to execute the steps of the methods provided in the above embodiments, including, for example: Obtain the instance mode and processing order of each of the multiple pending tasks on the GPU; The first task allocation scheme is determined based on the instance mode and processing order of each task to be processed. The first task allocation scheme is used to indicate the mapping relationship between the instances created on the GPU and the tasks to be processed. An optimization operation is performed on the first task allocation scheme to obtain a second task allocation scheme. The completion time of the second task allocation scheme is less than that of the first task allocation scheme. The optimization operation includes at least one of a first optimization operation and a second optimization operation; the first optimization operation is used to migrate a task to be processed specified in one instance to another instance between two instances with the same instance pattern; the second optimization operation is used to exchange a task to be processed specified in one instance with a task to be processed specified in another instance between two instances with the same instance pattern.

[0214] Non-transitory computer-readable storage media can be any available medium or data storage device that can be accessed by a processor, including but not limited to magnetic storage (e.g., floppy disks, hard disks, magnetic tapes, magneto-optical disks (MOs), etc.), optical storage (e.g., CDs, DVDs, BDs, HVDs, etc.), and semiconductor storage (e.g., ROMs, EPROMs, EEPROMs, non-volatile memory (NAND flash), solid-state drives (SSDs)).

[0215] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0216] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods of various embodiments or some parts of embodiments.

[0217] All actions involving the acquisition of signal information or data in this application were carried out in compliance with the relevant data protection laws and policies of the country where the application is located, and with the authorization granted by the owner of the relevant device. Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A task scheduling method, characterized in that, include: Obtain the instance mode and processing order of each of the multiple pending tasks in the graphics processing unit (GPU); A first task allocation scheme is determined based on the instance mode and processing order of each task to be processed. The first task allocation scheme is used to indicate the mapping relationship between the instances created on the GPU and the tasks to be processed. An optimization operation is performed on the first task allocation scheme to obtain a second task allocation scheme, the completion time of the second task allocation scheme being less than the completion time of the first task allocation scheme; The optimization operation includes at least one of a first optimization operation and a second optimization operation; the first optimization operation is used to migrate a task to be processed specified by one instance to another instance between two instances with the same instance pattern; the second optimization operation is used to exchange a task to be processed specified by one instance with a task to be processed specified by another instance between two instances with the same instance pattern.

2. The method according to claim 1, characterized in that, The process of obtaining the instance mode and processing order of each of the multiple tasks to be processed in the GPU includes: The instance mode of each task to be processed is determined based on the task attributes of each task among the multiple tasks to be processed in the GPU. Obtain the processing time for each of the pending tasks; Multiple task lists are created based on the multiple instance modes supported by the GPU, and each instance mode corresponds one-to-one with the task list; Based on the instance mode and processing time of each pending task, each pending task is added to the corresponding task list in sequence; The processing order of each task to be processed is determined according to the order of the tasks to be processed in the multiple task lists.

3. The method according to claim 2, characterized in that, The step of obtaining the processing time for each pending task includes: Obtain the execution time, creation time, and destruction time of each pending task; The processing time for each pending task is determined based on its execution time, creation time, and destruction time.

4. The method according to claim 2, characterized in that, The step of adding each pending task to its corresponding task list sequentially based on its instance mode and processing time includes: Based on the processing time of each task to be processed, the multiple tasks to be processed are sorted in descending order to obtain the total task sequence. Based on the instance pattern of each pending task in the overall task sequence, each pending task is added sequentially to the corresponding task list.

5. The method according to claim 2, characterized in that, The step of adding each pending task to its corresponding task list sequentially based on its instance mode and processing time includes: Based on the instance pattern of each task to be processed, the multiple tasks to be processed are divided into multiple task sets, and the task sets correspond one-to-one with the instance patterns. For each of the multiple task sets, the tasks to be processed contained within each task set are added to the corresponding task list in descending order of their processing times.

6. The method according to claim 1, characterized in that, The step of determining the first task allocation scheme based on the instance mode and processing order of each task to be processed includes: Based on the order of computing resources occupied by the multiple instance modes supported by the GPU from most to least, traverse the multiple instance modes. For the first instance pattern encountered during traversal, determine the first computing resource occupied by the first instance pattern, and determine the first instance based on the first computing resource; The unassigned tasks corresponding to the first instance mode are assigned to the first instance in order of processing. When the GPU's computing resources include a second computing resource other than the first computing resource, a second instance is determined based on the second computing resource, and at least a portion of the unallocated pending tasks corresponding to the second instance mode are allocated to the second instance; the second instance mode is an instance mode other than the first instance mode among the plurality of instance modes, and the duration of the first instance is greater than or equal to the duration of the second instance. If all the pending tasks have been assigned to the corresponding instances, the traversal ends to obtain the first task allocation scheme.

7. The method according to claim 6, characterized in that, The step of allocating at least a portion of the unassigned pending tasks corresponding to the second instance mode to the second instance includes: During the lifetime of the first instance, with the goal of maximizing the total processing time of the pending tasks allocated to the second instance, at least a portion of the pending tasks are selected from the unallocated pending tasks corresponding to the second instance mode. At least a portion of the tasks to be processed are assigned to the second instance.

8. The method according to claim 1, characterized in that, When the optimization operation includes the first optimization operation, the step of performing the optimization operation on the first task allocation scheme to obtain the second task allocation scheme includes: Obtain the first task chain of the first task allocation scheme, and the processing time of the first task chain; Iterate through the instances to be migrated associated with the first task chain; For each instance to be migrated out that is traversed, a corresponding instance to be migrated in is determined; the instance to be migrated in and the instance to be migrated out have the same instance pattern. If, among the pending tasks associated with the instance to be migrated out, there exists a first target task whose processing time is less than or equal to the idle time of the instance to be migrated in, then the first target task is migrated from the instance to be migrated out to the instance to be migrated in; the idle time of the instance to be migrated in is the difference between the completion time of the instance to be migrated out and the completion time of the instance to be migrated in. Update the first task chain and its processing time, and return to the step of traversing the instances to be migrated associated with the first task chain until the processing time of the first task chain no longer decreases, then end the traversal to obtain the second task allocation scheme.

9. The method according to claim 1, characterized in that, When the optimization operation includes the second optimization operation, the step of performing the optimization operation on the first task allocation scheme to obtain the second task allocation scheme includes: Obtain the first task chain of the first task allocation scheme, and the processing time of the first task chain; Iterate through the first instance to be exchanged associated with the first task chain; For each first instance to be swapped encountered, a corresponding second instance to be swapped is determined; the second instance to be swapped has the same instance mode as the first instance to be swapped, and the completion time of the second instance to be swapped is less than the completion time of the first instance to be swapped. If a second target task exists among the pending tasks associated with the first instance to be swapped, and a third target task exists among the pending tasks associated with the second instance to be swapped, and the processing time of the second target task is greater than the processing time of the third target task, and the difference between the processing times of the second target task and the third target task is less than the idle time of the second instance to be swapped, then the second target task and the third target task will be swapped; the idle time of the second instance to be swapped is the difference between the completion time of the first instance to be swapped and the completion time of the second instance to be swapped. Update the first task chain and its processing time, and return to the step of traversing the first instance to be swapped associated with the first task chain until the processing time of the first task chain no longer decreases, then end the traversal to obtain the second task allocation scheme.

10. The method according to claim 1, characterized in that, When the optimization operation includes the first optimization operation and the second optimization operation, the step of performing the optimization operation on the first task allocation scheme to obtain the second task allocation scheme includes: Perform the first optimization operation on the first task allocation scheme to obtain an intermediate task allocation scheme; The second optimization operation is performed on the intermediate task allocation scheme to obtain the second task allocation scheme.

11. The method according to claim 10, characterized in that, The step of performing the first optimization operation on the first task allocation scheme to obtain an intermediate task allocation scheme includes: Obtain the first task chain of the first task allocation scheme, and the processing time of the first task chain; Iterate through the instances to be migrated associated with the first task chain; For each instance to be migrated out that is traversed, a corresponding instance to be migrated in is determined; the instance to be migrated in and the instance to be migrated out have the same instance pattern. If, among the pending tasks associated with the instance to be migrated out, there exists a first target task whose processing time is less than or equal to the idle time of the instance to be migrated in, then the first target task is migrated from the instance to be migrated out to the instance to be migrated in; the idle time of the instance to be migrated in is the difference between the completion time of the instance to be migrated out and the completion time of the instance to be migrated in. Update the first task chain and its processing time, and return to the step of traversing the instances to be migrated associated with the first task chain until the processing time of the first task chain no longer decreases, then end the traversal to obtain an intermediate task allocation scheme.

12. The method according to claim 10, characterized in that, The step of performing the second optimization operation on the intermediate task allocation scheme to obtain the second task allocation scheme includes: Obtain the second task chain of the intermediate task allocation scheme, and the processing time of the second task chain; Iterate through the first instance to be exchanged associated with the second task chain; For each first instance to be swapped encountered, a corresponding second instance to be swapped is determined; the second instance to be swapped has the same instance mode as the first instance to be swapped, and the completion time of the second instance to be swapped is less than the completion time of the first instance to be swapped. If a second target task exists among the pending tasks associated with the first instance to be swapped, and a third target task exists among the pending tasks associated with the second instance to be swapped, and the processing time of the second target task is greater than the processing time of the third target task, and the difference between the processing times of the second target task and the third target task is less than the idle time of the second instance to be swapped, then the second target task and the third target task will be swapped; the idle time of the second instance to be swapped is the difference between the completion time of the first instance to be swapped and the completion time of the second instance to be swapped. Update the second task chain and its processing time, and return to the step of traversing the first instance to be swapped associated with the second task chain until the processing time of the second task chain no longer decreases, then end the traversal to obtain the second task allocation scheme.

13. A task scheduling device, characterized in that, include: The acquisition module is used to acquire the instance mode and processing order of each task among multiple tasks to be processed in the graphics processing unit (GPU). The determining module is used to determine a first task allocation scheme based on the instance mode and processing order of each task to be processed, wherein the first task allocation scheme is used to indicate the mapping relationship between the instances created on the GPU and the tasks to be processed; An optimization module is used to perform an optimization operation on the first task allocation scheme to obtain a second task allocation scheme, wherein the completion time of the second task allocation scheme is less than the completion time of the first task allocation scheme. The optimization operation includes at least one of a first optimization operation and a second optimization operation; the first optimization operation is used to migrate a task to be processed specified by one instance to another instance between two instances with the same instance pattern; the second optimization operation is used to exchange a task to be processed specified by one instance with a task to be processed specified by another instance between two instances with the same instance pattern.

14. An electronic device comprising a processor and a memory storing a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the task scheduling method according to any one of claims 1 to 12.

15. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the task scheduling method according to any one of claims 1 to 12.

16. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the task scheduling method according to any one of claims 1 to 12.

Images