Task scheduling method and device and electronic equipment

Abstract

The invention discloses a task scheduling method and device, and relates to the field of cross-cluster scheduling, and the method comprises the steps: receiving N tasks, N being a positive integer; first computing power resource demand information corresponding to each task is obtained, N scores corresponding to N computing power clusters are obtained, one score corresponds to one computing power cluster, the scores are used for representing the resource fragmentation degree of the computing power clusters, and the computing power resource demand information is used for indicating the computing power resource demand degree of the tasks; and scheduling each task to the corresponding computing power cluster based on the first computing power resource demand information and the N scores corresponding to each task.

Description

Technical Field

[0001] The present application relates to the field of cross-cluster scheduling, and in particular to a task scheduling method, device, and electronic device. Background Art

[0002] At present, with the development of resource scheduling technology, efficient resource scheduling can be achieved in a multi-task parallel environment.

[0003] In related technologies, GPU sharing technology is used to improve GPU utilization in heterogeneous cluster resource scheduling, and scheduling strategies are optimized by predicting task running time. For example, the shortest job optimization scheduling strategy is used to reduce the average completion time of tasks, thereby improving the efficiency of multi-task parallel processing.

[0004] However, after completing resource scheduling through the above method, scattered resource fragments often appear in the computing cluster. These resource fragments are usually wasted due to the lack of effective perception and utilization mechanisms, significantly reducing the resource utilization of the computing cluster. Summary of the Invention

[0005] The present application provides a task scheduling method, device and electronic device for improving resource utilization of a computing cluster.

[0006] To achieve the above objectives, this application adopts the following technical solutions:

[0007] In a first aspect, a task scheduling method is provided, the method comprising: receiving N tasks, where N is a positive integer; obtaining first computing power resource demand information corresponding to each task, and obtaining N scores corresponding to N computing power clusters, one score corresponding to one computing power cluster, the score being used to characterize the degree of resource fragmentation of the computing power cluster, and the computing power resource demand information being used to indicate the degree of computing power resource demand of the task; and scheduling each task to the corresponding computing power cluster based on the first computing power resource demand information and the N scores corresponding to each task.

[0008] Based on the above-mentioned task scheduling method, N tasks are received, where N is a positive integer; the first computing power resource requirement information corresponding to each task is obtained, and N scores corresponding to N computing power clusters are obtained, with one score corresponding to one computing power cluster. The score is used to characterize the degree of resource fragmentation of the computing power cluster, and the computing power resource requirement information is used to indicate the degree of computing power resource requirement of the task; based on the first computing power resource requirement information corresponding to each task and the N scores, each task is scheduled to the corresponding computing power cluster. In this solution, since the scores corresponding to the computing power clusters obtained can characterize the degree of resource fragmentation of the computing power clusters, the severity of the resource fragmentation of the computing power clusters can be determined through the scores corresponding to each computing power cluster, so that each task can be scheduled to the corresponding computing power cluster according to the computing power resource requirement information of each task; that is, by sensing resource fragmentation, each task is scheduled to the corresponding computing power cluster according to the computing power resource requirement information of each task. For example, tasks with low computing power resource requirements are scheduled to computing power clusters with severe resource fragmentation, and tasks with high computing power resource requirements are scheduled to computing power clusters with slight resource fragmentation. In this way, fragmented resources are integrated to fully tap the system potential, reduce resource waste, and improve the resource utilization of computing power clusters.

[0009] In one possible implementation, each task is scheduled to a corresponding computing power cluster based on the first computing power resource demand information and N scores corresponding to each task, including: when the first task is a high-demand task, the first task is assigned to a computing power cluster that meets a first condition, and the high-demand task is a task whose computing power resource demand information is greater than or equal to a preset threshold, and the first condition includes: the first computing power resource demand information of the first task is met among the N computing power clusters and the score is the lowest among the N scores; or, when the first task is a low-demand task, the first task is assigned to a computing power cluster that meets a second condition, and the low-demand task is a task whose computing power resource demand information is less than a preset threshold, and the second condition includes: the first computing power resource demand information of the first task is met among the N computing power clusters and the score is the highest among the N scores; wherein the first task is any one of the N tasks.

[0010] In one possible implementation, obtaining the first computing power resource requirement information corresponding to each task includes: obtaining the first information corresponding to the second task among N tasks, the first information including: task parameter information, global feature information of the first model corresponding to the second task, and operator indicator parameter information of the first model; inputting the first information into the first predictor, performing computing power resource prediction on the second task through the first predictor, and obtaining the second computing power resource requirement information corresponding to the independent operation of the second task, so as to obtain the second computing power resource requirement information corresponding to the independent operation of each task; inputting the second computing power resource requirement information corresponding to the independent operation of each task and the global feature information of the model corresponding to each task into the second predictor, performing computing power resource prediction on each task through the second predictor, and obtaining the first computing power resource requirement information corresponding to the parallel operation of each task.

[0011] In one possible implementation, N scores corresponding to N computing power clusters are obtained, including: calculating a first score corresponding to the graphics processor based on the video memory size of the graphics processor, the number of graphics processors, and the video memory distribution coefficient in the first computing power cluster, where the first computing power cluster is any one of the N computing power clusters; calculating a second score corresponding to the central processing unit based on the number of cores and the number of nodes of the central processing unit in the first computing power cluster; and performing a weighted summation of the first score and the second score based on the weight corresponding to the graphics processor and the weight corresponding to the central processing unit to obtain a score corresponding to the first computing power cluster, so as to obtain N scores corresponding to the N computing power clusters.

[0012] In one possible implementation, a first score corresponding to the graphics processor is calculated based on the graphics memory size, the number of graphics processors, and the memory distribution coefficient of the graphics processor in the first computing power cluster, including: dividing the free graphics memory size of the graphics processor by the total graphics memory size to obtain a first value; dividing the number of free graphics processors of the graphics processor by the total number of graphics processors to obtain a second value; and multiplying the first value, the second value, and the memory distribution coefficient to obtain the first score.

[0013] In one possible implementation, based on the number of cores and the number of nodes of the central processing unit in the first computing power cluster, a second score corresponding to the central processing unit is calculated, including: dividing the number of idle cores of the central processing unit by the total number of cores to obtain a third value; dividing the number of idle nodes of the central processing unit by the total number of nodes to obtain a fourth value; and multiplying the third value and the fourth value to obtain the second score.

[0014] In a second aspect, a task scheduling device is provided. The task scheduling device is applied to an electronic device and can also be a functional module in the electronic device for implementing the method described in the first aspect or any possible design of the first aspect. The task scheduling device can implement the functions performed by the electronic device in the above aspects or possible designs, and the functions can be implemented by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions. For example, the task scheduling device includes: a receiving unit, an acquisition unit, and a scheduling unit.

[0015] The receiving unit is used to receive N tasks, where N is a positive integer.

[0016] An acquisition unit is used to obtain the first computing power resource requirement information corresponding to each task received by the receiving unit, and to obtain N scores corresponding to N computing power clusters, where one score corresponds to one computing power cluster. The score is used to characterize the degree of resource fragmentation of the computing power cluster, and the computing power resource requirement information is used to indicate the degree of computing power resource requirement of the task.

[0017] The scheduling unit is configured to schedule each task to a corresponding computing power cluster based on the first computing power resource requirement information and the N scores corresponding to each task acquired by the acquisition unit.

[0018] The specific implementation of the task scheduling device can refer to the behavior function of the electronic device in the task scheduling method provided in the first aspect or any possible design of the first aspect, and will not be repeated here. Therefore, the provided task scheduling device can achieve the same beneficial effects as the first aspect or any possible design of the first aspect.

[0019] In one possible implementation, the scheduling unit is specifically used to, when the first task is a high-demand task, assign the first task to a computing power cluster that meets a first condition. The high-demand task is a task whose computing power resource demand information is greater than or equal to a preset threshold. The first condition includes: the first computing power resource demand information of the first task is met among N computing power clusters and the score is the lowest among the N scores; or, when the first task is a low-demand task, assign the first task to a computing power cluster that meets a second condition. The low-demand task is a task whose computing power resource demand information is less than a preset threshold. The second condition includes: the first computing power resource demand information of the first task is met among N computing power clusters and the score is the highest among the N scores; wherein the first task is any one of the N tasks.

[0020] In one possible implementation, the acquisition unit is specifically used to obtain first information corresponding to the second task, where the first information includes: task parameter information, global feature information of the first model corresponding to the second task, and operator indicator parameter information of the first model, where the second task is any one of the N tasks; and input the first information into the first predictor, perform computing power resource prediction on the second task through the first predictor, and obtain the second computing power resource requirement information corresponding to the independent operation of the second task, so as to obtain the second computing power resource requirement information corresponding to the independent operation of each task; and input the second computing power resource requirement information corresponding to the independent operation of each task and the global feature information of the model corresponding to each task into the second predictor, perform computing power resource prediction on each task through the second predictor, and obtain the first computing power resource requirement information corresponding to the parallel operation of each task.

[0021] In one possible implementation, the acquisition unit is specifically used to calculate a first score corresponding to the graphics processor based on the video memory size of the graphics processor, the number of graphics processors, and the video memory distribution coefficient in the first computing power cluster, where the first computing power cluster is any one of the N computing power clusters; and to calculate a second score corresponding to the central processing unit based on the number of cores and the number of nodes of the central processing unit in the first computing power cluster; and to perform a weighted sum of the first score and the second score based on the weight corresponding to the graphics processor and the weight corresponding to the central processing unit to obtain a score corresponding to the first computing power cluster, so as to obtain N scores corresponding to the N computing power clusters.

[0022] In one possible implementation, the acquisition unit is specifically configured to divide the free video memory size of the graphics processor by the total video memory size to obtain a first value; divide the number of free graphics processors of the graphics processor by the total number of graphics processors to obtain a second value; and multiply the first value, the second value, and the video memory distribution coefficient to obtain a first score.

[0023] In one possible implementation, the acquisition unit is specifically used to divide the number of idle cores of the central processing unit by the total number of cores to obtain a third value; and divide the number of idle nodes of the central processing unit by the total number of nodes to obtain a fourth value; and multiply the third value and the fourth value to obtain a second score.

[0024] In a third aspect, a task scheduling device is provided. The task scheduling device may be an electronic device or a chip or system-on-chip in the electronic device. The task scheduling device may implement the functions performed by the electronic device in the aforementioned aspects or possible designs. The functions may be implemented through hardware. For example, in one possible design, the task scheduling device may include a processor and a communication interface. The processor may be used to support the task scheduling device in implementing the functions involved in the aforementioned first aspect or any possible design of the first aspect.

[0025] In another possible design, the task scheduling device may further include a memory for storing computer-executable instructions and data necessary for the task scheduling device. When the task scheduling device is running, the processor executes the computer-executable instructions stored in the memory to cause the task scheduling device to perform the task scheduling method described in the first aspect or any possible design of the first aspect.

[0026] In a fourth aspect, a task scheduling device is provided, which can be a chip or a system-on-chip. The task scheduling device can implement the functions performed by the task scheduling device in the above-mentioned aspects or possible designs. The functions can be implemented through hardware. For example, in one possible design, the task scheduling device can include: a processor and a communication interface. The processor can be used to support the task scheduling device in implementing the functions involved in the above-mentioned first aspect or any possible design of the first aspect.

[0027] In another possible design, the task scheduling device may further include a memory for storing computer-executable instructions and data necessary for the task scheduling device. When the task scheduling device is running, the processor executes the computer-executable instructions stored in the memory to cause the task scheduling device to perform the task scheduling method described in the first aspect or any possible design of the first aspect.

[0028] In the fifth aspect, a computer-readable storage medium is provided, which may be a readable non-volatile storage medium. The computer-readable storage medium stores computer instructions or programs, and when the computer is run on a computer, enables the computer to execute the task scheduling method described in the first aspect or any possible design of the above aspects.

[0029] In a sixth aspect, a computer program product comprising instructions is provided, which, when executed on a computer, enables the computer to execute the task scheduling method described in the first aspect or any possible design of the above aspects.

[0030] In a seventh aspect, a task scheduling device is provided. The task scheduling device may be a chip or a system-on-chip, and the task scheduling device includes one or more processors and one or more memories. The one or more memories are coupled to the one or more processors and are used to store computer program code. The computer program code includes computer instructions. When the one or more processors execute the computer instructions, the task scheduling device performs the task scheduling method described in the first aspect or any possible design of the first aspect.

[0031] In an eighth aspect, an electronic device is provided, comprising: a memory and a processor, wherein the memory is used to store one or more programs; and the processor is used to implement the task scheduling method as described in any one of the above-mentioned first aspects when the one or more programs are executed by the processor.

[0032] In a ninth aspect, a chip system is provided, comprising a processor and a communication interface. The chip system can be used to implement the functions performed by the task scheduling device in the first aspect or any possible design of the first aspect. In one possible design, the chip system further comprises a memory configured to store program instructions and / or data. The chip system can be composed solely of a chip or can include a chip and other discrete components, without limitation.

[0033] Among them, the technical effects brought about by any design method from the second aspect to the ninth aspect can refer to the technical effects brought about by the above-mentioned first aspect, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 One of the flowcharts of a task scheduling method provided in an embodiment of the present application;

[0035] Figure 2 This is a second flowchart of a task scheduling method provided in an embodiment of the present application;

[0036] Figure 3 Flowchart 3 of a task scheduling method provided in an embodiment of the present application;

[0037] Figure 4 Flowchart 4 of a task scheduling method provided in an embodiment of the present application;

[0038] Figure 5 Flowchart 5 of a task scheduling method provided in an embodiment of the present application;

[0039] Figure 6 Flowchart 6 of a task scheduling method provided in an embodiment of the present application;

[0040] Figure 7 One of the structural diagrams of a task scheduling framework provided in an embodiment of the present application;

[0041] Figure 8 A second structural diagram of a task scheduling framework provided in an embodiment of the present application;

[0042] Figure 9 A third structural diagram of a task scheduling framework provided in an embodiment of the present application;

[0043] Figure 10 A fourth structural diagram of a task scheduling framework provided in an embodiment of the present application;

[0044] Figure 11 Flowchart 7 of a task scheduling method provided in an embodiment of the present application;

[0045] Figure 12 Flowchart 8 of a task scheduling method provided in an embodiment of the present application;

[0046] Figure 13 A schematic diagram of the structure of a task scheduling device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0047] In order to enable ordinary people in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings.

[0048] It should be noted that the terms "first," "second," and the like in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or precedence. It should be understood that the numbers used in this manner are interchangeable where appropriate so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the embodiments of the present application as detailed in the appended claims.

[0049] It will also be understood that the term “comprising” indicates the presence of described features, integers, steps, operations, elements and / or components, but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements and / or components.

[0050] The following is a detailed explanation of the professional terms involved in the embodiments of this application.

[0051] Resource fragmentation: This occurs when system resources, such as memory, storage, and network bandwidth, are divided into multiple, discontinuous pieces, making them inefficiently utilized or allocated. This reduces resource utilization, increases management overhead, and can lead to performance degradation or system instability.

[0052] Fragmentation Resource Awareness: Fragmentation resource awareness refers to the storage system's strategy for managing the performance and availability of fragmented data on storage devices. Fragmentation occurs when data is stored on a device in small chunks rather than in large, contiguous blocks. Fragmented data leads to low storage efficiency, long access times, and low device utilization. Fragmentation resource awareness addresses fragmentation by prioritizing data allocation to unfragmented areas, using a merging algorithm to combine adjacent fragmented data blocks into larger ones, and using a reserved space mechanism to reserve a certain amount of storage space specifically for merging and organizing fragmented data.

[0053] Resource forecasting is the process of estimating system resources, such as computing power, memory, storage, and network bandwidth, by analyzing historical data, current status, or future needs. Its core goal is to predict resource consumption trends in advance, thereby optimizing resource allocation, avoiding resource shortages or waste, and improving system performance and stability.

[0054] Shortest Job First (SJF): This is a resource scheduling strategy that prioritizes tasks with the shortest estimated execution time to minimize average system wait time, improve throughput, or optimize resource utilization. It is commonly used in scenarios such as operating system process scheduling, cloud computing task allocation, and manufacturing assembly line management.

[0055] Host Performance Measurement (HPM): Host performance measurement is the process of evaluating the efficiency, stability, and resource utilization of computer systems, such as servers and personal computers, during operation using a series of metrics and tools. Its purpose is to identify performance bottlenecks, optimize resource allocation, and ensure that the system can meet business needs.

[0056] High-demand tasks are tasks that place significant and concentrated demands on system resources, such as the central processing unit (CPU), memory, input / output (I / O), and network, or on quality of service (QoS), such as latency and throughput. These tasks typically require priority resource allocation or special scheduling strategies to avoid becoming system performance bottlenecks or impacting business objectives.

[0057] Low-demand tasks: These tasks have low requirements for system resources, such as CPU, memory, and I / O, or have low requirements for execution timeliness and quality of service (QoS), such as latency and throughput. These tasks typically allow for delayed execution and can be flexibly scheduled when resources are idle to optimize overall resource utilization.

[0058] A cluster is a technical architecture that connects multiple computers, also known as nodes, through software and networks to form a unified system that collaborates to complete specific tasks. Clusters aim to improve system performance, availability, and scalability, and are widely used in cloud computing, big data, high-performance computing, and other fields.

[0059] Model parameters: Model parameters are learnable or configurable variables within the model that directly affect the model's training, inference capabilities, and performance.

[0060] The task scheduling method provided in the embodiment of the present application is described in detail below with reference to the accompanying drawings through specific embodiments and their application scenarios.

[0061] At present, how to achieve efficient resource scheduling in a dynamic, multi-tasking parallel environment is a key issue that needs to be solved urgently. Current technologies have carried out extensive research in resource prediction, scheduling strategy optimization and resource utilization improvement, and have made significant progress in multiple directions.

[0062] Related technologies usually perform resource scheduling through the following five methods.

[0063] Method 1, in terms of resource prediction, proposes to use an adaptive neural network model to train and optimize based on historical data and dynamically adjust the prediction structure to improve the accuracy and efficiency of cloud resource allocation.

[0064] Method 2: Proposes to combine deep learning and reinforcement learning methods to analyze personnel and resource needs through data-driven predictive models to optimize artificial intelligence computing power scheduling.

[0065] Method 3: In heterogeneous cluster resource scheduling, GPU sharing technology is used to improve GPU utilization, and the scheduling strategy is optimized by predicting the task running time. The SJF scheduling strategy is adopted to reduce the average completion time of tasks. For high-GPU demand tasks that are difficult to schedule, a reservation and packaging strategy is adopted to reserve high-end GPU machines for these tasks, while packaging other workloads onto relatively low-end GPU machines.

[0066] Method 4: The scheduling algorithm dynamically evaluates node load through HPM, screens trusted nodes, and achieves load balancing and efficient job allocation.

[0067] Method 5: An algorithm based on two-level dynamic scheduling is proposed: global scheduling dynamically allocates tasks based on node performance, and intra-node scheduling collaboratively allocates tasks between the CPU and GPU through a dual-queue mechanism, significantly improving resource utilization and computing efficiency.

[0068] The key to the related technologies in heterogeneous cluster resource scheduling is how to efficiently predict the performance of tasks and allocate resources. However, the above methods are usually based on the traditional task level or node level analysis to perform resource scheduling, but such methods face the following problems when dealing with complex task scenarios: (1) Insufficient multi-task parallel prediction capabilities: Existing methods usually perform performance prediction for a single task or a simple task combination, lacking effective modeling of resource competition and coordination relationships between tasks, resulting in insufficient performance prediction accuracy in multi-task parallel scenarios, affecting the effectiveness of scheduling decisions. (2) Lack of in-depth analysis at the operator level: Most current technologies remain at a coarse-grained prediction level and cannot perform in-depth analysis at the operator level, making it difficult to identify key operators that affect model performance. (3) Unresolved resource fragmentation: After resource allocation, scattered resource fragments often appear. These fragments are usually wasted due to the lack of effective perception and utilization mechanisms, significantly reducing overall resource utilization. In particular, when dealing with tasks that require large-scale parallel computing, the resource fragmentation problem will be further exacerbated. Existing methods are unable to dynamically perceive and efficiently integrate fragmented resources, resulting in the overall efficiency and scalability of resource scheduling being limited.

[0069] In view of this, an embodiment of the present application provides a task scheduling method for improving resource utilization of a computing cluster.

[0070] The task scheduling method includes: receiving N tasks, where N is a positive integer; obtaining first computing resource demand information corresponding to each task, and obtaining N scores corresponding to N computing clusters, one score corresponding to one computing cluster, and the score being used to characterize the degree of resource fragmentation of the computing cluster; and scheduling each task to the corresponding computing cluster based on the first computing resource demand information and the N scores corresponding to each task.

[0071] Based on the above scheme, since the corresponding scores of the computing power clusters obtained can represent the degree of resource fragmentation of the computing power cluster, the severity of the resource fragmentation of the computing power cluster can be determined through the corresponding scores of each computing power cluster, and each task can be scheduled to the corresponding computing power cluster based on the computing power resource requirement information of each task. In other words, by sensing resource fragmentation, each task is scheduled to the corresponding computing power cluster based on the computing power resource requirement information of each task. For example, tasks with low computing power resource requirements are scheduled to computing power clusters with severe resource fragmentation, and tasks with high computing power resource requirements are scheduled to computing power clusters with less resource fragmentation. In this way, by integrating fragmented resources, the system potential is fully tapped, resource waste is reduced, and the resource utilization rate of the computing power cluster is improved.

[0072] The execution subject of the task scheduling method provided in the embodiment of the present application may be a task scheduling device, which may be an electronic device or a functional module in the electronic device. The following takes the task scheduling device as an example to illustrate the technical solution provided in the embodiment of the present application.

[0073] The method provided in the embodiments of the present application is described in detail below with reference to the accompanying drawings.

[0074] The embodiment of the present application provides a task scheduling method. Figure 1 As shown, the task scheduling may include S101 to S103.

[0075] S101. The task scheduling device receives N tasks, where N is a positive integer.

[0076] Optionally, in an embodiment of the present application, the task scheduling device may receive N tasks via a wired connection; or, the task scheduling device may receive N tasks via a wireless connection. The specific method may be determined based on actual use requirements and is not limited by the embodiment of the present application.

[0077] Exemplarily, the wireless connection mode may be any one of the following: Bluetooth connection or Wireless Fidelity (WiFi) connection.

[0078] Optionally, in an embodiment of the present application, the task type of the above-mentioned N tasks may be any one of the following: a high-demand task and a low-demand task.

[0079] For example, the above tasks may be computing tasks or communication tasks, etc. The specific tasks may be determined according to actual use requirements and are not limited in the embodiments of the present application.

[0080] S102. The task scheduling device obtains first computing resource demand information corresponding to each task, and obtains N scores corresponding to N computing clusters.

[0081] In an embodiment of the present application, one of the above-mentioned N scores corresponds to a computing power cluster, and the score is used to characterize the degree of resource fragmentation of the computing power cluster. The above-mentioned computing power resource demand information is used to indicate the degree of computing power resource demand of the task.

[0082] Optionally, in the embodiment of the present application, the above score can be an integer or a decimal.

[0083] For example, when the score is a decimal, only the last two decimal places may be retained.

[0084] It should be noted that the specific process of obtaining N scores corresponding to the N computing power clusters can be found in the following embodiment, and will not be described again here to avoid repetition.

[0085] It can be understood that the above-mentioned N tasks correspond to N first computing resource demand information, that is, one task corresponds to one first computing resource demand information.

[0086] It should be noted that the specific process of obtaining the first computing resource requirement information corresponding to each of the N tasks can be found in the following embodiments. To avoid repetition, it will not be repeated here.

[0087] Optionally, in the embodiment of the present application, the first computing resource demand information may include but is not limited to at least one of the following: GPU utilization, task execution time, memory usage, video memory usage, etc. The specific information may be determined based on actual usage requirements and is not limited in the embodiment of the present application.

[0088] It should be noted that the higher the above score, the more severe the resource fragmentation of the corresponding computing power cluster.

[0089] S103. The task scheduling device schedules each task to a corresponding computing power cluster based on the first computing power resource requirement information corresponding to each task and the N scores.

[0090] In an embodiment of the present application, the task scheduling device can schedule each of the N tasks to the corresponding computing power cluster based on the size of the first computing power resource demand information corresponding to each of the N tasks and the size of each of the N scores.

[0091] Optionally, in an embodiment of the present application, after determining that each task is scheduled to a corresponding computing power cluster, the task scheduling device may schedule each task to a computing power node in the corresponding computing power cluster.

[0092] Exemplarily, the task scheduling device can obtain K third scores of K computing nodes in the computing power cluster, and the third scores are used to characterize the degree of resource fragmentation of the computing power nodes. Then, the task scheduling device can schedule the task to the corresponding computing power node based on the computing power resource demand information corresponding to the task and the K third scores, where K is a positive integer.

[0093] It can be understood that one third score corresponds to one computing power node.

[0094] It should be noted that the specific implementation methods of determining the computing power cluster corresponding to each task and determining the target computing power node corresponding to each task can be found in the following embodiments. To avoid repetition, they will not be described here.

[0095] Optionally, in an embodiment of the present application, the task scheduling device may store each of the N tasks in a task cache queue, and schedule each of the N tasks in sequence through the task cache queue.

[0096] Optionally, in an embodiment of the present application, the task scheduling device may store the task descriptor corresponding to each of the N tasks in a task cache queue.

[0097] Exemplarily, the above-mentioned task descriptor may include: model type, input data scale, parameter quantity, FLOPs, first resource computing power resource requirement information corresponding to the task, and other information.

[0098] Optionally, in an embodiment of the present application, the task cache queue may adjust the task descriptors in the task cache queue through dynamic priority.

[0099] Exemplarily, the above-mentioned dynamic priority can be determined according to at least one of the following: the importance of the task, the waiting time and the urgency of the resource demand, the network status of the computing power cluster, and the computing power information of the computing power cluster.

[0100] Optionally, in an embodiment of the present application, to avoid resource starvation, the task scheduling device can also determine dynamic priorities based on time. For example, tasks that have not been assigned for a long time will have their priorities gradually increased. In this way, through intelligent management of the task queue, the task scheduling device can still maintain high throughput and low latency in a high-load environment.

[0101] In the task scheduling method provided in the embodiment of the present application, the task scheduling device can receive N tasks, where N is a positive integer; then, the first computing power resource requirement information corresponding to each task is obtained, and N scores corresponding to N computing power clusters are obtained, one score corresponding to one computing power cluster, the score is used to characterize the degree of resource fragmentation of the computing power cluster, and the computing power resource requirement information is used to indicate the degree of computing power resource requirement of the task; finally, based on the first computing power resource requirement information corresponding to each task and the N scores, each task is scheduled to the corresponding computing power cluster. In this solution, since the scores corresponding to the computing power clusters obtained by the task scheduling device can characterize the degree of resource fragmentation of the computing power cluster, the severity of resource fragmentation of the computing power cluster can be determined through the scores corresponding to each computing power cluster, so that the task scheduling device can schedule each task to the corresponding computing power cluster according to the computing power resource requirement information of each task; that is, the task scheduling device schedules each task to the corresponding computing power cluster according to the computing power resource requirement information of each task by sensing resource fragmentation. For example, tasks with low computing power resource requirements are scheduled to computing power clusters with severe resource fragmentation, and tasks with high computing power resource requirements are scheduled to computing power clusters with slight resource fragmentation. In this way, fragmented resources are integrated to fully tap the system potential, reduce resource waste, and improve the resource utilization of computing power clusters.

[0102] In some embodiments, combined Figure 1 ,like Figure 2 As shown, the above S103 can be specifically implemented through the following S103a or S103b.

[0103] S103a: When the first task is a high-demand task, the task scheduling device allocates the first task to a computing power cluster that meets the first condition.

[0104] In an embodiment of the present application, the above-mentioned high-demand task is a task whose computing power resource demand information is greater than or equal to a preset threshold. The above-mentioned first condition includes: the first computing power resource demand information of the first task is met in N computing power clusters and the score is the lowest among the N scores. The above-mentioned first task is any one of the N tasks.

[0105] Optionally, in the embodiment of the present application, the preset threshold value may be preset by the task scheduling device or customized by the user, and may be determined specifically based on actual usage requirements.

[0106] Optionally, in an embodiment of the present application, the above-mentioned preset threshold may be one or more, and the number of the preset thresholds may be determined by the number of computing resources included in the first computing resource demand information.

[0107] For example, assuming that the first computing power resource demand information includes: GPU utilization, task running time, memory usage size, and video memory usage size, and the number of computing power resources is 4, the number of preset thresholds is also 4.

[0108] Optionally, in an embodiment of the present application, when there are multiple preset thresholds, each of the multiple preset thresholds corresponds to a computing power resource.

[0109] Example 1: There are 4 preset thresholds. The first threshold among the 4 preset thresholds corresponds to GPU utilization, and the first threshold is 50%. The second threshold among the 4 preset thresholds corresponds to task running time, and the second threshold is 2s. The third threshold among the 4 preset thresholds corresponds to memory usage size, and the third threshold is 20M. The fourth threshold among the 4 preset thresholds corresponds to video memory usage size, and the fourth threshold is 1M.

[0110] It can be understood that when the number of computing resources in the first computing resource demand information is multiple, the demand for each computing resource in the first computing resource demand information must be greater than or equal to the corresponding preset threshold.

[0111] Example 2. In combination with Example 1 above, taking the GPU utilization of 60%, the task running time of 5s, the memory usage size of 30M, and the video memory usage size of 2M included in the above-mentioned first computing power resource demand information as an example, the task scheduling device can compare the demand for computing power resources included in the first computing power resource demand information with the corresponding thresholds respectively, that is, the GPU utilization of 60% in the first computing power resource demand information is compared with 50% of the 4 preset thresholds; the task running time of 5s in the first computing power resource demand information is compared with 2s of the 4 preset thresholds; the memory usage size of 30M in the first computing power resource demand information is compared with 20M of the 4 preset thresholds; the video memory usage size of 2M in the first computing power resource demand information is compared with 1M of the 4 preset thresholds, and determine that the demand for each computing power resource in the first computing power resource demand information must be greater than or equal to the corresponding preset threshold, so that the task scheduling device determines that the first task is a high-demand task.

[0112] Optionally, in an embodiment of the present application, the task scheduling device may determine whether the first task is a high-demand task based on the task type of the first task.

[0113] For example, when the task type of the first task is a communication task, the task scheduling apparatus may determine that the first task is a high-demand task.

[0114] Optionally, in an embodiment of the present application, the first computing resource requirement information in the N computing clusters that meets the first task may be that the amount of idle resources in the N computing clusters must be greater than or equal to the first computing resource requirement.

[0115] Example 3: Assume that the N computing clusters mentioned above are 5 computing clusters, and the corresponding scores of the 5 computing clusters are 1, 2, 3, 4, and 5 respectively; assume that the computing clusters among the 5 computing clusters that can meet the first computing resource demand information are computing cluster 2, computing cluster 4, and computing cluster 5 respectively. After sorting the scores corresponding to the above 5 computing clusters, computing cluster 2 can be determined as the computing cluster corresponding to the first task, and the first task can be scheduled to computing cluster 2.

[0116] Optionally, in an embodiment of the present application, the task scheduling device may directly assign the first task to the computing power cluster corresponding to the lowest score among the N scores.

[0117] Example 4: In combination with the above Example 3, the task scheduling device can directly schedule the first task to the computing power cluster corresponding to score 1, that is, computing power cluster 1.

[0118] It should be noted that, in combination with the above example 4, since the above example 4 does not take into account the first computing power resource requirement of the first task, after determining the computing power cluster 1, if the idle resources in the computing power cluster 1 are less than the first computing power resource requirement, the task scheduling device can release continuous resources by integrating distributed fragmented resources or migrating tasks, thereby meeting the execution requirements of the first task.

[0119] Optionally, in an embodiment of the present application, after the task scheduling device determines the computing power cluster corresponding to the first task, the task scheduling device may assign the first task to a computing power node in the computing power cluster corresponding to the first task that meets the third condition.

[0120] In an embodiment of the present application, the above-mentioned third condition may be: the first computing resource requirement of the first task is met among the K computing nodes and the score is the lowest among the K third scores.

[0121] Example 5: Assume that the K computing nodes mentioned above are 5 computing nodes, and the third scores corresponding to the 5 computing nodes are 1, 2, 3, 4, and 5 respectively; assume that the computing nodes among the 5 computing nodes that can meet the first computing resource demand information are computing node 2, computing node 4, and computing node 5 respectively. After sorting the third scores corresponding to the above 5 computing nodes, computing node 2 can be determined as the computing node corresponding to the first task, and the first task can be scheduled to computing node 2.

[0122] Optionally, in an embodiment of the present application, the task scheduling device may directly assign the first task to the computing power node corresponding to the lowest score among the K third scores in the computing power cluster corresponding to the first task.

[0123] Example 6: In combination with the above Example 5, the task scheduling device can directly schedule the first task to the computing power node corresponding to the third score 1, that is, computing power node 1.

[0124] It should be noted that, in combination with the above example 6, since the above example 6 does not take into account the first computing resource requirement of the first task, after determining the computing node 1, if the idle resources in the computing node 1 are less than the first computing resource requirement, the task scheduling device can release continuous resources by integrating distributed fragmented resources or migrating tasks, thereby meeting the execution requirements of the first task.

[0125] S103b: When the first task is a low-demand task, the task scheduling device allocates the first task to a computing power cluster that meets the second condition.

[0126] Optionally, in an embodiment of the present application, the task scheduling device may determine whether the first task is a low-demand task based on the task type of the first task.

[0127] For example, when the task type of the first task is a scheduled task, the task scheduling apparatus may determine that the first task is a low-demand task.

[0128] In an embodiment of the present application, the low-demand task is a task whose computing power resource demand information is less than a preset threshold, and the second condition includes: the first computing power resource demand information of the first task is met in N computing power clusters and the score is the highest among the N scores.

[0129] Optionally, in an embodiment of the present application, the above-mentioned computing power resource demand information being less than a preset threshold can be understood as: the demand for all computing power resources in the first computing power resource demand information must be less than the corresponding preset threshold.

[0130] It should be noted that the specific description of the computing resource demand information being less than the preset threshold can be found in the above embodiment. To avoid repetition, it will not be repeated here.

[0131] Optionally, in an embodiment of the present application, the first computing resource requirement information in the N computing clusters that meets the second task may be that the amount of idle resources in the N computing clusters must be greater than or equal to the first computing resource requirement.

[0132] Example 7: Assume that the N computing clusters mentioned above are 5 computing clusters, and the corresponding scores of the 5 computing clusters are 1, 2, 3, 4, and 5 respectively; assume that the computing clusters among the 5 computing clusters that can meet the first computing resource demand information are computing cluster 2, computing cluster 4, and computing cluster 5 respectively. After sorting the scores corresponding to the above 5 computing clusters, computing cluster 5 can be determined as the computing cluster corresponding to the first task, and the first task can be scheduled to computing cluster 5.

[0133] Optionally, in an embodiment of the present application, the task scheduling device may directly assign the first task to the computing power cluster corresponding to the highest score among the N scores.

[0134] Example 8: In combination with the above Example 7, the task scheduling device can directly schedule the first task to the computing power cluster corresponding to the score 5, that is, computing power cluster 5.

[0135] It should be noted that, in combination with the above example 8, since the above example 8 does not take into account the first computing resource requirement of the first task, after determining the computing cluster 5, if the idle resources in the computing cluster 5 are less than the first computing resource requirement, the task scheduling device can wait for the computing cluster 5 to release resources.

[0136] Optionally, in an embodiment of the present application, after the task scheduling device determines the computing power cluster corresponding to the first task, the task scheduling device may assign the first task to a computing power node in the computing power cluster corresponding to the first task that meets the fourth condition.

[0137] In an embodiment of the present application, the above-mentioned third condition may be: the first computing resource requirement of the first task is met among the K computing nodes and the score is the highest among the K third scores.

[0138] Example 9: Assume that the K computing nodes mentioned above are 5 computing nodes, and the third scores corresponding to the 5 computing nodes are 1, 2, 3, 4, and 5 respectively; assume that the computing nodes among the 5 computing nodes that can meet the first computing resource demand information are computing node 2, computing node 4, and computing node 5 respectively. After sorting the third scores corresponding to the above 5 computing nodes, computing node 5 can be determined as the computing node corresponding to the first task, and the first task can be scheduled to computing node 5.

[0139] Optionally, in an embodiment of the present application, the task scheduling device may directly assign the first task to the computing power node corresponding to the highest score among the K third scores in the computing power cluster corresponding to the first task.

[0140] Example 10: In combination with the above Example 9, the task scheduling device can directly schedule the first task to the computing power node corresponding to the third score 5, that is, computing power node 5.

[0141] It should be noted that, in combination with the above example 10, since the above example 10 does not take into account the first computing power resource requirement of the first task, after determining the computing power node 5, if the idle resources in the computing power node 5 are less than the first computing power resource requirement, the task scheduling device can wait for the computing power node 5 to release resources.

[0142] It should be noted that the above is only an example of one task among N tasks. For each of the N tasks, the task scheduling device can determine the computing power cluster corresponding to each task through the above embodiment. To avoid repetition, it will not be repeated here.

[0143] In the embodiments of the present application, the task scheduling device, through resource fragmentation awareness, prioritizes scheduling low-demand tasks to severely fragmented computing clusters or nodes, thereby improving the integrated utilization of scattered resources. At the same time, for high-demand tasks, the task scheduling device can meet their execution needs by integrating distributed fragmented resources or migrating tasks to release continuous resources, thereby enhancing the flexibility of the task scheduling device in scheduling tasks.

[0144] In some embodiments, combined Figure 1 ,like Figure 3 As shown, the "task scheduling device obtains the first computing resource demand information corresponding to each task" in the above S102 can be specifically implemented through the following S102a to S102c.

[0145] S102a: The task scheduling device obtains first information corresponding to the second task.

[0146] In an embodiment of the present application, the above-mentioned first information includes: task parameter information, global feature information of the first model corresponding to the second task, and operator indicator parameter information of the first model, and the above-mentioned second task is any one of the N tasks.

[0147] Optionally, in the embodiment of the present application, the task parameter information may include at least one of the following: system resources required for the task, task identifier, task description information, task data size, i.e., the size of the data contained in the task, etc. The specific information may be determined based on actual usage requirements and is not limited in the embodiment of the present application.

[0148] For example, the system resources required by the above tasks may include at least one of the following: CPU, GPU, memory, video memory, etc. The specific system resources may be determined according to actual use requirements and are not limited in the present embodiment.

[0149] For example, the task identifier may be any of the following: a text identifier, a digital identifier, a symbol identifier, etc. The specific identifier may be determined according to actual use requirements and is not limited in the present embodiment.

[0150] For example, the above text identifier may be a task name.

[0151] For example, the task description information may be a brief description of the task objective, such as “daily log cleaning”.

[0152] Optionally, in the embodiment of the present application, the global feature information of the first model may include at least one of the following: model parameters, number of floating-point operations (Flops), etc. The specific information may be determined according to actual use requirements and is not limited in the embodiment of the present application.

[0153] For example, the above-mentioned model parameters may include at least one of the following: number of layers, number of nodes, convolution kernel size, hyperparameters, weights, model type, etc. The specific parameters may be determined according to actual use requirements and are not limited in the embodiments of the present application.

[0154] Optionally, in the embodiment of the present application, the number of floating-point operations can be preset by the model or customized by the user. The specific number can be determined according to actual use requirements and is not limited by the embodiment of the present application.

[0155] Optionally, in the embodiment of the present application, the operator indicator parameter information of the first model may include at least one of the following performance data: execution time, memory usage, GPU / CPU resource utilization, I / O bottleneck, etc. The specific information can be determined based on actual usage requirements and is not limited by the embodiment of the present application.

[0156] Optionally, in an embodiment of the present application, the task scheduling device may perform operator analysis on the first model to improve the accuracy of operator indicator parameter information obtained by the task scheduling device.

[0157] Exemplarily, the task scheduling apparatus may perform operator analysis on the first model using a deep analysis tool.

[0158] For example, the aforementioned deep analysis tool may be torch.fx or torch.profiler.

[0159] For example, at runtime, torch.profiler is used to monitor the performance of the operators of the first model, including performance data such as execution time, memory usage, GPU / CPU resource utilization, and I / O bottlenecks, and to generate operator-level performance reports. Furthermore, to reduce analysis overhead and improve accuracy, an operator-level pruning method is introduced. Using execution time as the primary metric, the top 10 operators with the longest execution time in the model are selected. These operators are considered to have the most significant impact on model resource consumption. Their corresponding performance data is combined with global feature information to construct the input features of the first-level predictor.

[0160] In this way, the task scheduling device utilizes prediction technology for deep operator-level analysis. This technology incorporates the Torch framework's model parsing methods to achieve in-depth analysis of the model's structure and operating mechanisms at the operator level. By quantitatively analyzing the performance and parameter influence of each operator in the model, it identifies core parameters that have a critical impact on the model's operational efficiency and resource consumption, and uses these as input features for prediction. This approach effectively improves the accuracy and reliability of the prediction model while providing more targeted guidance for efficient resource allocation.

[0161] S102b. The task scheduling device inputs the first information into the first predictor, performs computing power resource prediction on the second task through the first predictor, and obtains the second computing power resource requirement information corresponding to the independent operation of the second task, so as to obtain the second computing power resource requirement information corresponding to the independent operation of each task.

[0162] Optionally, in an embodiment of the present application, the first predictor may include one or more sub-predictors.

[0163] Optionally, in an embodiment of the present application, the first predictor may be any one of the following: a random forest predictor, a decision tree predictor, or a gradient boosting tree predictor.

[0164] Preferably, the first predictor may be a random forest predictor.

[0165] Exemplarily, the task scheduling device can input the first information into each class tree in the random forest predictor, perform convolution prediction on the first information through each class tree, and obtain the prediction result obtained by each tree; then perform a majority vote on the prediction result of each class tree, and determine the prediction result of the tree with the most votes as the second computing resource demand information.

[0166] Optionally, in an embodiment of the present application, the first predictor may have a corresponding relationship with the second task.

[0167] For example, the above correspondence relationship may be preset by the task scheduling device; or the task scheduling device may determine the correspondence relationship in real time based on the model name corresponding to the second task. Specific correspondence relationships may be determined based on actual usage requirements and are not limited in this embodiment of the present application.

[0168] Optionally, in an embodiment of the present application, the task scheduling device may input the first information corresponding to each of the N tasks into the first predictor in sequence to obtain the second computing power resource requirement information corresponding to each of the N tasks when running independently.

[0169] Optionally, in an embodiment of the present application, the above-mentioned second computing resource demand information may include: GPU utilization, execution time, and memory requirements.

[0170] It should be noted that, for each of the N tasks, the specific process by which the task scheduling device obtains the second computing resource requirement information corresponding to each of the N tasks can be found in the above embodiment. To avoid repetition, it will not be repeated here.

[0171] For example, the second computing resource requirement information corresponding to each of the N tasks output by the first predictor can be expressed by the following formula (1):

[0172]

[0173] in, is the GPU utilization, is the execution time, is the memory requirement and N is the number of tasks.

[0174] S102c. The task scheduling device inputs the second computing power resources corresponding to each task in the N tasks when it is run independently and the global feature information of the model corresponding to each task in the N tasks into the second predictor, and uses the second predictor to predict the computing power resources for each task to obtain the first computing power resources corresponding to each task in the N tasks when it is run in parallel.

[0175] In an embodiment of the present application, the task scheduling device can splice the second computing power resources corresponding to each of the N tasks when running independently and the global feature information of the model corresponding to each of the N tasks, and input the spliced ​​information into the second predictor.

[0176] For example, the total model parameters of the model parameters in the global feature information of the model corresponding to each of the N tasks can be expressed by the following formula (2):

[0177]

[0178] Among them, P total is the total model parameter, P (i) is the model parameter of the model corresponding to a task, and N is the number of tasks.

[0179] As another example, the total number of floating-point operations in the global feature information of the model corresponding to each of the N tasks can be expressed by the following formula (3):

[0180]

[0181] Among them, FLOPs total Total floating point operations, FLOPs (i) is the number of floating-point operations of the model corresponding to a task, and N is the number of tasks.

[0182] Optionally, in an embodiment of the present application, the second predictor may be any one of the following: a random forest predictor, a decision tree predictor, or a gradient boosting tree predictor.

[0183] Preferably, the second predictor may be a random forest predictor.

[0184] It should be noted that the prediction process of the second predictor can be found in the above embodiment in detail, and will not be described again here to avoid repetition.

[0185] In an embodiment of the present application, the task scheduling device aims to accurately predict the resources required for task operation through a two-level predictor mechanism. This mechanism realizes a comprehensive analysis and prediction of the performance of a single task in the first-level predictor, including key indicators such as running time, GPU utilization, and memory usage. On this basis, the second-level predictor introduces the global feature data of the task, and through in-depth analysis of the performance of parallel tasks, quantifies the performance difference between parallel tasks and independent operation, and predicts the delay impact and required resources caused by parallel operation. This mechanism effectively improves the performance prediction capability and accuracy of complex task scenarios.

[0186] In some embodiments, combined Figure 1 ,like Figure 4 As shown, the "obtaining N scores corresponding to N computing power clusters" in the above S102 can be specifically implemented through the following S102d to S102f.

[0187] S102d. The task scheduling device calculates a first score corresponding to the graphics processor based on the memory size of the graphics processor, the number of graphics processors, and the memory distribution coefficient in the first computing power cluster.

[0188] In the embodiment of the present application, the first computing power cluster is any one of the N computing power clusters.

[0189] Optionally, in the embodiment of the present application, the above-mentioned video memory distribution coefficient may be preset by the task scheduling device or user-defined. The specific coefficient may be determined according to actual use requirements and is not limited by the embodiment of the present application.

[0190] It can be understood that the above-mentioned video memory distribution coefficient is used to characterize the uniformity of video memory among different GPUs. High concentration or high dispersion of video memory will affect the score.

[0191] Optionally, in an embodiment of the present application, the task scheduling device may calculate a third score corresponding to all graphics processors based on the video memory size of all graphics processors in the first computing power cluster, the number of all graphics processors, and the video memory distribution coefficients corresponding to all graphics processors.

[0192] S102e. The task scheduling device calculates a second score corresponding to the central processing unit based on the number of cores and the number of nodes of the central processing unit in the first computing power cluster.

[0193] Optionally, in an embodiment of the present application, the task scheduling device may calculate the second scores corresponding to all central processors based on the number of cores and the number of nodes of all central processors in the first computing power cluster.

[0194] S102f. The task scheduling device performs weighted summation on the first score and the second score based on the weight corresponding to the graphics processor and the weight corresponding to the central processing unit to obtain the score corresponding to the first computing power cluster, so as to obtain N scores corresponding to the N computing power clusters.

[0195] In the embodiment of the present application, the above weight is used to represent the importance of the processor to the score.

[0196] Optionally, in the embodiment of the present application, the weight corresponding to the graphics processor may be preset by the task scheduling device or user-defined. The specific weight may be determined based on actual usage requirements and is not limited by the embodiment of the present application.

[0197] Optionally, in the embodiment of the present application, the weight corresponding to the central processing unit may be preset by the task scheduling device or user-defined. The specific weight may be determined according to actual use requirements and is not limited by the embodiment of the present application.

[0198] Exemplarily, the task scheduling device may obtain the score corresponding to the first computing power cluster through the following formula (4).

[0199]

[0200] Among them, FS Cluster is the score corresponding to the first computing power cluster, w GPU .FS GPU For the first rating, w CPU .FS CPU is the second score, and M is the number of CPUs or GPUs.

[0201] It should be noted that for the score corresponding to each of the N computing power clusters, the task scheduling device can obtain the score corresponding to each computing power cluster through the above formula (4). To avoid repetition, it will not be repeated here.

[0202] Optionally, in an embodiment of the present application, for the third scores corresponding to the K nodes in each computing power cluster, the task scheduling device can obtain the score corresponding to each of the K nodes through the above formula (4). To avoid repetition, it will not be repeated here.

[0203] In an embodiment of the present application, the task scheduling device monitors the resource fragmentation status of the cluster in real time, and allocates low-fragmentation nodes to high-demand tasks and high-fragmentation nodes to low-demand tasks, thereby filling fragmented resources and improving overall resource utilization.

[0204] In some embodiments, combined Figure 4 ,like Figure 5 As shown, the above S102d can be specifically implemented through the following S201 to S203.

[0205] S201: The task scheduling apparatus divides the idle video memory size of the graphics processor by the total video memory size to obtain a first value.

[0206] S202: The task scheduling apparatus divides the number of idle graphics processors of the graphics processor by the total number of graphics processors to obtain a second value.

[0207] S203: The task scheduling device multiplies the first value, the second value, and the video memory distribution coefficient to obtain a first score.

[0208] Exemplarily, the specific process of obtaining the first score can be implemented by the following formula (5).

[0209]

[0210] Among them, FS GPU Rated first.

[0211] In the embodiment of the present application, the idle resource information and video memory distribution coefficient of the graphics processor can be used to accurately quantify the resource fragmentation degree of the graphics processor in the computing power cluster, thereby improving the accuracy of the task scheduling device in determining the resource fragmentation degree of the graphics processor.

[0212] In some embodiments, combined Figure 4 ,like Figure 6 As shown, the above S102f can be specifically implemented through the following S301 to S303.

[0213] S301 : The task scheduling apparatus divides the number of idle cores of a central processing unit by the total number of cores to obtain a third value.

[0214] S302: The task scheduling device divides the number of idle nodes of the central processing unit by the total number of nodes to obtain a fourth value.

[0215] S303: The task scheduling device multiplies the third value and the fourth value to obtain a second score.

[0216] Exemplarily, the specific process of obtaining the second score can be implemented by the following formula (6).

[0217]

[0218] Among them, FS CPU Rated second.

[0219] In the embodiment of the present application, the idle resource information of the central processing unit can be used to accurately quantify the resource fragmentation degree of the central processing unit in the computing power cluster, thereby improving the accuracy of the task scheduling device in determining the resource fragmentation degree of the central processing unit.

[0220] In some embodiments, the present application provides a task scheduling framework. The task scheduling architecture provided in the embodiments of the present application is explained in detail below.

[0221] like Figure 7 As shown, the task scheduling framework 10 includes: a wide-area scheduler module 11 and a cross-domain heterogeneous cluster module 12 , and the wide-area scheduler module 11 is connected to the cross-domain heterogeneous cluster module 12 .

[0222] In the embodiment of the present application, the wide-area scheduler module 11 is used to determine the cluster corresponding to each of the N tasks, and the cross-domain heterogeneous cluster module 12 is used to schedule each of the N tasks to the corresponding cluster.

[0223] In some embodiments, combined Figure 7 ,like Figure 8 As shown, the wide-area scheduler module 11 includes: a multi-level predictor module 13, an information collection module 14 and a scheduling module 15. The multi-level predictor module 13 and the information collection module 14 are respectively connected to the scheduling module 15, and the scheduling module 15 is connected to the cross-domain heterogeneous cluster module 12.

[0224] In an embodiment of the present application, the above-mentioned multi-level predictor module 13 is used to obtain the first computing power resource demand information corresponding to each task in N tasks, the above-mentioned information collection module 14 is used to obtain the score corresponding to each computing power cluster in N computing power clusters, and the above-mentioned scheduling module 15 is used to determine the corresponding computing power cluster for each task based on the first computing power resource demand information corresponding to each task in N tasks when running in parallel and the score corresponding to each computing power cluster in N computing power clusters.

[0225] In the embodiment of the present application, the scheduling decision strategy is the core logic of the scheduling module, and its main goal is to formulate the optimal task allocation plan based on the task requirements and cluster status. Each task determines the resource requirements based on the results of the multi-level predictor, and then selects the target cluster in combination with the score of the fragmentation perception module. For example, for tasks with high computing power requirements, priority is given to nodes in the cluster with the lowest fragmentation score to ensure the integrity of resources; for tasks with low computing power requirements, priority is given to nodes in the cluster with a higher fragmentation score to fill the fragmented resources. In addition, the scheduling module supports dynamic adjustment strategies, regularly checks the status of running tasks, and migrates low-demand tasks to other nodes to free up resources required for high-demand tasks. When migrating tasks, network bandwidth and data transmission delay are taken into account to ensure that the migration cost is minimized. Through this dynamic optimization mechanism, the system can adapt to real-time load changes and continuously improve the overall task scheduling efficiency.

[0226] In some embodiments, such as Figure 9 As shown, the multi-level predictor module 13 includes: a primary predictor 16 , an inference database 17 and a secondary predictor 18 . The primary predictor 16 and the inference database 17 are respectively connected to the secondary predictor 18 , and the secondary predictor 18 is connected to the scheduling module 15 .

[0227] In an embodiment of the present application, the first-level predictor 16 is used to predict the second computing power resource demand information corresponding to each of the N tasks, and the second-level predictor 18 is used to predict the first computing power resource demand information corresponding to each of the N tasks based on the second computing power resource demand information corresponding to each of the N tasks and the global feature information of all models stored in the inference database 17.

[0228] Optionally, in the embodiment of the present application, the first-level predictor 16 may be one or more.

[0229] It should be noted that there is only one secondary predictor 18 .

[0230] In the embodiments of the present application, a comprehensive analysis and prediction of the performance of a single task is implemented in the first-level predictor, including key indicators such as runtime, GPU utilization, and memory usage. On this basis, the second-level predictor introduces global feature data of the task, conducts in-depth analysis of the performance of parallel tasks, quantifies the performance difference between parallel tasks and independent execution, and predicts the delay impact and required resources caused by parallel execution. This mechanism effectively improves the performance prediction capability and accuracy of complex task scenarios.

[0231] In some embodiments, combined Figure 8 ,like Figure 10As shown, the above-mentioned cross-domain heterogeneous cluster module 12 includes: a task cache queue 18 and N computing power clusters 19 , the task cache queue 18 is connected to the N computing power clusters 19 , and the task cache queue 18 is connected to the scheduling module 15 .

[0232] In the embodiment of the present application, the task cache queue 18 is used to determine the scheduling order of tasks, and the N computing power clusters 19 are used to determine the computing power cluster corresponding to each of the N tasks.

[0233] In this embodiment of the present application, the scheduling module can set up a cache queue to store submitted tasks. The scheduling module collects computing power information and network status from each cluster in real time, and combines a multi-level predictor to perform fine-grained analysis of task requirements and plan each cluster's task queue. The scheduler regularly updates the task allocation plan based on the cluster's dynamic state. This architecture effectively coordinates the matching of tasks and resources, improving overall cluster utilization and task processing efficiency.

[0234] The embodiment of the present application provides a task scheduling framework. Since the corresponding scores of the computing power clusters obtained can represent the degree of resource fragmentation of the computing power clusters, the severity of the resource fragmentation of the computing power clusters can be determined through the scores corresponding to each computing power cluster, so that each task can be scheduled to the corresponding computing power cluster according to the computing power resource demand information of each task; that is, by sensing resource fragmentation, each task is scheduled to the corresponding computing power cluster according to the computing power resource demand information of each task. For example, tasks with low computing power resource requirements are scheduled to computing power clusters with severe resource fragmentation, and tasks with high computing power resource requirements are scheduled to computing power clusters with slight resource fragmentation; in this way, the fragmented resource integration method is used to fully tap the system potential, reduce resource waste, and improve the resource utilization rate of the computing power cluster.

[0235] like Figure 11 As shown, the task scheduling method provided in the embodiment of the present application is specifically explained below through specific examples.

[0236] Exemplarily, the task scheduling device receives N tasks, inputs the N tasks into the multi-level predictor module in the wide-area scheduler module to obtain the first computing power resource requirement information corresponding to each of the N tasks during parallel operation, obtains the score corresponding to each of the N clusters through the information collection module in the wide-area scheduler module 11, and inputs the score and the first computing power resource requirement information corresponding to each of the N tasks during parallel operation into the scheduling module in the wide-area scheduler module to determine the computing power cluster corresponding to each of the N tasks.

[0237] Then, the first computing resource demand information corresponding to each of the N tasks and the computing cluster information corresponding to each of the N tasks are input into the task cache queue in the cross-domain heterogeneous cluster module 12. The cross-domain heterogeneous cluster module schedules each of the N tasks to the computing node in the corresponding computing cluster in sequence according to the task order in the task cache queue.

[0238] like Figure 12 As shown, the following is a specific example to explain in detail the process of obtaining the first computing power resource demand information corresponding to each of N tasks through a multi-level predictor module provided in an embodiment of the present application.

[0239] For example, after receiving N tasks, the task scheduling device obtains first information corresponding to each of the N tasks. Figure 12 In the example, “model name, input model size, parameters and Flops” are used. The task scheduling device sequentially inputs the first information corresponding to each task into multiple first-level predictors 16 to obtain the GPU utilization, runtime and memory size corresponding to each task. The GPU utilization, runtime and memory size corresponding to each task are combined with the Flops and model parameters of all models stored in the inference database 17. Figure 12 Parameters are used as input to the secondary predictor to predict the first computing resource requirements when each task runs in parallel.

[0240] The various solutions in the above embodiments of the present application can be combined under the premise that there is no contradiction.

[0241] In the embodiment of the present application, the task scheduling device can be divided into functional modules or functional units according to the above method example. For example, each functional module or functional unit can be divided according to each function, or two or more functions can be integrated into one processing module. The above-mentioned integrated module can be implemented in the form of hardware or in the form of software functional modules or functional units. Among them, the division of modules or units in the embodiment of the present application is schematic and is only a logical functional division. There may be other division methods in actual implementation.

[0242] In the case of dividing each functional module into corresponding functional modules, Figure 13 FIG2 shows a schematic structural diagram of a task scheduling device 70 , which can be used to execute the functions of the electronic device involved in the above embodiments. Figure 13 The task scheduling device 70 shown may include: a receiving unit 71 , an acquiring unit 72 and a scheduling unit 73 .

[0243] The receiving unit 71 is used to receive N tasks, where N is a positive integer.

[0244] The acquisition unit 72 is used to obtain the first computing power resource requirement information corresponding to each task received by the receiving unit 71, and to obtain N scores corresponding to N computing power clusters, where one score corresponds to one computing power cluster. The score is used to characterize the degree of resource fragmentation of the computing power cluster, and the computing power resource requirement information is used to indicate the degree of computing power resource requirement of the task.

[0245] The scheduling unit 73 is configured to schedule each task to a corresponding computing power cluster based on the first computing power resource requirement information and the N scores corresponding to each task acquired by the acquisition unit 72 .

[0246] In one possible implementation, the scheduling unit 73 is specifically used to, when the first task is a high-demand task, assign the first task to a computing power cluster that meets a first condition. The high-demand task is a task whose computing power resource demand information is greater than or equal to a preset threshold. The first condition includes: the first computing power resource demand information of the first task is met in N computing power clusters and the score is the lowest among the N scores; or, when the first task is a low-demand task, assign the first task to a computing power cluster that meets a second condition. The low-demand task is a task whose computing power resource demand information is less than a preset threshold. The second condition includes: the first computing power resource demand information of the first task is met in N computing power clusters and the score is the highest among the N scores; wherein the first task is any one of the N tasks.

[0247] In one possible implementation, the acquisition unit 72 is specifically used to obtain the first information corresponding to the second task, where the first information includes: task parameter information, global feature information of the first model corresponding to the second task, and operator indicator parameter information of the first model, where the second task is any one of the N tasks; and input the first information into the first predictor, perform computing power resource prediction on the second task through the first predictor, and obtain the second computing power resource requirement information corresponding to the independent operation of the second task, so as to obtain the second computing power resource requirement information corresponding to the independent operation of each task; and input the second computing power resource requirement information corresponding to the independent operation of each task and the global feature information of the model corresponding to each task into the second predictor, perform computing power resource prediction on each task through the second predictor, and obtain the first computing power resource requirement information corresponding to the parallel operation of each task.

[0248] In one possible implementation, the acquisition unit 72 is specifically used to calculate a first score corresponding to the graphics processor based on the video memory size of the graphics processor, the number of graphics processors, and the video memory distribution coefficient in the first computing power cluster, where the first computing power cluster is any one of the N computing power clusters; and to calculate a second score corresponding to the central processing unit based on the number of cores and the number of nodes of the central processing unit in the first computing power cluster; and to perform a weighted sum of the first score and the second score based on the weight corresponding to the graphics processor and the weight corresponding to the central processing unit to obtain a score corresponding to the first computing power cluster, so as to obtain N scores corresponding to the N computing power clusters.

[0249] In one possible implementation, the acquisition unit 72 is specifically configured to divide the free video memory size of the graphics processor by the total video memory size to obtain a first value; divide the number of free graphics processors of the graphics processor by the total number of graphics processors to obtain a second value; and multiply the first value, the second value, and the video memory distribution coefficient to obtain a first score.

[0250] In one possible implementation, the acquisition unit 72 is specifically used to divide the number of idle cores of the central processing unit by the total number of cores to obtain a third value; divide the number of idle nodes of the central processing unit by the total number of nodes to obtain a fourth value; and multiply the third value and the fourth value to obtain a second score.

[0251] The embodiment of the present application provides a task scheduling device. Since the corresponding score of the computing power cluster obtained can represent the degree of resource fragmentation of the computing power cluster, the severity of the resource fragmentation of the computing power cluster can be determined through the score corresponding to each computing power cluster. The computing power resource demand information is used to indicate the degree of computing power resource demand of the task, so that each task can be scheduled to the corresponding computing power cluster according to the computing power resource demand information of each task; that is, by sensing resource fragmentation, each task is scheduled to the corresponding computing power cluster according to the computing power resource demand information of each task. For example, tasks with low computing power resource demand are scheduled to computing power clusters with severe resource fragmentation, and tasks with high computing power resource demand are scheduled to computing power clusters with slight resource fragmentation; in this way, the fragmented resource integration method is used to fully tap the system potential, reduce resource waste, and improve the resource utilization rate of the computing power cluster.

[0252] As another possible implementation method, Figure 13 The acquisition unit 72 and the scheduling unit 73 in the embodiment can be replaced by a processor, which can integrate the functions of the acquisition unit 72 and the scheduling unit 73. Figure 13 The receiving unit 71 in the embodiment may be replaced by a transceiver or a transceiver unit, which may integrate the functions of the receiving unit 71.

[0253] The embodiment of the present application also provides a computer-readable storage medium. All or part of the processes in the above-mentioned method embodiments can be completed by a computer program to instruct the relevant hardware, and the program can be stored in the above-mentioned computer-readable storage medium. When the program is executed, it can include the processes of the above-mentioned method embodiments. The computer-readable storage medium can be the internal storage unit of the task scheduling device of any of the above-mentioned embodiments, such as the hard disk or memory of the task scheduling device. The above-mentioned computer-readable storage medium can also be an external storage device of the above-mentioned terminal device, such as a plug-in hard disk, a smart memory card (smartmedia card, SMC), a secure digital (secure digital, SD) card, a flash card (flashcard), etc. equipped on the above-mentioned terminal device. Furthermore, the above-mentioned computer-readable storage medium can also include both the internal storage unit of the above-mentioned task scheduling device and an external storage device. The above-mentioned computer-readable storage medium is used to store the above-mentioned computer program and other programs and data required by the above-mentioned task scheduling device. The above-mentioned computer-readable storage medium can also be used to temporarily store data that has been output or is to be output.

[0254] It should be noted that the terms "first" and "second" in the specification, claims, and drawings of this application are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units that are not listed, or may optionally include other steps or units that are inherent to these processes, methods, products, or devices.

[0255] It should be understood that in the present application, "at least one (item)" refers to one or more, "more than one" refers to two or more, "at least two (items)" refers to two or three and more than three, and "and / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0256] Through the description of the above implementation methods, technical personnel in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0257] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the modules or units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0258] The units described as separate components may or may not be physically separate, and the components shown as units may be one physical unit or multiple physical units, that is, they may be located in one place or distributed in multiple places. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0259] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0260] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling a device (which can be a single-chip microcomputer, chip, etc.) or a processor to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a ROM, a RAM, a magnetic disk, or an optical disk.

[0261] The above is only a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or replacements within the technical scope disclosed in this application should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A task scheduling method, characterized in that: The method comprises: Receive N tasks, where N is a positive integer; Obtain first computing resource requirement information corresponding to each task, and obtain N scores corresponding to N computing clusters, one score corresponding to each computing cluster. The score is used to characterize the degree of resource fragmentation of the computing cluster, and the computing resource requirement information is used to indicate the degree of computing resource requirement of the task; Based on the first computing resource requirement information corresponding to each task and the N scores, each task is scheduled to a corresponding computing cluster.

2. The task scheduling method according to claim 1, characterized in that: The step of scheduling each task to a corresponding computing cluster based on the first computing resource requirement information corresponding to each task and the N scores includes: If the first task is a high-demand task, the first task is assigned to a computing power cluster that meets a first condition, where the high-demand task is a task whose computing power resource requirement information is greater than or equal to a preset threshold, and the first condition includes: the N computing power clusters meet the first computing power resource requirement information of the first task and have the lowest score among the N scores; or If the first task is a low-demand task, assign the first task to a computing power cluster that meets a second condition, where the low-demand task is a task whose computing power resource requirement information is less than the preset threshold, and the second condition includes: the N computing power clusters meet the first computing power resource requirement information of the first task and have the highest score among the N scores; The first task is any one of the N tasks.

3. The task scheduling method according to claim 1, wherein: The obtaining of the first computing resource requirement information corresponding to each task includes: Acquire first information corresponding to a second task, the first information including: task parameter information, global feature information of a first model corresponding to the second task, and operator indicator parameter information of the first model, where the second task is any one of the N tasks; Inputting the first information into a first predictor, performing computing resource prediction on the second task using the first predictor, and obtaining second computing resource requirement information corresponding to independently running the second task, thereby obtaining second computing resource requirement information corresponding to independently running each of the tasks; The second computing power resource requirement information corresponding to the independent operation of each task and the global feature information of the model corresponding to each task are input into the second predictor, and the computing power resources of each task are predicted by the second predictor to obtain the first computing power resource requirement information corresponding to the parallel operation of each task.

4. The task scheduling method according to claim 1, wherein: Obtaining N scores corresponding to N computing power clusters includes: Calculate a first score corresponding to a graphics processor based on a memory size of the graphics processor in a first computing power cluster, the number of the graphics processors, and a memory distribution coefficient, where the first computing power cluster is any one of the N computing power clusters; Calculate a second score corresponding to the central processing unit based on the number of cores and the number of nodes of the central processing unit in the first computing power cluster; Based on the weight corresponding to the graphics processor and the weight corresponding to the central processing unit, the first score and the second score are weightedly summed to obtain the score corresponding to the first computing power cluster, so as to obtain the N scores corresponding to the N computing power clusters, where the weight is used to represent the importance of the processor to the score.

5. The task scheduling method according to claim 4, characterized in that: The calculating, based on the memory size of the graphics processors in the first computing power cluster, the number of the graphics processors, and the memory distribution coefficient, to obtain a first score corresponding to the graphics processors includes: Performing a division operation on the free video memory size of the graphics processor and the total video memory size to obtain a first value; Performing a division operation on the number of idle graphics processors of the graphics processor and the total number of graphics processors to obtain a second value; The first value, the second value, and the video memory distribution coefficient are multiplied to obtain the first score.

6. The task scheduling method according to claim 4, characterized in that: The calculating, based on the number of cores and the number of nodes of the central processing unit in the first computing power cluster, a second score corresponding to the central processing unit includes: performing a division operation on the number of idle cores and the total number of cores of the central processing unit to obtain a third value; Dividing the number of idle nodes of the central processing unit by the total number of nodes to obtain a fourth value; The third value and the fourth value are multiplied to obtain the second score.

7. A task scheduling device, characterized in that: The task scheduling device includes: a receiving unit, an acquiring unit and a scheduling unit; The receiving unit is used to receive N tasks, where N is a positive integer; The acquiring unit is configured to acquire first computing resource requirement information corresponding to each task received by the receiving unit, and acquire N scores corresponding to N computing clusters, one score corresponding to one computing cluster, the score being used to characterize the degree of resource fragmentation of the computing cluster, and the computing resource requirement information being used to indicate the degree of computing resource requirement of the task; The scheduling unit is configured to schedule each task to a corresponding computing power cluster based on the first computing power resource requirement information corresponding to each task and the N scores acquired by the acquisition unit.

8. The task scheduling device according to claim 7, characterized in that: The scheduling unit is specifically configured to, when the first task is a high-demand task, assign the first task to a computing power cluster that meets a first condition, wherein the high-demand task is a task whose computing power resource requirement information is greater than or equal to a preset threshold, and the first condition includes: the first computing power resource requirement information of the first task is met among the N computing power clusters and the score is the lowest among the N scores; or If the first task is a low-demand task, assign the first task to a computing power cluster that meets a second condition, where the low-demand task is a task whose computing power resource requirement information is less than the preset threshold, and the second condition includes: the N computing power clusters meet the first computing power resource requirement information of the first task and have the highest score among the N scores; The first task is any one of the N tasks.

9. The task scheduling device according to claim 7, characterized in that: The acquiring unit is specifically configured to acquire first information corresponding to a second task, the first information including task parameter information, global feature information of a first model corresponding to the second task, and operator indicator parameter information of the first model, where the second task is any one of the N tasks; Inputting the first information into a first predictor, performing computing resource prediction on the second task using the first predictor, and obtaining second computing resource requirement information corresponding to independently running the second task, thereby obtaining second computing resource requirement information corresponding to independently running each of the tasks; The second computing power resource requirement information corresponding to each task when it is run independently and the global feature information of the model corresponding to each task are input into the second predictor, and computing power resources are predicted for each task through the second predictor to obtain the first computing power resource requirement information corresponding to each task when it is run in parallel.

10. The task scheduling device according to claim 7, characterized in that: The acquiring unit is specifically configured to calculate a first score corresponding to a graphics processor based on a video memory size of the graphics processor in a first computing power cluster, the number of the graphics processors, and a video memory distribution coefficient, where the first computing power cluster is any one of the N computing power clusters; Calculate a second score corresponding to the central processing unit based on the number of cores and the number of nodes of the central processing unit in the first computing power cluster; Based on the weight corresponding to the graphics processor and the weight corresponding to the central processing unit, the first score and the second score are weightedly summed to obtain the score corresponding to the first computing power cluster, so as to obtain the N scores corresponding to the N computing power clusters, where the weight is used to represent the importance of the processor to the score.

11. The task scheduling device according to claim 10, characterized in that: The acquiring unit is specifically configured to perform a division operation on the free video memory size and the total video memory size of the graphics processor to obtain a first value; Performing a division operation on the number of idle graphics processors of the graphics processor and the total number of graphics processors to obtain a second value; The first value, the second value, and the video memory distribution coefficient are multiplied to obtain the first score.

12. An electronic device, characterized in that: The method comprises a processor, a memory, and a program or instruction stored in the memory and executable on the processor, wherein the program or instruction, when executed by the processor, implements the steps of the task scheduling method according to any one of claims 1 to 6.

Images