Job scheduling method and related device

Abstract

The invention discloses a job scheduling method and a related device, and relates to the field of cloud computing, and the method comprises the steps: obtaining a resource demand in a job opening request in response to the job opening request submitted by a front-end service platform, and screening candidate resource nodes meeting the resource demand from available resource nodes, acquiring a historical resource utilization rate index value and a historical demand matching index value of the candidate resource node, performing weighted summation on the historical resource utilization rate index value and the historical demand matching index value of the candidate resource node, and calculating a comprehensive index score of the candidate resource node, and determining the candidate resource with the highest comprehensive index score as the target resource node, thereby generating a scheduling scheme, and improving the utilization rate of platform resources.

Description

Technical Field

[0001] The present application relates to the field of cloud computing technology, and in particular to a job scheduling method and related devices. Background Art

[0002] With the development of cloud computing technology, cloud computing services are becoming more and more mature. Platform providers such as large model training platforms, video rendering farms, and supercomputing task platforms need to call the IaaS (Infrastructure as a Service) cloud host and physical machine resources of cloud service providers at the bottom layer, and provide PaaS (Platform as a Service) / SaaS (Software as a Service) platform services to end customers at the upper layer for use by upper-layer platform operations.

[0003] Currently, in job scenarios, the job scheduling method of platform providers mainly considers resource availability and resource costs to select the optimal resources. If resources are insufficient, new resources will be added. The scientific nature of the overall resource utilization and resource distribution of the platform are not considered, and there is no tracking of whether the actual operation of resources after activation matches the resource activation specifications. Summary of the invention

[0004] In view of the above problems, the present application provides a job scheduling method and related devices, which match resources according to the actual operation of the job and keep the global resource utilization rate within a scientific range. The specific scheme is as follows:

[0005] The first aspect of the present application provides a job scheduling method, comprising:

[0006] In response to a job opening request submitted by a front-end business platform, obtaining resource requirements in the job opening request;

[0007] Get resource information of available resource nodes;

[0008] According to the resource information of the available resource nodes, selecting candidate resource nodes that meet the resource requirement from the available resource nodes;

[0009] Acquire a historical resource utilization index value and a historical demand matching index value of the candidate resource node, wherein the historical demand matching index value represents the difference between resources applied for by all jobs of the candidate resource node within a preset historical period and a resource utilization peak value;

[0010] Performing a weighted summation on the historical resource utilization index value and the historical demand matching index value to calculate the comprehensive index score of the candidate resource node;

[0011] The candidate resource node with the highest comprehensive indicator score is determined as the target resource node, and a first scheduling scheme is generated.

[0012] In a possible implementation, obtaining the historical resource utilization index value of the candidate resource node includes:

[0013] Obtaining CPU utilization, GPU utilization, and memory utilization of the candidate resource node within a preset historical period;

[0014] The CPU utilization, GPU utilization and memory utilization of the candidate resource node in a preset historical period are weightedly summed to obtain the historical resource utilization index value.

[0015] In a possible implementation, obtaining the historical demand matching index value of the candidate resource node includes:

[0016] Obtain the total amount of CPUs applied for by all jobs, the total amount of GPUs applied for by all jobs, and the total amount of memory applied for by all jobs of the candidate resource node within a preset historical period;

[0017] Obtaining the 95th peak value of the CPU, the 95th peak value of the GPU, and the 95th peak value of the memory of the candidate resource node within a preset historical period;

[0018] Calculate respectively a first difference between the total CPU amount applied by all jobs of the candidate resource node in a preset historical period and the 95% peak value of the CPU, a second difference between the total GPU amount applied by all jobs and the 95% peak value of the GPU, and a third difference between the total memory amount applied by all jobs and the 95% peak value of the memory;

[0019] The first difference, the second difference, and the third difference corresponding to the candidate resource node are weightedly summed to obtain the historical demand matching index value.

[0020] In a possible implementation, if there is no candidate resource node that meets the resource requirement among the available resource nodes, the job scheduling method further includes:

[0021] Determine the resource pool with the highest average value of the comprehensive index score as the target resource pool;

[0022] Determine a resource type and resource specification that meet the resource demand, and generate a second scheduling solution according to the target resource pool, the resource type and the resource specification;

[0023] Sending a resource activation instruction to a resource platform, wherein the resource activation instruction carries the target resource pool, the resource type, and the resource specification;

[0024] After the resource activation is completed, the activated resource node is bound to the second scheduling scheme.

[0025] In a possible implementation, the job scheduling method further includes:

[0026] When the last job running in the resource node is completed, a resource release operation is performed on the resource node.

[0027] A second aspect of the present application provides a job scheduling device, comprising:

[0028] A resource requirement acquisition unit, configured to respond to a job opening request submitted by a front-end business platform and acquire resource requirements in the job opening request;

[0029] A resource information acquisition unit, used to acquire resource information of available resource nodes;

[0030] A resource node screening unit, configured to screen candidate resource nodes that meet the resource requirement from the available resource nodes according to the resource information of the available resource nodes;

[0031] An index acquisition unit, used to acquire a historical resource utilization index value and a historical demand matching index value of the candidate resource node, wherein the historical demand matching index value represents the difference between the resources applied for by all jobs of the candidate resource node within a preset historical period and the resource utilization peak value;

[0032] An index calculation unit, used to perform weighted summation of the historical resource utilization index value and the historical demand matching index value to calculate the comprehensive index score of the candidate resource node;

[0033] The scheduling scheme generating unit is used to determine the candidate resource node with the highest comprehensive index score as the target resource node and generate a first scheduling scheme.

[0034] The third aspect of the present application provides a job scheduling system, including: a job scheduler, a resource controller, an indicator collector, a first API gateway, a message queue and a database;

[0035] The first API gateway is used to receive a job activation request submitted by a front-end business platform, send a request index of the job activation request to the message queue, and store the job activation request in the database;

[0036] The job scheduler is used to extract the job opening request from the message queue and the database, obtain the resource information of the available resource nodes from the resource controller, screen the candidate resource nodes that meet the resource requirements from the available resource nodes based on the resource information of the available resource nodes, obtain the comprehensive index score of the candidate resource node from the index collector, determine the candidate resource node with the highest comprehensive index score as the target resource node, and generate a first scheduling plan, wherein the comprehensive index score is calculated based on the historical resource utilization index value and the historical demand matching index value of the candidate resource node, and the historical demand matching index value represents the difference between the resources applied for by all jobs of the candidate resource node within a preset historical period and the resource utilization peak value.

[0037] In a possible implementation, the job scheduling system further includes: a second API gateway;

[0038] The job scheduler is also used to obtain the comprehensive index score of each resource node in each resource pool from the index collector when there is no candidate resource node that meets the resource demand among the available resource nodes, determine the resource pool with the highest average value of the comprehensive index score as the target resource pool, determine the resource type and resource specification that meet the resource demand, generate a second scheduling plan according to the target resource pool, the resource type and the resource specification, and send a resource activation request to the resource controller;

[0039] The resource controller calls the second API gateway to send a resource activation instruction to the resource platform based on the target resource pool, the resource type and the resource specifications in the resource activation request. The resource activation instruction carries the target resource pool, the resource type and the resource specifications. After the resource activation is completed, the activated resource node is bound to the second scheduling scheme.

[0040] The fourth aspect of the present application provides a computer program product, including computer-readable instructions. When the computer-readable instructions are executed on an electronic device, the electronic device implements the job scheduling method of the above-mentioned first aspect or any implementation method of the first aspect.

[0041] A fifth aspect of the present application provides a computer storage medium, which carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement the job scheduling method of the above-mentioned first aspect or any implementation of the first aspect.

[0042] By means of the above technical scheme, the present application provides a method and related device for scheduling a job, which responds to a job opening request submitted by a front-end business platform, obtains resource requirements in the job opening request, selects candidate resource nodes that meet the resource requirements from available resource nodes, obtains historical resource utilization index values ​​and historical demand matching index values ​​of the candidate resource nodes, performs weighted summation of the historical resource utilization index values ​​and historical demand matching index values ​​of the candidate resource nodes, calculates the comprehensive index score of the candidate resource nodes, and determines the candidate resource with the highest comprehensive index score as the target resource node, thereby generating a scheduling scheme. The present application determines whether to determine the candidate resource node as the target resource node based on the historical resource utilization index value and the historical demand matching index value of the candidate resource node, analyzes the resource utilization rate of the candidate resource node within a preset period and the difference between the resources applied for all jobs and the actual resource utilization peak value, that is, the resource utilization rate and the matching rationality of the underlying resources and the upper-level requirements are used as the judgment basis to determine the scheduling method of the upper-level job, and the resources are matched according to the actual operation of the job, thereby improving the resource utilization rate of the platform. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and the originals and elements are not necessarily drawn to scale.

[0044] Figure 1 A flowchart of a job scheduling method provided in an embodiment of the present application;

[0045] Figure 2 A schematic diagram of a system architecture provided for an embodiment of the present application;

[0046] Figure 3 A schematic diagram of the structure of a job scheduling device provided in an embodiment of the present application;

[0047] Figure 4 Another system architecture diagram provided for an embodiment of the present application. DETAILED DESCRIPTION

[0048] The following describes the embodiments of the present application in conjunction with the drawings in the embodiments of the present application. The terms used in the implementation method section of the present application are only used to explain the specific embodiments of the present application, and are not intended to limit the present application.

[0049] The embodiments of the present application are described below in conjunction with the accompanying drawings. Those skilled in the art will appreciate that, with the development of technology and the emergence of new scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0050] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and need not be used to describe a specific order or sequential order. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, which is only to describe the distinction mode adopted by the objects of the same attributes when describing in the embodiments of the present application. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, so that the process, method, system, product or equipment comprising a series of units need not be limited to those units, but may include other units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0051] The embodiment of the present application provides a method for scheduling a job. The method for scheduling a job in the embodiment of the present application is described in detail below in conjunction with the accompanying drawings.

[0052] Reference Figure 1 , Figure 1 A flowchart of a job scheduling method provided in an embodiment of the present application is shown as follows: Figure 1 As shown, a job scheduling method provided in an embodiment of the present application is applied to Figure 2 The job scheduling system shown in Figure 2 The upstream of the job scheduling system is the front-end business platform, and the downstream of the job scheduling system is the resource platform. The resource platform includes multiple resource pools, and the performance of different resource pools may be different. The job scheduling method may include steps 101 to 106, and these steps are described in detail below.

[0053] 101: In response to a job opening request submitted by a front-end business platform, obtaining resource requirements in the job opening request;

[0054] Users submit job activation requests through the front-end business platform to call resources in the resource platform.

[0055] The types of jobs are different in different application scenarios. For example, in a video rendering application scenario, the job is a video rendering job, and in a large model training application scenario, the job is a large model training job.

[0056] The job opening request includes resource requirements, which include at least one of CPU requirements, GPU requirements, and memory requirements. Furthermore, the job opening request may also include a job scenario, which may be video rendering, large model training, and the like.

[0057] 102: Get resource information of available resource nodes;

[0058] Exemplarily, resource information of all currently available resource nodes is obtained from the resource controller.

[0059] The available resource nodes are resource nodes that have been activated. The resource information of the available resource nodes includes information such as remaining resources. The remaining resources include the remaining CPU size, the remaining GPU size, and the remaining memory size.

[0060] 103: screening candidate resource nodes that meet the resource requirement from the available resource nodes according to the resource information of the available resource nodes;

[0061] Exemplarily, compare the remaining CPU size in the resource information of the available resource node with the CPU requirement in the resource demand, the remaining GPU size in the resource information of the available resource node with the GPU requirement in the resource demand, and the remaining memory size in the resource information of the available resource node with the memory requirement in the resource demand, and filter candidate resource nodes that meet the resource demand from the available resource nodes.

[0062] The remaining resources of the candidate resource node are not less than the resource requirement,

[0063] 104: Obtain historical resource utilization index values ​​and historical demand matching index values ​​of the candidate resource nodes;

[0064] The historical resource utilization index of the candidate resource node indicates whether the candidate resource node is fully utilized in a preset historical period.

[0065] The preset historical period is pre-set, such as the past week.

[0066] Exemplarily, a method for obtaining the historical resource utilization index value of the candidate resource node is:

[0067] The CPU utilization, GPU utilization, and memory utilization of the candidate resource node in a preset historical period are obtained, and the CPU utilization, GPU utilization, and memory utilization of the candidate resource node in the preset historical period are weighted summed to obtain a historical resource utilization index value.

[0068] The utilization rate can be the average utilization rate, the 95 peak utilization rate or the median utilization rate. The average utilization rate is used when the upper-level operation is relatively simple and the differences between the operations are not large. The 95 peak utilization rate is used when the resource usage is unstable and the peaks and troughs are large. The median utilization rate is used when the operation scenarios are very different and the characteristics are dispersed. It should be noted that once one of the above utilization rates is selected, each candidate resource node needs to calculate the historical resource utilization index value according to the utilization rate.

[0069] Exemplarily, the weights of CPU utilization, GPU utilization, and memory utilization represent cost coefficients of the CPU, GPU, and memory. For example, the weights of CPU utilization, GPU utilization, and memory utilization may be 0.3, 0.5, and 0.2, respectively.

[0070] Taking the preset historical period of the past seven days and the utilization as the average utilization as an example, the historical demand matching index value of the candidate resource node = (0.3×seven-day CPU average utilization + 0.5×seven-day GPU average utilization + 0.2×seven-day memory utilization) / (0.3+0.5+0.2).

[0071] The historical demand matching index value indicates the difference between the resources requested by all jobs in the candidate resource node within the preset historical period and the resource utilization peak value, that is, whether the resource specifications requested by the job meet the actual resource requirements. If the historical demand matching index value is too large, it is considered that the job has too many resources, and the resource node can actually carry more jobs.

[0072] Exemplarily, a method for obtaining a historical demand matching index value of a candidate resource node includes the following steps A1-A4:

[0073] A1: Obtain the total CPU usage, total GPU usage, and total memory usage of all jobs requested by the candidate resource node within a preset historical period.

[0074] A2: Obtain the 95th peak value of the CPU, GPU, and memory of the candidate resource node within a preset historical period;

[0075] The 95% peak value of the CPU of the candidate resource node within the preset historical period indicates the maximum usage that the CPU can achieve 95% of the time. The 95% peak value of the GPU and the 95% peak value of the memory are similar to the 95% peak value of the CPU and are not described here.

[0076] A3: Calculate the first difference between the total CPU amount requested by all jobs and the 95% peak value of CPU, the second difference between the total GPU amount requested by all jobs and the 95% peak value of GPU, and the third difference between the total memory amount requested by all jobs and the 95% peak value of memory for the candidate resource node in a preset historical period.

[0077] A4: Perform a weighted summation of the first difference, the second difference, and the third difference corresponding to the candidate resource nodes to obtain a historical demand matching index value.

[0078] Taking the preset historical period as the past seven days, and the weights of CPU, GPU, and memory as 0.3, 0.5, and 0.2 respectively, as an example, the historical demand matching index value of the candidate resource node = (0.3×((total CPU amount requested by all jobs - 95% peak value of CPU in seven days) / total CPU amount of the node)+0.5×((total GPU amount requested by jobs - 95% peak value of GPU in seven days) / total GPU amount of the node)+0.2×((total memory amount requested by jobs - 95% peak value of memory in seven days) / total memory of the node)) / (0.3+0.5+0.2).

[0079] 105: performing weighted summation on the historical resource utilization index value and the historical demand matching index value to calculate the comprehensive index score of the candidate resource node;

[0080] Exemplarily, priority is given to scheduling jobs to candidate resource nodes with allocation expectations below expectation, that is, scheduling jobs to candidate resource nodes with high historical demand matching index values. The weight of the historical demand matching index is greater than the weight of the historical resource utilization index. For example, if it is allocated as 0.3 and 0.7, the comprehensive index score = historical resource utilization index value × 0.3 + historical demand matching index value × 0.7.

[0081] 106: Determine the candidate resource node with the highest comprehensive indicator score as the target resource node, and generate a first scheduling solution.

[0082] If there is only one candidate resource node, the candidate resource node is determined as the target resource node.

[0083] The first scheduling scheme indicates that the job is allocated to the target resource node. The status of the first scheduling scheme is scheduled and allocated, and includes information such as a unique identification ID of the target resource node.

[0084] It should also be noted that during the job scheduling process, in order to avoid data loss, the job will no longer be scheduled again.

[0085] The present embodiment discloses a method for scheduling a job, which responds to a job opening request submitted by a front-end business platform, obtains resource requirements in the job opening request, selects candidate resource nodes that meet the resource requirements from available resource nodes, obtains historical resource utilization index values ​​and historical demand matching index values ​​of the candidate resource nodes, performs weighted summation of the historical resource utilization index values ​​and historical demand matching index values ​​of the candidate resource nodes, calculates the comprehensive index scores of the candidate resource nodes, and determines the candidate resource with the highest comprehensive index score as the target resource node, thereby generating a scheduling plan. The present embodiment determines whether to determine the candidate resource node as the target resource node based on the historical resource utilization index values ​​and historical demand matching index values ​​of the candidate resource nodes, analyzes the resource utilization of the candidate resource nodes within a preset period and the difference between the resources applied for all jobs and the actual resource utilization peak value, that is, the scheduling method of the upper-level job is determined based on the resource utilization rate and the matching rationality of the underlying resources and the upper-level requirements, and matches the resources according to the actual operation of the job, thereby improving the resource utilization rate of the platform.

[0086] If there are no candidate resource nodes that meet the resource requirements among the available resource nodes, it is necessary to send a resource activation instruction to the resource platform.

[0087] In a possible implementation, first determine the resource pool with the highest average value of the comprehensive index score as the target resource pool, specifically obtain the comprehensive index score of each resource node in each resource pool in the resource platform, and calculate the average value of the comprehensive index score of each resource node in each resource pool, so as to determine the resource pool with the highest average value of the comprehensive index score as the target resource pool. Then determine the resource type and resource specification that meet the resource requirements in the job opening request, generate a second scheduling scheme based on the target resource pool, resource type and resource specification, the status of the second scheduling scheme is scheduled and unallocated, and generate a resource opening request to bind it. Then send a resource opening instruction to the resource platform, the resource opening instruction carries the target resource pool, resource type and resource specification, so that the resource platform opens the resource nodes of the corresponding resource type and resource specification in the target resource pool. After the resource opening is completed, bind the opened resource nodes to the second scheduling scheme, and update the status of the second scheduling scheme to scheduled and allocated.

[0088] In a possible implementation, when the last job running in a resource node is completed, a resource release operation is performed on the resource node to avoid idle resource nodes. Resource nodes with high utilization rates can be retained, and resource nodes with low utilization rates will be naturally eliminated, so that job scheduling and resource operation and maintenance are managed in a unified manner.

[0089] In addition, the job scheduling method provided in this embodiment is non-invasive to the existing platform and can be deployed independently without relying on third-party capabilities. It can be connected based on the customer's existing data interface and can be deployed and integrated quickly and lightly.

[0090] A job scheduling method provided by an embodiment of the present application is introduced above, and a device for executing the above job scheduling method will be introduced below.

[0091] See also Figure 3 , Figure 3 This is a schematic diagram of the structure of a job scheduling device provided in an embodiment of the present application. Figure 3 As shown, the job scheduling device includes:

[0092] The resource requirement acquisition unit 301 is used to respond to the job opening request submitted by the front-end business platform and acquire the resource requirement in the job opening request;

[0093] The resource information acquisition unit 302 is used to acquire resource information of available resource nodes;

[0094] A resource node screening unit 303 is used to screen candidate resource nodes that meet the resource requirements from the available resource nodes according to the resource information of the available resource nodes;

[0095] An index acquisition unit 304 is used to acquire a historical resource utilization index value and a historical demand matching index value of the candidate resource node, wherein the historical demand matching index value represents the difference between the resources applied for by all jobs of the candidate resource node within a preset historical period and the resource utilization peak value;

[0096] An index calculation unit 305 is used to perform a weighted summation of the historical resource utilization index value and the historical demand matching index value to calculate a comprehensive index score of the candidate resource node;

[0097] The scheduling scheme generating unit 306 is configured to determine the candidate resource node with the highest comprehensive indicator score as the target resource node and generate a first scheduling scheme.

[0098] In a possible implementation, the indicator acquisition unit 304 includes:

[0099] The first indicator acquisition subunit is used to obtain the CPU utilization, GPU utilization and memory utilization of the candidate resource node within a preset historical period; perform weighted summation on the CPU utilization, GPU utilization and memory utilization of the candidate resource node within the preset historical period to obtain the historical resource utilization indicator value.

[0100] In a possible implementation, the indicator acquisition unit 304 includes:

[0101] The second indicator acquisition sub-unit is used to obtain the total CPU amount applied for by all jobs, the total GPU amount applied for by all jobs, and the total memory amount applied for by all jobs of the candidate resource node within the preset historical period; obtain the 95% peak value of the CPU, the 95% peak value of the GPU, and the 95% peak value of the memory of the candidate resource node within the preset historical period; respectively calculate the first difference between the total CPU amount applied for by all jobs and the 95% peak value of the CPU, the second difference between the total GPU amount applied for by all jobs and the 95% peak value of the GPU, and the third difference between the total memory amount applied for by all jobs and the 95% peak value of the memory of the candidate resource node within the preset historical period; perform weighted summation of the first difference, the second difference, and the third difference corresponding to the candidate resource node to obtain the historical demand matching indicator value.

[0102] In a possible implementation, the job scheduling device further includes:

[0103] A resource pool determination unit, configured to determine a resource pool having a highest average value of the comprehensive index score as a target resource pool;

[0104] A resource parameter determination unit, configured to determine a resource type and a resource specification that meet the resource requirement, and generate a second scheduling solution according to the target resource pool, the resource type and the resource specification;

[0105] An instruction sending unit, used for sending a resource activation instruction to a resource platform, wherein the resource activation instruction carries the target resource pool, the resource type and the resource specification;

[0106] The data binding unit is used to bind the opened resource node with the second scheduling scheme after the resource opening is completed.

[0107] In a possible implementation, the job scheduling device further includes:

[0108] The resource releasing unit is used to execute a resource releasing operation on the resource node after the last job running in the resource node is completed.

[0109] This embodiment discloses a job scheduling device, which responds to a job opening request submitted by a front-end business platform, obtains resource requirements in the job opening request, selects candidate resource nodes that meet the resource requirements from available resource nodes, obtains historical resource utilization index values ​​and historical demand matching index values ​​of the candidate resource nodes, performs weighted summation of the historical resource utilization index values ​​and historical demand matching index values ​​of the candidate resource nodes, calculates the comprehensive index scores of the candidate resource nodes, and determines the candidate resource with the highest comprehensive index score as the target resource node, thereby generating a scheduling plan. This embodiment determines whether to determine the candidate resource node as the target resource node based on the historical resource utilization index values ​​and historical demand matching index values ​​of the candidate resource nodes, analyzes the resource utilization of the candidate resource nodes within a preset period and the difference between the resources applied for all jobs and the actual resource utilization peak, that is, the resource utilization rate and the matching rationality of the underlying resources and the upper-level requirements are used as the judgment basis to determine the scheduling method of the upper-level job, and the resources are matched according to the actual operation of the job, thereby improving the platform resource utilization rate.

[0110] The present application also provides a job scheduling system. Figure 4 The job scheduling system includes: a job scheduler, a resource controller, an indicator collector, a first API gateway, a second API gateway, a message queue and a database. The first API gateway and the second API gateway realize the internal and external interaction of the job scheduling system, such as authority identification and information authentication.

[0111] The first API gateway is used to receive a job activation request submitted by a front-end business platform, send a request index of the job activation request to the message queue, and store the job activation request in the database;

[0112] The job scheduler is used to extract the job opening request from the message queue and the database, obtain the resource information of the available resource nodes from the resource controller, screen the candidate resource nodes that meet the resource requirements from the available resource nodes based on the resource information of the available resource nodes, obtain the comprehensive index score of the candidate resource node from the index collector, determine the candidate resource node with the highest comprehensive index score as the target resource node, and generate a first scheduling plan, wherein the comprehensive index score is calculated based on the historical resource utilization index value and the historical demand matching index value of the candidate resource node, and the historical demand matching index value represents the difference between the resources applied for by all jobs of the candidate resource node within a preset historical period and the resource utilization peak value.

[0113] The job scheduler is also used to obtain the comprehensive index score of each resource node in each resource pool from the index collector when there is no candidate resource node that meets the resource demand among the available resource nodes, determine the resource pool with the highest average value of the comprehensive index score as the target resource pool, determine the resource type and resource specification that meet the resource demand, generate a second scheduling plan according to the target resource pool, the resource type and the resource specification, and send a resource activation request to the resource controller;

[0114] The resource controller calls the second API gateway to send a resource activation instruction to the resource platform based on the target resource pool, the resource type and the resource specifications in the resource activation request. The resource activation instruction carries the target resource pool, the resource type and the resource specifications. After the resource activation is completed, the activated resource node is bound to the second scheduling scheme.

[0115] The job scheduler sends job deployment and activation instructions to the target resource node in the scheduling plan according to the scheduled and allocated scheduling plan, and calls the first API gateway to return the job activation status to the front-end business platform.

[0116] In addition, the resource controller can implement resource lifecycle management:

[0117] 1. Global resource information collection: Collect the configuration information of resource nodes, including the specifications, types, operators, and geographic locations of resource nodes. Collect the operating status of resource nodes, including the current remaining CPU size, available memory size, CPU and memory utilization, resource over-allocation ratio, allocatable CPU usage, allocatable memory usage, resource validity period (equal to the validity period of the latest job running on the resource node), whether the current status is available, etc. And synchronize the comprehensive indicator scores of resource nodes, which come from the indicator collector.

[0118] 2. Resource addition: Collect resource addition message requests in the message queue and database. The resource addition message clearly states the resource specifications and node locations that need to be added.

[0119] 3. Resource release: After the last job running in a resource node is completed, the resource node is released to avoid the resource node running idle.

[0120] 4. Global resource parameter configuration: used to configure global parameters, such as the over-allocation ratio of each resource pool, the monitoring information collection cycle, the comprehensive indicator value acquisition cycle (i.e. the preset historical cycle mentioned above), the maximum validity period of resources, etc.

[0121] The job scheduling system provided in this embodiment can be applied to a variety of application scenarios:

[0122] 1. Video rendering platform

[0123] The job scheduling system serves as a video rendering platform. The downstream of the video rendering platform connects to the resource pools of multiple manufacturers, provides video rendering services to the upstream, and uses the job scheduling system to improve the utilization rate of resource nodes, thereby increasing the ultimate business profit of the video rendering service provider.

[0124] 2. Large model training platform

[0125] As a large model training platform, the job scheduling system consumes a lot of CPU / GPU resources. The job scheduling system can maximize the value of model analysis and minimize resource costs by assigning job values ​​to different model jobs and scheduling jobs reasonably. This job scheduling system can achieve a better return on investment.

[0126] 3. Hybrid cloud job scheduling

[0127] When an enterprise's own resources are insufficient and there are sudden demands for operations, it can use the job scheduling system to connect to multiple third-party public cloud resource pools. In the event of sudden operations, it can flexibly call backward IaaS resources to achieve the best cost while meeting the resource needs of its own operations.

[0128] An embodiment of the present application also provides a computer program product including computer-readable instructions. When the computer-readable instructions are executed on an electronic device, the electronic device implements any one of the job scheduling methods provided in the embodiments of the present application.

[0129] A computer-readable storage medium is also provided in an embodiment of the present application. The storage medium carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement any job scheduling method provided in an embodiment of the present application.

[0130] It should also be noted that the device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed over multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, in the drawings of the device embodiments provided by the present application, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines.

[0131] Through the description of the above implementation mode, the technicians in the field can clearly understand that the present application can be implemented by means of software plus necessary general hardware, and of course, it can also be implemented by special hardware including special integrated circuits, special CPUs, special memories, special components, etc. In general, all functions completed by computer programs can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be various, such as analog circuits, digital circuits or special circuits. However, for the present application, software program implementation is a better implementation mode in more cases. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer floppy disk, a U disk, a mobile hard disk, a ROM, a RAM, a disk or an optical disk, etc., including a number of instructions to enable a computer device (which can be a personal computer, a training device, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0132] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0133] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, a computer, a training device, or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, training device, or data center. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device, a data center, etc. that includes one or more available media integrations. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)), etc.

Claims

1. A job scheduling method, characterized in that: include: In response to a job opening request submitted by a front-end business platform, obtaining resource requirements in the job opening request; Get resource information of available resource nodes; According to the resource information of the available resource nodes, selecting candidate resource nodes that meet the resource requirement from the available resource nodes; Acquire a historical resource utilization index value and a historical demand matching index value of the candidate resource node, wherein the historical demand matching index value represents the difference between resources applied for by all jobs of the candidate resource node within a preset historical period and a resource utilization peak value; Performing a weighted summation on the historical resource utilization index value and the historical demand matching index value to calculate the comprehensive index score of the candidate resource node; The candidate resource node with the highest comprehensive indicator score is determined as the target resource node, and a first scheduling scheme is generated.

2. The job scheduling method according to claim 1, characterized in that: Obtaining the historical resource utilization index value of the candidate resource node includes: Obtaining CPU utilization, GPU utilization, and memory utilization of the candidate resource node within a preset historical period; The CPU utilization, GPU utilization and memory utilization of the candidate resource node in a preset historical period are weightedly summed to obtain the historical resource utilization index value.

3. The job scheduling method according to claim 1, characterized in that: Obtaining the historical demand matching index value of the candidate resource node includes: Obtain the total amount of CPUs applied for by all jobs, the total amount of GPUs applied for by all jobs, and the total amount of memory applied for by all jobs of the candidate resource node within a preset historical period; Obtaining the 95th peak value of the CPU, the 95th peak value of the GPU, and the 95th peak value of the memory of the candidate resource node within a preset historical period; Calculate respectively a first difference between the total CPU amount applied by all jobs of the candidate resource node in a preset historical period and the 95% peak value of the CPU, a second difference between the total GPU amount applied by all jobs and the 95% peak value of the GPU, and a third difference between the total memory amount applied by all jobs and the 95% peak value of the memory; The first difference, the second difference, and the third difference corresponding to the candidate resource node are weightedly summed to obtain the historical demand matching index value.

4. The job scheduling method according to claim 1, characterized in that: If there is no candidate resource node that meets the resource requirement among the available resource nodes, the job scheduling method further includes: Determine the resource pool with the highest average value of the comprehensive index score as the target resource pool; Determining a resource type and resource specification that meet the resource demand, and generating a second scheduling solution according to the target resource pool, the resource type and the resource specification; Sending a resource activation instruction to a resource platform, wherein the resource activation instruction carries the target resource pool, the resource type, and the resource specification; After the resource activation is completed, the activated resource node is bound to the second scheduling scheme.

5. The job scheduling method according to claim 1, characterized in that: The job scheduling method also includes: When the last job running in the resource node is completed, a resource release operation is performed on the resource node.

6. A job scheduling device, characterized in that: include: A resource requirement acquisition unit, configured to respond to a job opening request submitted by a front-end business platform and acquire resource requirements in the job opening request; A resource information acquisition unit, used to acquire resource information of available resource nodes; A resource node screening unit, configured to screen candidate resource nodes that meet the resource requirement from the available resource nodes according to the resource information of the available resource nodes; An index acquisition unit, used to acquire a historical resource utilization index value and a historical demand matching index value of the candidate resource node, wherein the historical demand matching index value represents the difference between the resources applied for by all jobs of the candidate resource node within a preset historical period and the resource utilization peak value; An index calculation unit, used to perform weighted summation of the historical resource utilization index value and the historical demand matching index value to calculate the comprehensive index score of the candidate resource node; The scheduling scheme generating unit is used to determine the candidate resource node with the highest comprehensive index score as the target resource node and generate a first scheduling scheme.

7. A job scheduling system, characterized in that: include: Job scheduler, resource controller, indicator collector, first API gateway, message queue and database; The first API gateway is used to receive a job activation request submitted by a front-end business platform, send a request index of the job activation request to the message queue, and store the job activation request in the database; The job scheduler is used to extract the job opening request from the message queue and the database, obtain the resource information of the available resource nodes from the resource controller, screen the candidate resource nodes that meet the resource requirements from the available resource nodes based on the resource information of the available resource nodes, obtain the comprehensive index score of the candidate resource node from the index collector, determine the candidate resource node with the highest comprehensive index score as the target resource node, and generate a first scheduling plan, wherein the comprehensive index score is calculated based on the historical resource utilization index value and the historical demand matching index value of the candidate resource node, and the historical demand matching index value represents the difference between the resources applied for by all jobs of the candidate resource node within a preset historical period and the resource utilization peak value.

8. The job scheduling system according to claim 7, characterized in that: The job scheduling system also includes: a second API gateway; The job scheduler is also used to obtain the comprehensive index score of each resource node in each resource pool from the index collector when there is no candidate resource node that meets the resource demand among the available resource nodes, determine the resource pool with the highest average value of the comprehensive index score as the target resource pool, determine the resource type and resource specification that meet the resource demand, generate a second scheduling plan according to the target resource pool, the resource type and the resource specification, and send a resource activation request to the resource controller; The resource controller calls the second API gateway to send a resource activation instruction to the resource platform based on the target resource pool, the resource type and the resource specifications in the resource activation request. The resource activation instruction carries the target resource pool, the resource type and the resource specifications. After the resource activation is completed, the activated resource node is bound to the second scheduling scheme.

9. A computer program product, characterized in that It comprises computer-readable instructions, and when the computer-readable instructions are executed on an electronic device, the electronic device implements the job scheduling method as claimed in any one of claims 1 to 5.

10. A computer storage medium, characterized in that: The storage medium carries one or more computer programs, and when the one or more computer programs are executed by an electronic device, the electronic device can implement the job scheduling method as described in any one of claims 1 to 5.

Images