Resource scheduling method, system and device

Abstract

The application provides a resource scheduling method, system and device. The method comprises the following steps: obtaining at least one service processing request from a first cluster, wherein the service processing request comprises a resource requirement amount required by a service in the service processing request; generating a resource scheduling task according to the at least one service processing request, wherein the resource scheduling task comprises a sum of the resource requirement amounts of each service processing request; obtaining a total resource amount of idle computing nodes, and when the total resource amount of the idle computing nodes is less than a total resource requirement amount, allocating the idle computing nodes corresponding to the total resource amount to the first cluster and confirming a to-be-allocated resource requirement amount; and allocating at least one new idle computing node to the first cluster according to the to-be-allocated resource requirement amount, wherein the resource amount allocated to the first cluster is greater than or equal to the total resource requirement amount. Through the above method, resources can be allocated to the first cluster in batches, so that the cluster resource utilization rate is high and the service processing speed is high.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer, and particularly relates to a resource scheduling method, system and device. BACKGROUND

[0002] With the development of Internet technology, a high performance computing (HPC) cluster is widely applied. Big data, artificial intelligence (AI) and the like can run on the HPC cluster. A unified scheduler in the HPC cluster can schedule cluster resources (computing nodes in the cluster) according to a business processing request, so that the corresponding business can run on the computing nodes in the cluster. For example, the business processing request can be an AI business processing request, a big data business processing request and the like. However, in the related art, the resource utilization rate of the cluster resources is low, and the business processing speed is low. SUMMARY

[0003] The present application provides a resource scheduling method, system and device, which improves the cluster resource utilization rate and the business processing speed.

[0004] In a first aspect, the present application provides a resource scheduling method, comprising:

[0005] obtaining at least one business processing request from a first cluster, wherein the business processing request comprises a resource requirement amount required for running a business in the business processing request;

[0006] generating a resource scheduling task according to the at least one business processing request, wherein the resource scheduling task comprises a total resource requirement amount, and the total resource requirement amount is a sum of the resource requirement amount of each business processing request;

[0007] obtaining a total resource amount of idle computing nodes, and when the total resource amount of idle computing nodes is less than the total resource requirement amount, allocating idle computing nodes corresponding to the total resource amount to the first cluster, and confirming a to-be-allocated resource requirement amount, wherein the to-be-allocated resource requirement amount is the total resource requirement amount minus the total resource amount;

[0008] allocating new idle computing nodes to the first cluster according to the to-be-allocated resource requirement amount at least once, wherein the resource amount allocated to the first cluster is greater than or equal to the total resource requirement amount.

[0009] The technical solution can generate a resource scheduling task according to at least one service processing request, allocate idle computing nodes to the first cluster according to the resource scheduling task, and allocate at least one newly added idle computing node to the first cluster. Through the above method, resources can be allocated to the service processing request as soon as possible, so that the resource utilization of the cluster is high and the service processing speed is high.

[0010] In a possible implementation, the allocating at least one newly added idle computing node to the first cluster according to the to-be-allocated resource quantity includes:

[0011] generating a shadow task according to the to-be-allocated resource quantity, the shadow task including the to-be-allocated resource quantity;

[0012] allocating at least one newly added idle computing node to the first cluster according to the shadow task.

[0013] In the technical solution, a shadow task can be generated according to a to-be-allocated resource quantity, and at least one newly added idle computing node can be allocated to the first cluster according to the shadow task. Through the above method, the purpose of allocating resources to the first cluster in batches is achieved, and the overall resource utilization in the cluster is improved.

[0014] In a possible implementation, the allocating at least one newly added idle computing node to the first cluster according to the shadow task includes:

[0015] determining an i th to-be-allocated resource quantity according to the total resource quantity and the total resource quantity that has been allocated to the first cluster;

[0016] generating an i th shadow task according to the i th to-be-allocated resource quantity;

[0017] allocating resources to the first cluster for the i+1 th time according to the i th shadow task;

[0018] wherein the i is sequentially taken as 1, 2, and so on, until the resource quantity allocated to the first cluster is greater than or equal to the total resource quantity.

[0019] In the technical solution, a shadow task can be generated according to a total resource quantity and a resource quantity that has been allocated to the first cluster, and resources can be continuously allocated to the first cluster according to the shadow task. Through the above method, idle computing nodes in the cluster can be allocated to the first cluster in time, so that the resource utilization in the cluster is high.

[0020] In a possible implementation, the generating a resource scheduling task according to the at least one service processing request includes:

[0021] parsing the at least one service processing request to obtain resource demand amounts of the multiple resource types in each service processing request;

[0022] determining total resource demand amounts of the multiple resource types according to the resource demand amounts of the multiple resource types in each service processing request;

[0023] generating the resource scheduling task according to the total resource demand amounts of the multiple resource types.

[0024] In the technical solution, the resource scheduling task can be generated according to the at least one service processing request, so that the resource scheduling platform allocates resources to the first cluster according to the resource scheduling task.

[0025] In a possible implementation manner, after the idle computing node is allocated to the first cluster, the method further includes:

[0026] determining a second computing node allocated to the first cluster;

[0027] switching a state of the second computing node from an idle state to an occupied state.

[0028] In the technical solution, after a computing node in the cluster is occupied, the state of the computing node can be switched from the idle state to the occupied state, so that the resource scheduling platform avoids allocating the same computing node to multiple first clusters repeatedly.

[0029] In a possible implementation manner, the method further includes:

[0030] obtaining resource usage information of each computing node of the first cluster;

[0031] if it is determined according to the resource usage information that no service is running on a third computing node in the first cluster, recycling the third computing node from the first cluster and switching a state of the third computing node to an idle state.

[0032] In the technical solution, the idle computing node in the first cluster can be recycled in time, so that the computing node can be allocated with a service processing task again, and the resource utilization rate is high.

[0033] In a second aspect, the present application provides a computing device, which includes a memory and a processor, the processor being connected with the memory,

[0034] the processor is configured to generate a resource scheduling task according to at least one service processing request of a first cluster, the resource scheduling task including total resource demand amounts required by services for running the at least one service processing request, the total resource demand amounts being a sum of resource demand amounts of each service processing request;

[0035] The processor is further configured to acquire a total resource amount of idle computing nodes, and when the total resource amount of idle computing nodes is less than the total resource demand amount, allocate the idle computing nodes corresponding to the total resource amount to the first cluster, and determine a to-be-allocated resource demand amount, which is the total resource demand amount minus the total resource amount.

[0036] The processor is further configured to allocate at least one new idle computing node to the first cluster according to the to-be-allocated resource demand amount, wherein the resource amount allocated to the first cluster is greater than or equal to the total resource demand amount.

[0037] In the above technical solution, a resource scheduling task can be generated according to at least one service processing request, idle computing nodes can be allocated to the first cluster according to the resource scheduling task, and a shadow task can be generated; at least one new idle computing node can be allocated to the first cluster according to the shadow task. Through the above method, resources can be allocated to the first cluster in batches, so that the overall resource utilization rate in the cluster is high and the service processing speed is fast.

[0038] In a possible implementation manner,

[0039] The processor is further configured to generate a shadow task when the total resource amount is less than the total resource demand amount, and allocate at least one new idle computing node to the first cluster according to the shadow task.

[0040] The processor is further configured to acquire resource usage information of each computing node in the first cluster, and if it is determined according to the resource usage information that no service is running on a third computing node in the first cluster, recycle the third computing node from the first cluster, and switch the state of the third computing node to an idle state.

[0041] In the above technical solution, resources can be continuously allocated to the first cluster according to the shadow task, the purpose of allocating resources to the first cluster in batches is achieved, and the overall resource utilization rate in the cluster is improved. In addition, idle resources in the first cluster can be recycled, so that the service processing task can be allocated to the computing node again, and the resource utilization rate is high.

[0042] In a third aspect, the present application provides a resource scheduling system, comprising the computing device of the second aspect and at least one first cluster, wherein,

[0043] The computing device is configured to execute the method of any one of the first aspect to allocate idle computing nodes to the first cluster and allocate at least one new idle computing node to the first cluster;

[0044] The first cluster is configured to schedule the allocated idle computing nodes to execute the at least one service processing request.

[0045] In the technical solution, the resource scheduling task is generated according to the at least one service processing request, the idle computing nodes are allocated to the first cluster according to the resource scheduling task, and the shadow task is generated; and the newly added idle computing nodes are allocated to the first cluster at least once according to the shadow task. Through the method, the resources can be allocated to the first cluster in batches, so that the overall resource utilization rate in the cluster is high, and the service processing speed is fast.

[0046] In a fourth aspect, the present application provides a resource scheduling device, comprising a processor and a memory connected with the processor in communication;

[0047] The memory stores a computer program;

[0048] The processor executes the computer program to implement the method according to any one of the first aspect.

[0049] In the technical solution, the resource scheduling task is generated according to the at least one service processing request, the idle computing nodes are allocated to the first cluster according to the resource scheduling task, and the shadow task is generated; and the newly added idle computing nodes are allocated to the first cluster at least once according to the shadow task. Through the method, the resources can be allocated to the first cluster in batches, so that the overall resource utilization rate in the cluster is high, and the service processing speed is fast.

[0050] In a fifth aspect, the present application provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a computer to implement the method according to any one of the first aspect.

[0051] In the technical solution, the resource scheduling task is generated according to the at least one service processing request, the idle computing nodes are allocated to the first cluster according to the resource scheduling task, and the shadow task is generated; and the newly added idle computing nodes are allocated to the first cluster at least once according to the shadow task. Through the method, the resources can be allocated to the first cluster in batches, so that the overall resource utilization rate in the cluster is high, and the service processing speed is fast.

[0052] In a sixth aspect, the present application provides a computer program product, comprising a computer program, and the computer program is executed by a computer to implement the method according to any one of the first aspect.

[0053] In the technical solution, the resource scheduling task is generated according to the at least one service processing request, the idle computing nodes are allocated to the first cluster according to the resource scheduling task, and the shadow task is generated; and the newly added idle computing nodes are allocated to the first cluster at least once according to the shadow task. Through the method, the resources can be allocated to the first cluster in batches, so that the overall resource utilization rate in the cluster is high, and the service processing speed is fast. Attached Figure Description

[0054] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0055] Figure 1 This is a schematic diagram of the architecture of a unified scheduling platform;

[0056] Figure 2 This is a schematic diagram of a hybrid scheduling architecture;

[0057] Figure 3 A schematic diagram of a resource scheduling system architecture provided in this application embodiment;

[0058] Figure 4 A flowchart illustrating a resource scheduling method provided in an embodiment of this application;

[0059] Figure 5 A flowchart illustrating another resource scheduling method provided in an embodiment of this application;

[0060] Figure 6 This is a schematic diagram illustrating a process for allocating resources to a first cluster, as provided in an embodiment of this application.

[0061] Figure 7 A schematic diagram of the structure of a resource scheduling platform provided in this application;

[0062] Figure 8 A schematic diagram of another resource scheduling platform provided in this application;

[0063] Figure 9 A schematic diagram of a recycling computing node provided in this application;

[0064] Figure 10 A resource scheduling system provided in this application embodiment;

[0065] Figure 11 This is a schematic diagram of the hardware structure of a resource scheduling device provided in an embodiment of this application. Detailed Implementation

[0066] The exemplary embodiments will be described in detail herein with reference to the attached drawings. The description herein refers to the accompanying drawings, which show by way of example the exemplary embodiments. In the following description, like reference numerals refer to like elements, unless the context clearly dictates otherwise. The following description of exemplary embodiments is not representative of all possible embodiments consistent with the present application. Instead, it is merely an example of apparatus and methods consistent with some aspects of the present application as detailed in the appended claims.

[0067] It should be noted that the terms "comprises", "comprising", or any other variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by "comprises... a" does not, without more constraints, exclude the existence of additional identical elements in the process, method, article, or apparatus that comprises the element.

[0068] For ease of understanding, first, two resource scheduling methods are exemplarily shown in Figures 1-2

[0069] Figure 1 An architecture schematic diagram of a unified scheduling platform is shown. Please refer to Figure 1 The unified scheduling platform can be connected with Yet Another Resource Negotiator (YARN) and Kubernetes (K8s) at the same time. For example, the unified scheduling platform can be YuniKorn.

[0070] YARN and K8s can be a third-party cluster resource manager.

[0071] YARN can receive a type of service submitted by a user and send the type of service to the unified scheduling platform. For example, the type of service can be a big data service. K8s can receive another type of service submitted by a user and send the another type of service to the unified scheduling platform. For example, the another type of service can be an AI service. It should be understood that the number of services sent by YARN to the unified scheduling platform and the number of services sent by K8s to the unified scheduling platform can be one or more.

[0072] ​After receiving services from YARN and Kubernetes, the unified scheduling platform can schedule cluster resources for the received services. The platform schedules resources as follows: if the available idle resources in the cluster are less than the resource amount requested by the service request, the platform can wait until the available idle resources are greater than or equal to the requested resource amount before scheduling cluster resources for the service request. However, this results in some idle resources in the cluster going unused, and service requests not being processed in a timely manner, leading to low cluster resource utilization and slow service processing speed.

[0073] Moreover, in the above methods, YARN and K8s do not participate in resource scheduling, and resource scheduling is only performed by a unified scheduling platform. The unified scheduling platform needs to be designed to be compatible with different cluster resource managers to perform resource scheduling for services sent by different cluster resource managers, which makes the business complexity of the unified scheduling platform high and the business overhead large.

[0074] Figure 2 This is a schematic diagram of a hybrid scheduling architecture. Please refer to [link / reference]. Figure 2 Both the first and second scheduling platforms can connect to (communicate with) the coordination platform. For example, the first scheduling platform can be a Sigma scheduler, the second scheduling platform can be a Fuxi scheduler, and the coordination platform can be Level-0.

[0075] The coordination platform can schedule cluster resources. It acts as a coordination controller, coordinating resource allocation between the first and second scheduling platforms. Specifically, the first or second scheduling platform schedules cluster resources as follows: if the available idle resources in the cluster are less than the resource amount requested by the service processing request, the first or second scheduling platform can wait until the available idle resources in the cluster are greater than or equal to the resource amount requested by the service processing request before scheduling cluster resources for the request. However, this results in some idle resources in the cluster not being utilized, and service processing requests not being processed in a timely manner, leading to low cluster resource utilization and low service processing speed.

[0076] Furthermore, in the above method, the first and second scheduling platforms need to negotiate with a coordination platform when scheduling resources. If the cluster's business throughput is high, the coordination platform needs to frequently negotiate resource allocation between the first and second scheduling platforms, resulting in low resource scheduling efficiency and slow business processing speed.

[0077] In view of this, embodiments of this application provide a resource scheduling method to improve resource utilization and increase business processing speed.

[0078] Below, in conjunction withFigure 3 The resource scheduling system architecture involved in the embodiments of this application will be described.

[0079] Figure 3 This is a schematic diagram of a resource scheduling system architecture provided for an embodiment of this application. Please refer to [link / reference]. Figure 3 The resource scheduling system includes: a resource pool, a resource scheduling platform, and multiple schedulers (e.g., ...). Figure 3 The resource scheduling platform can communicate with the resource pool and multiple schedulers, and the multiple schedulers can communicate with the resource pool.

[0080] A resource scheduling platform can be a resource management and scheduling system within a cluster. It can be a software program or a device. When it's a software program, it can be stored in the memory of a computing device and run through its processor. This computing device can be a server, computer, mobile phone, or other device with processing capabilities. When it's a device, it can also be a server, computer, mobile phone, or other device with processing capabilities. The resource scheduling platform can schedule and allocate idle nodes in the resource pool based on the resource information required by the user's submitted business processing requests.

[0081] The resource pool can be any of the compute nodes in the cluster (e.g., ...). Figure 3 The cluster is a collection of computing nodes M, ..., and computing nodes N. For example, computing nodes can be servers, desktop computers, tablets, or artificial intelligence devices. Computing nodes can also be computing units, which are components capable of computing power, such as motherboards and CPUs. Computing nodes in the cluster can connect to a resource scheduling platform via an interface, allowing the platform to update the status of the computing nodes in real time. The status of a computing node can be idle, partially occupied, or fully occupied. It should be noted that the number of computing nodes M in the resource scheduling system can be different from the number of schedulers n, or the number of computing nodes M and the number of schedulers n can be the same; this application does not impose any limitations on this.

[0082] The scheduler can be a software program or a device. When the scheduler is a software program, it can be stored in the memory of a computing device and run by the device's processor. The computing device can be a server, computer, mobile phone, or other computing device with processing capabilities. When the scheduler is a device, it can be a server, computer, mobile phone, or other computing device with processing capabilities.

[0083] In some embodiments, when the resource scheduling platform is a software program, the resource scheduling platform can be stored in a computing device or in a computing node of the resource pool.

[0084] In some embodiments, when the scheduler is a software program, the scheduler can be stored in a computing device or in a computing node of the resource pool. In some embodiments, a plurality of schedulers can be stored in the same computing device or computing node, and one or more schedulers can be stored in the same computing device or computing node as the resource scheduling platform.

[0085] In the resource scheduling system described above, a user can submit a service processing request to the scheduler, and the service processing request can include the resource requirement amount required for running the service. For example, the user can send a service processing request to the scheduler through a client. The resource scheduling platform can periodically obtain the service processing request from the scheduler, determine the total resource requirement amount corresponding to the service processing request, and allocate the idle computing nodes in the resource pool to the corresponding scheduler, so that the scheduler can run at least one sub-service of the service corresponding to the service processing request on the idle computing nodes. It should be understood that the scheduler can be connected to each computing node in the resource pool through a wired or wireless manner.

[0086] It should be noted that the resource scheduling platform obtaining the service processing request from the scheduler mainly includes the following two cases:

[0087] Case 1: The resource scheduling platform can actively read the service processing request from each scheduler.

[0088] In this case, there is no need to configure the function of sending the service processing request for the current scheduler, so that the improvement cost of the scheduler is small.

[0089] Case 2: Each scheduler can actively send the service processing request to the resource scheduling platform.

[0090] In this case, the function of sending the service processing request at a fixed time can be configured at each scheduler, so that the complexity of the resource scheduling platform is low.

[0091] It should be noted that for any one scheduler, the computing nodes allocated to the scheduler by the resource scheduling platform and the scheduler can form a sub-cluster. For example, Figure 3 For example, the scheduler 1, the computing node 1, and the computing node 2 can form a sub-cluster, and the scheduler n, the computing node M-1, and the computing node M can form a sub-cluster.

[0092] Optionally, the user can also directly submit a service processing request to the resource scheduling platform, and the service processing request can include the resource requirement amount required when the service is running. After receiving the service processing request submitted by the user, the resource scheduling platform can run the service processing request on the idle computing nodes or the partially occupied computing nodes in the resource pool.

[0093] It should be noted that the types of service processing requests received by each resource scheduler and the resource scheduling platform can be different. For example, Figure 3 For example, the scheduler 1 can receive service A, the scheduler 2 can receive service B, and the resource scheduling platform can receive service C. Services A, B, and C can be different types of services.

[0094] When allocating resources, if the amount of resources that can be provided by the idle computing nodes in the current cluster is less than the total resource requirement amount corresponding to the at least one service processing request, the resource scheduling platform can first allocate the idle computing nodes in the current cluster to the corresponding scheduler, and the scheduler allocates at least one sub-service to the idle computing node, so that the idle computing node runs the at least one sub-service. When there are new idle computing nodes in the resource pool, the resource scheduling platform continues to allocate resources to the scheduler until the amount of resources allocated to the scheduler is greater than or equal to the total resource requirement amount corresponding to the at least one service processing request.

[0095] Through the above resource scheduling method, the idle computing nodes in the resource pool can be allocated to the scheduler in time, so that the resource utilization rate in the resource pool is high. In addition, only one resource scheduling platform is used in the present application, and compared with the architecture of multiple resource scheduling platforms, resource negotiation between multiple resource scheduling platforms is not required, so that the service processing speed is fast.

[0096] In some embodiments, the resource scheduling platform can receive the service processing request of each scheduler, and allocate resources to multiple schedulers in a predetermined order, for example, the predetermined order can be the time order of receiving the service processing request, or the priority order of the schedulers, etc., which is not limited in the present application.

[0097] In the following, the technical solutions shown in the present application will be described in detail through specific embodiments. It should be noted that the following specific embodiments can exist independently or can be combined with each other, and the same or similar contents will not be repeated in different embodiments.

[0098] Figure 4 A flowchart of a resource scheduling method provided by an embodiment of the present application is shown. Please refer to Figure 4 The method can include:

[0099] S401. Obtain a service processing request from a first cluster, the service processing request comprising resource requirement amount required for processing a service in the service processing request.

[0100] The execution subject of the embodiment can be a resource scheduling platform, or a resource scheduling device in the resource scheduling platform.

[0101] In the embodiment, the resource scheduling platform, the scheduler and the plurality of computing nodes in the resource pool can form a service processing cluster.

[0102] The first cluster can be a sub-cluster in the service processing cluster. The first cluster can comprise a scheduler and a plurality of computing nodes.

[0103] In the embodiment, the service processing cluster can comprise one or more first clusters. Each first cluster can be a cluster for a different type of service, and the schedulers included in different types of first clusters are different. For example, the first cluster can be a K8s cluster, and the K8s cluster can comprise a K8s scheduler. The first cluster can receive a service processing request submitted by a user. That is, the user can submit a service processing request to the first cluster.

[0104] In the implementation process, the user can submit a service processing request to the scheduler in the first cluster. The service processing request can be one service request, or a plurality of service processing requests. The resource scheduling platform can obtain the service processing request submitted by the user from the scheduler of the first cluster.

[0105] It should be understood that one first cluster can receive one type of service processing request. For example, if the first cluster is a K8s cluster, the service processing request that the first cluster can receive is an AI service processing request.

[0106] In the embodiment, one service processing cluster can comprise a plurality of first clusters. Each first cluster can receive one type of service processing request submitted by a user. The resource scheduling platform can obtain the service processing request from each first cluster.

[0107] The resource types can comprise memory, central processing unit (CPU) and graphics processing unit (GPU), etc.

[0108] For example, one service processing request can comprise the requirement amount of memory, the requirement amount of CPU and the requirement amount of GPU.

[0109] It should be noted that memory and CPU are required during service running, and therefore, the service processing request can comprise at least two types of resource requirement amount.

[0110] S402, generating a resource scheduling task according to the service processing request, the resource scheduling task including a total resource demand, the total resource demand being a sum of resource demands of each service processing request.

[0111] The resource scheduling task can be a task of allocating idle computing nodes of the first cluster according to the total resource amount required by the service processing request.

[0112] In this embodiment, at least one service processing request can be parsed and processed to obtain resource demands of various resource types in each service processing request; the total resource demands of various resource types are determined according to the resource demands of various resource types in each service processing request; and the resource scheduling task is generated according to the total resource demands of various resource types.

[0113] In the specific implementation process, the service processing requests received by the first cluster can be obtained from the first cluster at a fixed time interval, and the resource demands of various resource types corresponding to the service processing requests are parsed and obtained.

[0114] It should be understood that the number of service processing requests obtained from the first cluster can be one or more.

[0115] In one possible implementation, if the number of service processing requests obtained from the first cluster is one, the resource scheduling task can be generated according to the resource demands of various resource types included in the service processing request. The resource scheduling task can include the resource demands of various resource types corresponding to the service processing request.

[0116] For example, it is assumed that the number of service processing requests obtained from the first cluster is one. The service processing request includes a demand amount of memory and a demand amount of CPU, and the demand amount of memory is 256G and the demand amount of CPU is 500 cores. The total resource demand of memory included in the service processing request is 256G, and the total resource demand of CPU is 500 cores. The resource demands included in the resource scheduling task can be: memory 256G, CPU 500 cores.

[0117] In another possible implementation, if the number of service processing requests obtained from the first cluster is more than one, the resource scheduling task can be generated according to the resource demands of various resource types included in the multiple service processing requests. The resource scheduling task can include the resource demands of various resource types corresponding to the multiple service processing requests.

[0118] For example, assume that the number of service processing requests obtained from the first cluster is ten. Each of the ten service processing requests includes only a memory requirement and a CPU requirement, and the memory requirement is 128G and the CPU requirement is 100 cores. The total resource requirement of the ten service processing requests includes a total memory requirement of 1280G and a total CPU requirement of 1000 cores. The resource requirement included in the resource scheduling task can be: memory 1280G, CPU 1000 cores.

[0119] S403, obtain the total resource amount of the idle computing nodes, and when the total resource amount of the idle computing nodes is less than the total resource requirement, allocate the idle computing nodes corresponding to the total resource amount to the first cluster, and confirm the to-be-allocated resource requirement.

[0120] The to-be-allocated resource requirement is the total resource requirement minus the total resource amount. It should be understood that the to-be-allocated resource amount is the resource amount that the resource scheduling platform needs to allocate to the first cluster after allocating the current idle computing nodes to the first cluster.

[0121] The idle computing nodes can be nodes in the cluster in an idle state. It should be understood that the resource scheduling platform can obtain the node state of each node in the cluster. The number of idle computing nodes can be one or more.

[0122] The total resource amount of the idle computing nodes can be the sum of the resource amounts of all idle nodes.

[0123] Optionally, if the number of idle computing nodes in the resource pool is one, the total resource amount of the idle computing nodes can be the resource amount on the one idle computing node.

[0124] Optionally, if the number of idle computing nodes in the resource pool is multiple, the total resource amount of the idle computing nodes can be the sum of the resource amounts on the multiple idle computing nodes.

[0125] In this embodiment, the total resource amount of the current idle computing nodes in the resource pool can be greater than or equal to the total resource requirement included in the resource scheduling task; or the total resource amount of the current idle computing nodes in the resource pool can be less than the total resource requirement included in the resource scheduling task.

[0126] In one possible implementation, if the total resource amount of the current idle computing nodes in the resource pool is greater than or equal to the total resource requirement, the resource scheduling platform can allocate the required resource amount to the first cluster through one resource allocation.

[0127] For example, if the total resource requirement included in the resource scheduling task is: 128G of memory and 372 cores of CPU, and there is a computing node 1 idle in the cluster, and the total resource of the computing node 1 is: 256G of memory and 500 cores of CPU, the resource scheduling platform can allocate the computing node 1 to the first cluster, so that the scheduler in the first cluster can schedule the corresponding service to the computing node 1.

[0128] In another possible implementation, if the total resource of the idle computing node in the cluster is less than the total resource requirement included in the resource scheduling task, the resource scheduling platform can first allocate the idle computing node corresponding to the total resource to the first cluster, and confirm the resource to be allocated.

[0129] For example, if the total resource requirement included in the resource scheduling task is: 1000G of memory, 500 cores of CPU, and 40 cards of GPU, and there is only a computing node 1 idle in the cluster, and the total resource of the computing node 1 is: 256G of memory, 128 cores of CPU, and 5 cards of GPU, the resource scheduling platform can allocate the computing node 1 to the first cluster. The resource required by the resource scheduling task is: 744G of memory, 372 cores of CPU, and 35 cards of GPU. That is, the resource to be allocated is: 744G of memory, 372 cores of CPU, and 35 cards of GPU.

[0130] It can be understood that the card in the GPU 5 card refers to one GPU card, and generally one GPU card can be provided with one GPU chip.

[0131] S404, allocate the newly added idle computing node to the first cluster according to the resource requirement to be allocated at least once, wherein the resource allocated to the first cluster is greater than or equal to the total resource requirement.

[0132] In this embodiment, after the resource scheduling platform determines the resource to be allocated, if there is a newly added idle computing node in the resource pool, the resource scheduling platform can continue to allocate the newly added idle computing node to the first cluster according to the resource requirement to be allocated. Until the resource allocated to the first cluster is greater than or equal to the total resource requirement

[0133] For example, assuming that the resource to be allocated is: 744G of memory, 372 cores of CPU, and 35 cards of GPU, and there is a newly added idle computing node in the resource pool, and the newly added idle computing node is computing node 2, and the total resource of the computing node 2 is: 1024G of memory, 500 cores of CPU, and 40 cards of GPU. The total resource of various types in the computing node 2 is greater than the resource to be allocated. The resource scheduling platform can allocate the computing node 2 to the first cluster. At this time, the resource scheduling platform completes the resource allocation of the first cluster.

[0134] In the resource scheduling method provided in this embodiment, the resource scheduling platform can obtain at least one service processing request from the first cluster and generate a resource scheduling task according to the at least one service processing request. If the total resource amount of the currently idle computing nodes in the cluster is less than the resource requirement amount included in the resource scheduling task, the resource scheduling platform can allocate resources to the first cluster in batches. In this way, the overall resource utilization of the cluster is improved, and the service processing speed of the first cluster is also accelerated, so that the cluster has a higher resource utilization and a higher service processing speed.

[0135] In addition, the resource scheduling platform in this embodiment can allocate idle computing nodes to the scheduler, so that the scheduler can allocate the services in the service processing requests to the idle computing nodes to run. That is, in this embodiment, the execution subject for allocating resources for each service request is the scheduler, and the resource scheduling platform does not need to allocate services for the service processing requests, so that the resource scheduling platform does not need to be compatible with different schedulers, and the business complexity of the resource scheduling platform is lower and the business overhead is smaller.

[0136] In Figure 4 Based on the embodiment, if the total resource amount of the currently idle computing nodes in the cluster is less than the total resource requirement amount included in the resource scheduling task, the resource scheduling platform can generate a shadow task and allocate resources to the first cluster in batches according to the shadow task. Next, the process of allocating resources to the first cluster in batches according to the shadow task will be described in detail. Figure 5 The above scheme will be described in detail.

[0137] Figure 5 Another resource scheduling method provided in this embodiment is shown in the flowchart. Please refer to Figure 5 The method can include the following steps.

[0138] S501, obtaining a service processing request from a first cluster, the service processing request including a resource requirement amount required for running a service in the service processing request.

[0139] It should be noted that the specific implementation of S501 can refer to S401, which will not be described here.

[0140] S502, performing analysis processing on the service processing request to obtain the resource requirement amount in each service processing request.

[0141] In this embodiment, the process of analyzing each service processing request is the same, and next, only one of the service processing requests will be taken as an example to describe the analysis processing of the service processing request by the resource scheduling platform.

[0142] For example, it is assumed that the resource scheduling platform obtains a service processing request 1 from the first cluster. The service processing request 1 includes a requirement of 256G memory and a requirement of 300 cores of CPU. After the resource scheduling platform parses the service processing request 1, the resource scheduling platform can obtain the resource requirement of each resource type corresponding to the service processing request 1, i.e., the requirement of 256G memory and the requirement of 300 cores of CPU.

[0143] S503, determining a total resource requirement according to the resource requirement in each service processing request, and generating a resource scheduling task.

[0144] In this embodiment, each service processing request can include a resource requirement of one resource type, or each service processing request can include resource requirements of multiple types of resources. In the implementation process, the resource scheduling platform can sum the resource requirements of the same type of resources in multiple service processing requests according to the resource type to determine the resource requirement of each resource type.

[0145] For example, it is assumed that three service processing requests are obtained from the first cluster, and the resource requirements of various resources included in the three service processing requests can be as shown in Table 1:

[0146] Table 1

[0147]

[0148] The resource requirements of the service processing request 1, the service processing request 2 and the service processing request 3 are summed to obtain the total resource requirement of each resource type. The total resource requirement of each resource type can be as shown in Table 2:

[0149] Table 2

[0150]

[0151] In this embodiment, the resource scheduling platform can generate a resource scheduling task according to the resource requirements included in the obtained service processing requests. The resource scheduling task includes the total resource requirement, and the total resource requirement is the sum of the resource requirements of each service processing request.

[0152] For example, it is assumed that the total resource requirement is as shown in Table 2, and the total resource requirement included in the resource scheduling task can be: 52 cores of CPU, 1408G of memory, and 14 GPU cards.

[0153] S504, obtaining a total resource amount of idle computing nodes, and determining whether the total resource amount of idle computing nodes is greater than or equal to the total resource requirement;

[0154] If yes, S505 is performed;

[0155] If no, S506 is performed.

[0156] It should be noted that if the total amount of each type of resource on the idle computing node is greater than or equal to the corresponding amount of total resource demand, the resource scheduling platform can determine that the total amount of resources of the idle computing node is greater than or equal to the total resource demand; otherwise, if the total amount of any type of resource on the idle computing node is less than the corresponding amount of total resource demand, the resource scheduling platform can determine that the total amount of resources of the idle computing node is less than the total resource demand.

[0157] For example, assume that the total resource demand in the resource scheduling task includes: 52 cores of CPU, 1408G of memory, and 14 cards of GPU. If the total amount of CPU in the idle computing node is greater than or equal to 52 cores, the total amount of memory is greater than or equal to 1408G, and the total amount of GPU is greater than or equal to 14 cards, the resource scheduling platform can determine that the total amount of resources of the idle computing node is greater than or equal to the total resource demand. If the total amount of CPU in the idle computing node is less than 52 cores, or the total amount of memory is less than 1408G, or the total amount of GPU is less than 14 cards, the resource scheduling platform can determine that the total amount of resources of the idle computing node is less than the total resource demand.

[0158] S505, determining a target computing node in the idle computing node according to the total resource demand, and allocating the target computing node to the first cluster.

[0159] It should be understood that the number of idle computing nodes can be one or more. If there are currently multiple available idle computing nodes, the resource scheduling platform can determine one or more idle computing nodes in the multiple idle computing nodes according to a preset order, and allocate the determined idle computing node, i.e., the target computing node, to the first cluster.

[0160] The preset order can be the registration order of all computing nodes at the initialization of the cluster, or can be an order preset according to the address of the computing node, and the application does not limit this.

[0161] S506, allocating the idle computing node to the first cluster, and generating a shadow task, and allocating at least one newly added idle computing node to the first cluster according to the shadow task.

[0162] In this embodiment, after the resource scheduling platform allocates the idle computing node to the first cluster, the resource scheduling platform can determine the to-be-allocated resource demand, and can generate a shadow task according to the to-be-allocated resource demand. The shadow task can include the to-be-allocated resource demand.

[0163] The shadow task can be a task of allocating an idle computing node to the first cluster according to the to-be-allocated resource demand. The number of shadow tasks can be one or more.

[0164] Shadow tasks can include various types of resource requirements. These various types of resource requirements in a shadow task can be: the total resource requirements of various types included in the resource scheduling task, minus the total amount of corresponding resources of various types that can be provided by the idle computing nodes already allocated to the first cluster.

[0165] For example, suppose the total resource requirements of the resource scheduling task are: 52 CPU cores, 1424GB memory, and 14 GPUs. The total resources that the idle computing nodes already allocated to the first cluster can provide are: 32 CPU cores, 1024GB memory, and 4 GPUs. Then the resource requirements of the shadow task could be: 20 CPU cores, 400GB memory, and 10 GPUs.

[0166] In this embodiment, shadow tasks can be generated in the following way, and resources can be allocated to the first cluster based on the shadow tasks: determine the i-th resource quantity to be allocated based on the total resource demand and the total resource quantity already allocated to the first cluster; generate the i-th shadow task based on the i-th resource quantity to be allocated; and perform the (i+1)-th resource allocation for the first cluster based on the i-th shadow task; where i takes the values ​​1, 2, ..., until the resource quantity allocated to the first cluster is greater than or equal to the total resource demand.

[0167] The amount of resources to be allocated can be the amount of resources that need to be allocated to the first cluster.

[0168] The i-th resource to be allocated can be the resource requirement included in the i-th shadow task.

[0169] Below, in conjunction with Figure 6 The process of generating shadow tasks and allocating resources to the first cluster based on the shadow tasks is explained.

[0170] Figure 6 This is a schematic diagram illustrating a resource allocation process for a first cluster, provided as an embodiment of this application. Please refer to... Figure 6 The total resource requirements for the resource scheduling task are: 500 CPU cores and 1000GB of memory. Initially, there is one idle compute node 1 in the cluster. Assume that idle compute node 1 can provide the following resources: 128 CPU cores and 256GB of memory.

[0171] When the resource scheduling platform performs the first resource allocation for the first cluster, it can allocate computing node 1 to the first cluster.

[0172] Based on the total resource demand and the resources provided by computing node 1, the resource scheduling platform determines the first resource to be allocated as: 372 CPU cores and 744GB of memory. Therefore, the resource demand for the first shadow task can be: 372 CPU cores and 744GB of memory.

[0173] When there is a newly added idle computing node (computing node 2) in the cluster, the resource scheduling platform can perform the second resource allocation for the first cluster according to the first shadow task, that is, allocate computing node 2 to the first cluster.

[0174] The resource requirement included in the first shadow task can be: CPU 372 cores, memory 744G. It is assumed that the resource amount that computing node 2 can provide is: CPU 100 cores, memory 300G.

[0175] The resource scheduling platform can determine the second to-be-allocated resource amount as: CPU 272 cores, memory 444G according to the resource requirement included in the first shadow task and the resource amount that computing node 2 provides. The resource requirement included in the second shadow task can be: CPU 272 cores, memory 444G.

[0176] When there is a newly added idle computing node (computing node 3) in the cluster, the resource scheduling platform can perform the third resource allocation for the first cluster according to the second shadow task, that is, allocate computing node 3 to the first cluster.

[0177] The resource requirement included in the second shadow task can be: CPU 272 cores, memory 444G. It is assumed that the resource amount that computing node 3 can provide is: CPU 96 cores, memory 256G.

[0178] The resource scheduling platform can determine the third to-be-allocated resource amount as: CPU 76 cores, memory 188G according to the resource requirement included in the second shadow task and the resource amount that computing node 3 provides. The resource requirement included in the third shadow task can be: CPU 76 cores, memory 188G.

[0179] When there is a newly added idle computing node (computing node 4) in the cluster, the resource scheduling platform can perform the fourth resource allocation for the first cluster according to the third shadow task, that is, allocate computing node 4 to the first cluster.

[0180] The resource requirement included in the third shadow task can be: CPU 76 cores, memory 188G. It is assumed that the resource amount that computing node 4 can provide is: CPU 128 cores, memory 256G. Since the resource amount that computing node 4 can provide is greater than the resource requirement included in the third shadow task, no to-be-allocated resource amount needs to be determined after computing node 4 is allocated to the first cluster.

[0181] In this embodiment, if the total resource amount in the i+1th idle computing node is greater than or equal to the resource requirement amount included in the ith shadow task, the i+1th idle computing node can be allocated to the first cluster, and no shadow task is generated. If the total resource amount in the i+1th idle computing node is less than the resource requirement amount included in the ith shadow task, the i+1th idle computing node can be allocated to the first cluster, and the i+1th shadow task can be generated to continue allocating resources to the first cluster.

[0182] For example, the total resource amount (CPU 128 cores, memory 256G) in the 4th idle computing node is greater than the resource requirement amount (CPU 76 cores, memory 188G) included in the 3rd shadow task, and the 4th idle computing node can be allocated to the first cluster according to the resource requirement amount included in the 3rd shadow task, and no shadow task is generated. Figure 6

[0183] For example, the total resource amount (CPU 96 cores, memory 256G) in the 3rd idle computing node is less than the resource requirement amount (CPU 272 cores, memory 444G) included in the 2nd shadow task, and the 3rd idle computing node can be allocated to the first cluster, and the 3rd shadow task can be generated. Figure 6

[0184] In this embodiment, the resource scheduling platform can allocate idle computing nodes to the first cluster according to the service processing request obtained from the first cluster. If the total resource amount of the current idle computing node is less than the total resource requirement amount required by the service processing request, the resource scheduling platform can continue to allocate idle computing nodes to the first cluster when there is a new idle computing node in the resource pool. The scheduler in the first cluster can run at least one service corresponding to the service processing request on the idle computing node. It should be noted that the manner in which the scheduler runs the service corresponding to the service processing request according to the total resource amount that the idle computing node can provide and the total resource requirement amount required by the service processing request can be known from related technologies, and will not be described here.

[0185] S507, updating the state of the second computing node.

[0186] The second computing node can be an idle computing node currently allocated to the first cluster by the resource scheduling platform.

[0187] In this embodiment, after allocating the idle computing node to the first cluster, the method further includes: determining a second computing node allocated to the first cluster; and updating the state of the second computing node from an idle state to a fully occupied state.

[0188] ​​It should be noted that, for any idle computing node in the cluster, after the resource scheduling platform allocates the computing node to the first cluster, the scheduler in the first cluster can expand the computing node to the first cluster, so that the first cluster can run business on the computing node, and the computing node is no longer idle. Therefore, the resource scheduling platform can update the state of the computing node to the full occupancy state.

[0189] In the implementation process, the resource scheduling platform can send node information of the idle computing node (for example, the node information can include the name, Internet Protocol (IP) address of the computing node, and the amount of resources that can be provided) to the scheduler in the first cluster. After the scheduler in the first cluster receives the node information, it can incorporate the corresponding computing node into the first cluster through an expansion command. After the scheduler incorporates the idle computing node into the first cluster, it can run business on the computing node.

[0190] S508, updating the state of the third computing node.

[0191] The third computing node can be an idle computing node in the first cluster.

[0192] In this embodiment, the resource usage information of each computing node of the first cluster can also be obtained; if it is determined according to the resource usage information that no business is running on the third computing node in the first cluster, the third computing node is recovered from the first cluster, and the state of the third computing node is updated to an idle state.

[0193] It should be understood that, for any computing node in the first cluster, if the currently running business on the computing node ends and no other business runs on the computing node, the computing node can be idle. Then the resource scheduling platform can recover the computing node.

[0194] In the implementation process, the resource scheduling platform can periodically monitor whether there is a running business on each computing node in the first cluster. For any computing node in the first cluster, if the resource scheduling platform continuously monitors that there is no running business on the computing node more than or equal to a preset threshold. Then the resource scheduling platform can recover the computing node from the first cluster, and update the state of the computing node to an idle state. For example, after the resource scheduling platform determines the idle computing node in the first cluster, the scheduler in the first cluster can be triggered to execute a shrink command, so that the resource scheduling platform can recover the computing node from the first cluster.

[0195] It should be understood that, after the resource management platform recovers any computing node in the first cluster, the first cluster can be shrunk, that is, the computing node no longer belongs to the first cluster.

[0196] For example, assuming that the monitoring period of the resource scheduling platform is 60 seconds and the preset threshold is 10 times, the resource scheduling platform can monitor whether there is a running service on any one computing node in the first cluster every 60 seconds. If the resource scheduling platform monitors 10 times in succession that there is no running service on the computing node, the resource scheduling platform can recycle the computing node from the first cluster and update the state of the computing node to an idle state.

[0197] It should be noted that S508 can be executed after S507, or S508 can be executed before S507, or S508 can be executed simultaneously with S507, and the embodiments of the present application do not limit this.

[0198] In the resource scheduling method provided in this embodiment, the resource scheduling platform can obtain a service processing request, analyze and process the service processing request, obtain the resource requirement amount in each service processing request, determine the total resource requirement amount according to the resource requirement amount in each service processing request, and generate a resource scheduling task according to the total resource requirement amount. If the total resource amount of the idle computing nodes in the cluster is less than the total resource requirement amount included in the resource scheduling task, the resource scheduling platform can allocate resources to the first cluster in batches. In this way, not only the overall resource utilization rate of the cluster is improved, but also the service processing speed of the first cluster is accelerated, so that the cluster has a high resource utilization rate and a high service processing speed.

[0199] Next, through a specific example, the technical solutions shown in any one of the above embodiments are described in detail.

[0200] Assuming that the resource scheduling platform obtains 10 service processing requests from the first cluster. The resource requirement amount included in each of the 10 service processing requests can be: CPU 50 cores and memory 100G. After the resource scheduling platform analyzes and processes the 10 service processing requests, the resource scheduling platform can determine that the total resource requirement amount of the 10 service processing requests is: CPU 500 cores and memory 1000G.

[0201] The entire cluster can include 10 computing nodes, and assuming that only computing node 1 is idle at present, and the total resource amount of computing node 1 is: CPU 300 cores and memory 624G. The resource scheduling platform can first allocate computing node 1 to the first cluster and generate shadow task 1. The resource requirement amount included in shadow task 1 can be: CPU 200 cores and memory 376G. At this time, the scheduler in the first cluster can schedule 6 service processing requests to run on computing node 1. In this way, not only the overall resource utilization rate of the computing nodes in the cluster is improved, but also the processing speed of the service processing requests in the first cluster is accelerated.

[0202] When a new idle computing node is added to the cluster, the resource scheduling platform can continue to allocate resources to the first cluster based on shadow task 1. Assume the newly added idle computing node is computing node 2, and its total resources are: 300 CPU cores and 500GB of memory. Since the total CPU resources of computing node 2 are greater than the CPU resources required by shadow task 1, and the total memory resources of computing node 2 are also greater than the CPU resources required by shadow task 1, the resource scheduling platform can allocate computing node 2 to the first cluster without generating a new shadow task.

[0203] It should be understood that after allocating compute node 1 and compute node 2 to the first cluster, the resource scheduling platform can update the status of compute node 1 and compute node 2 from idle to occupied. Furthermore, after any currently running service on any compute node in the first cluster finishes running and no other service is running on that compute node, the resource scheduling platform can reclaim that compute node from the first cluster and update its status from fully occupied to idle.

[0204] In the resource scheduling method described above, when the total resources in the cluster are insufficient to meet the demands of the current business processing requests, resources can be allocated to the first cluster in batches. This method improves the overall resource utilization of the cluster and accelerates the processing speed of business requests, resulting in higher resource utilization and faster business processing speed within the cluster.

[0205] In this application, users can also directly submit business processing requests to the resource scheduling platform. After receiving the business processing request submitted by the user, the resource scheduling platform can run the business processing request on a computing node with idle resources in the resource pool. For example, a computing node with idle resources can be an idle computing node, or a computing node with idle resources can be a partially occupied computing node.

[0206] In one possible implementation, after the resource scheduling platform runs the user-submitted business processing request on an idle computing node, if there are still remaining resources on the computing node, the resource scheduling platform can update the status of the computing node to a partially occupied state; if there are no remaining resources on the computing node, the resource scheduling platform can update the status of the computing node to a fully occupied state.

[0207] In another possible implementation, after the resource scheduling platform runs the user-submitted business processing request on a partially occupied computing node, if there are still remaining resources on the computing node, the resource scheduling platform will not update the status of the computing node; if there are no remaining resources on the computing node, the resource scheduling platform can update the status of the computing node to fully occupied.

[0208] It should be noted that the resource scheduling platform can continue to run other user directly submitted business processing requests on the partially occupied computing nodes.

[0209] Figure 7 A structural schematic diagram of a resource scheduling platform provided in the present application is shown in FIG. 1. Figure 7 The resource scheduling platform 10 includes a cluster adapter 11, a task manager 12, a scheduling engine 13, and a scale manager 14, wherein,

[0210] The cluster adapter 11 is configured to obtain at least one business processing request from the first cluster, and the business processing request includes resource demand amounts of multiple resource types.

[0211] The cluster adapter 11 is further configured to generate a resource scheduling task according to the at least one business processing request, and the resource scheduling task includes total resource demand amounts of the multiple resource types.

[0212] The task manager 12, the scheduling engine 13, and the scale manager 14 are configured to perform at least one resource allocation for the first cluster according to the resource scheduling task until the allocated resource amount for the first cluster is greater than or equal to the total resource demand amount.

[0213] The resource scheduling platform provided in the present embodiment can be used to execute the technical solutions shown in any of the method embodiments, and has similar principles and technical effects, which will not be repeated here.

[0214] Figure 8 Another structural schematic diagram of a resource scheduling platform provided in the present application is shown in FIG. 2. Figure 8 The resource scheduling platform 10 further includes an information collector 15, wherein,

[0215] The task manager 12 is further configured to generate a shadow task when the total resource amount is less than the total resource demand amount.

[0216] The scale manager 14 is further configured to recycle the computing nodes that do not run tasks from the first cluster.

[0217] The information collector 15 is configured to obtain resource usage information of each computing node of the first cluster.

[0218] In the present embodiment, the resource scheduling platform can be a software program or a device.

[0219] When the resource scheduling platform is a software program, the resource scheduling platform can be stored in the memory of a computing device and run through the processor of the computing device. The computing device can be a server, a computer, a mobile phone, or any computing device with processing capability. When the resource scheduling platform is a software program, the cluster adapter, the task manager, the scheduling engine, the capacity expansion and contraction manager, and the information collector can be components in the resource scheduling platform.

[0220] When the resource scheduling platform is a device, the resource scheduling platform can be a server, a computer, a mobile phone, or any computing device with processing capability. When the resource scheduling platform is a device, the cluster adapter, the task manager, the scheduling engine, the capacity expansion and contraction manager, and the information collector can be functional modules in the resource scheduling platform.

[0221] In the embodiments of the present application, the number of the first clusters can be one or more. When the number of the first clusters is more than one, the cluster adapter 11 can obtain service processing requests from each of the first clusters respectively, and generate resource scheduling tasks according to the obtained service processing requests.

[0222] For any one of the resource scheduling tasks, the task manager 12, the scheduling engine 13, and the capacity expansion and contraction manager 14 can allocate idle computing nodes to the first clusters according to the resource scheduling tasks.

[0223] Figure 9 A schematic diagram of recycling computing nodes is provided in the present application. Referring to FIG. 4, Figure 9 The information collector 15 can periodically collect resource usage information of each computing node in the first cluster. For any one of the computing nodes in the first cluster, if the information collector 15 continuously monitors that there is no running service on the computing node for a number of times greater than or equal to a preset threshold value, the information collector 15 can report the computing node to the capacity expansion and contraction manager 14. The capacity expansion and contraction processor 14 can trigger the first cluster to contract, so that the scheduling engine 13 can recycle the computing node.

[0224] The resource scheduling platform provided in the embodiments of the present application can be used to execute the technical solutions shown in any of the method embodiments described above, and has similar principles and technical effects, which will not be described here.

[0225] Figure 10 A resource scheduling system is provided in the embodiments of the present application. Referring to FIG. 5, Figure 10 The resource scheduling system 20 includes Figure 7 or Figure 8 the resource scheduling platform 10 and at least one first cluster as described above, wherein

[0226] The resource scheduling platform 10 is configured to execute the method shown in any of the method embodiments described above to perform at least one resource allocation for the first cluster.

[0227] The first cluster is configured to schedule the obtained resources to execute the at least one service processing request.

[0228] The resource scheduling system provided by the embodiment can be used to execute the technical solutions shown in any of the method embodiments, and has similar implementation principles and technical effects, which are not described here in detail.

[0229] Figure 11 A hardware structure diagram of a resource scheduling device provided by the embodiment is provided. Please refer to Figure 11 The resource scheduling device 30 includes a processor 31 and a memory 32, wherein the processor 31 and the memory 32 can communicate; for example, the processor 31 and the memory 32 communicate through a communication bus 33, the memory 32 is configured to store program instructions, and the processor 31 is configured to call the program instructions in the memory to execute the resource scheduling method shown in any of the above method embodiments.

[0230] Optionally, the resource scheduling device 30 can also include a communication interface, which can include a transmitter and / or a receiver.

[0231] Optionally, the processor can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the present application can be directly embodied as hardware processor execution, or executed by a combination of hardware and software modules in the processor.

[0232] The embodiment of the present application also provides a computing device, which includes a memory and a processor, wherein the processor is connected with the memory,

[0233] The processor is configured to generate a resource scheduling task according to at least one service processing request of the first cluster, the resource scheduling task including a total resource requirement amount required by services for running the at least one service processing request, and the total resource requirement amount being a sum of resource requirement amounts of each service processing request.

[0234] The processor is further configured to acquire a total resource amount of idle computing nodes, and when the total resource amount of idle computing nodes is less than the total resource demand amount, allocate the idle computing nodes corresponding to the total resource amount to the first cluster, and determine a to-be-allocated resource demand amount, which is the total resource demand amount minus the total resource amount.

[0235] The processor is further configured to allocate at least one new idle computing node to the first cluster according to the to-be-allocated resource demand amount, where the resource amount allocated to the first cluster is greater than or equal to the total resource demand amount.

[0236] In a possible implementation, the processor is further configured to generate a shadow task when the total resource amount is less than the total resource demand amount, and allocate at least one new idle computing node to the first cluster according to the shadow task.

[0237] The processor is further configured to acquire resource usage information of each computing node in the first cluster, and if it is determined according to the resource usage information that no service is running on a third computing node in the first cluster, recycle the third computing node from the first cluster, and switch the state of the third computing node to an idle state.

[0238] The computing device provided in the embodiment can be used to execute the technical solutions shown in any of the above method embodiments, and has similar implementation principles and technical effects, which will not be repeated here.

[0239] The embodiment of the present application further provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a computer to implement the resource scheduling method executed by any of the above method embodiments, and has similar implementation principles and technical effects, which will not be repeated here.

[0240] The embodiment of the present application further provides a computer program product, which includes a computer program, and the computer program is executed by a computer to implement the resource scheduling method executed by any of the above method embodiments, and has similar implementation principles and technical effects, which will not be repeated here.

[0241] All or part of the steps of the above method embodiments can be completed by program instruction related hardware. The foregoing program can be stored in a readable memory. When the program is executed, the steps of the above method embodiments are executed; and the foregoing memory (storage medium) includes: read-only memory (English: read-only memory, abbreviation: ROM), RAM, flash memory, hard disk, solid state disk, magnetic tape (English: magnetic tape), floppy disk (English: floppy disk), optical disc (English: optical disc), and any combination thereof.

[0242] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, as well as a combination of flows and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable processing unit to produce a machine, so that the instructions executed by the computer or other programmable processing unit generate a device implemented in the flowcharts Figure 1 one or more flows and / or blocks Figure 1 an apparatus that implements the functions specified in the flow(s) or block(s).

[0243] These computer program instructions can also be stored in a computer-readable memory that can direct the computer or other programmable processing unit to work in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including instruction apparatus, which implements the functions specified in the flowcharts Figure 1 one or more flows and / or blocks Figure 1 an apparatus that implements the functions specified in the flow(s) or block(s).

[0244] These computer program instructions can also be loaded into a computer or other programmable processing unit, so that a series of operation steps are performed on the computer or other programmable processing unit to produce a computer-implemented process, so that the instructions executed on the computer or other programmable processing unit provide a process for implementing the functions specified in the flowcharts Figure 1 one or more flows and / or blocks Figure 1 an apparatus that implements the functions specified in the flow(s) or block(s).

[0245] Obviously, those skilled in the art can make various modifications and variations to the embodiments of the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the embodiments of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

[0246] In the present application, the term "comprising" and its variants can refer to non-limiting inclusion; the term "or" and its variants can refer to "and / or". In the present application, the terms "first", "second", etc. are used to distinguish similar objects, and do not necessarily indicate a specific order or sequence. In the present application, "multiple" refers to two or more. "And / or", which describes the relationship between the associated objects, means that there can be three relationships, for example, A and / or B can represent: A alone, A and B together, and B alone. The character " / " generally represents an "or" relationship between the associated objects.

Claims

1. A resource scheduling method, characterized in that, The method comprises: obtaining at least one service processing request from a first cluster, the service processing request comprising a resource requirement amount required for running a service in the service processing request; generating a resource scheduling task according to the at least one service processing request, the resource scheduling task comprising a total resource requirement amount, the total resource requirement amount being a sum of the resource requirement amounts of each service processing request; obtaining a total resource amount of idle computing nodes, when the total resource amount of idle computing nodes is less than the total resource requirement amount, allocating the idle computing nodes corresponding to the total resource amount to the first cluster, and confirming a to-be-allocated resource requirement amount, the to-be-allocated resource requirement amount being the total resource requirement amount minus the total resource amount, the idle computing nodes being computing nodes in the first cluster; generating a shadow task according to the to-be-allocated resource requirement amount, the shadow task comprising the to-be-allocated resource requirement amount; allocating new idle computing nodes to the first cluster at least once according to the shadow task, wherein the resource amount allocated to the first cluster is greater than or equal to the total resource requirement amount.

2. The method of claim 1, wherein, The allocating new idle computing nodes to the first cluster at least once according to the shadow task comprises: determining an i-th to-be-allocated resource amount according to the total resource requirement amount and a total resource amount already allocated to the first cluster; generating an i-th shadow task according to the i-th to-be-allocated resource amount; performing i+1-th resource allocation to the first cluster according to the i-th shadow task; wherein the i is sequentially taken as 1, 2, …, until the resource amount allocated to the first cluster is greater than or equal to the total resource requirement amount.

3. The method according to any of claims 1-2, characterized in that, The generating a resource scheduling task according to the at least one service processing request comprises: performing parsing processing on the at least one service processing request to obtain the resource requirement amount in each service processing request; determining the total resource requirement amount according to the resource requirement amount in each service processing request; generating the resource scheduling task according to the total resource requirement amount.

4. The method according to any one of claims 1-2, characterized in that, After allocating idle computing nodes to the first cluster, the method further comprises: determining a second computing node allocated to the first cluster; switching a state of the second computing node from an idle state to an occupied state.

5. The method according to any of claims 1-2, characterized by, The method further comprises: obtaining resource usage information of each computing node of the first cluster; if it is determined according to the resource usage information that no service is running on a third computing node in the first cluster, recycling the third computing node from the first cluster, and switching a state of the third computing node to an idle state.

6. A computing device, comprising: The computing device comprises a memory and a processor, the processor being connected with the memory, the processor is configured to generate a resource scheduling task according to at least one service processing request of a first cluster, the resource scheduling task comprising a total resource requirement amount required for running a service of the at least one service processing request, the total resource requirement amount being a sum of resource requirement amounts of each service processing request. The processor is further configured to acquire a total resource amount of idle computing nodes, allocate the idle computing nodes corresponding to the total resource amount to the first cluster when the total resource amount of the idle computing nodes is less than the total resource requirement, and confirm a to-be-allocated resource requirement, which is the total resource requirement minus the total resource amount. The processor is further configured to generate a shadow task according to the to-be-allocated resource requirement, the shadow task including the to-be-allocated resource requirement, and allocate at least one new idle computing node to the first cluster according to the shadow task, wherein the resource amount allocated to the first cluster is greater than or equal to the total resource requirement.

7. The computing device of claim 6, wherein, The processor is further configured to acquire resource usage information of each computing node of the first cluster, and recycle a third computing node in the first cluster and switch a state of the third computing node to an idle state if it is determined that no service is running on the third computing node according to the resource usage information.

8. A resource scheduling system, characterized by, The computing device and the at least one first cluster of claim 6 or 7, wherein, The computing device is configured to perform the method of any one of claims 1-5 to allocate idle computing nodes to the first cluster and allocate at least one new idle computing node to the first cluster; The first cluster is configured to schedule the allocated idle computing nodes to execute the at least one service processing request.

9. A resource scheduling device, characterized by, The computing device comprises: a processor and a memory connected to the processor in communication; The memory stores a computer program; The processor executes the computer program to implement the method of any one of claims 1-5.

Images