Resource scheduling method, device and equipment and computer readable storage medium

Abstract

The invention discloses a resource scheduling method, device and equipment and a computer readable storage medium, and belongs to the technical field of resource scheduling. The method comprises the steps that a scheduler receives a first job and determines a cache block corresponding to a first resource according to the first resource required by the first job and the incidence relation between the available resource and the cache block, and the cache block comprises a node set, the available resource and the incidence relation between the available resource and the node set; under the condition that the first resource corresponds to the first cache block, the scheduler determines a first candidate node set in the first cache block, nodes in the first candidate node set are schedulable nodes, and available resources are all larger than or equal to the first resource. The scheduler schedules resources of the schedulable node for use by the first job. Therefore, the scheduler takes the first resource as an index to search the incidence relation between the resource and the cache block, the candidate node set meeting the first resource can be quickly searched, each node does not need to be traversed to search the node matched with the first resource, the calculation amount is reduced, and the scheduling efficiency is improved.

Description

Technical Field

[0001] This application relates to the technical field of resource scheduling, and particularly to a resource scheduling method, apparatus, device, and computer-readable storage medium. Background Art

[0002] A distributed computing cluster refers to a situation where multiple computers (or nodes) can be located in different places, and these computers can be centralized to implement the same service. The scheduler is the brain of the distributed computing cluster and is used to match resources for a job in the computing cluster according to the job submitted by the user. Therefore, the resource allocation and scheduling efficiency of the scheduler play a key role in the resource utilization rate and throughput of the computing cluster.

[0003] In a related technology, when the scheduler schedules a specified job, in order to check whether the available resources of each node meet the job requirements of the specified job, the scheduler needs to sequentially traverse the resources of each node in the computing cluster, resulting in a large amount of computation and low scheduling efficiency. Summary of the Invention

[0004] This application provides a resource scheduling method, apparatus, device, and computer-readable storage medium to solve the problems provided by the related technology. The technical solutions are as follows:

[0005] In a first aspect, a resource scheduling method is provided. The execution subject of the method can be a scheduler. The method includes: receiving a first job, where the first job includes first resources required by the first job; determining a cache block corresponding to the first resources according to the first resources and the association relationship between available resources and cache blocks, where the cache block includes a node set, available resources, and the association relationship between the available resources and the node set; in the case where the first resources correspond to a first cache block, determining a first candidate node set in the first cache block, where the available resources of each candidate node in the first candidate node set are greater than or equal to the first resources, and the nodes in the first candidate node set are schedulable nodes; scheduling the resources of the schedulable nodes for the first job to use. In this application, by using the first resources as an index to find the association relationship between available resources and cache blocks, through the association relationship, a candidate node set that meets the first resources can be quickly found, without the need to find the nodes that match the first resources by traversing each node multiple times, reducing the amount of computation and improving the scheduling efficiency.

[0006] In an alternative manner, the method further includes: when the first resource does not correspond to any cache block, sequentially traversing the available resources of each node in each available cache block, where the available cache blocks include nodes whose available resources are greater than or equal to the first resource; determining a first node set in the available cache blocks, where the available resources of each node in the first node set are greater than or equal to the first resource; when the available resources of each designated node in the first node set in the initialization node set are greater than or equal to the first resource, determining each designated node as a schedulable node; the initialization node set is obtained by clustering each node in the resource pool according to the first job.

[0007] In this application, by filtering out the available cache blocks and traversing the available resources of each node in the available cache blocks, the traversal of unavailable cache blocks is saved. In addition, a first node set with available resources greater than or equal to the first resource is determined in the available cache blocks, and then the resources of the nodes in the first node set are further judged in the initialization node set, so as to find the nodes that meet the first resource, improving the scheduling efficiency.

[0008] In an alternative manner, the method further includes: determining a second node set in the available cache blocks, where the available resources of each node in the second node set are less than the first resource; batch filtering the second node set, and the nodes in the second node set are unschedulable nodes. In this application, by filtering out the available cache blocks and traversing the available resources of each node in the available cache blocks, the traversal of unavailable cache blocks is saved. In addition, the node set with available resources less than the first resource is batch filtered in the available cache blocks, so that these nodes do not need to be considered during resource scheduling, improving the scheduling efficiency.

[0009] In an alternative manner, the cache block further stores the association relationship between the node set and the job set. After batch filtering the second node set, the method further includes: determining a second cache block in the available cache blocks, where the second cache block includes the nodes other than each designated node in the first node set and / or the nodes in the second node set; when scheduling resources for a second job, batch skipping the resource scheduling of each node in the second cache block to schedule the resources of the nodes in the cache blocks other than the second cache block, and the second job and the first job belong to the same job set.

[0010] In this application, by determining the second cache block in the available cache blocks, and the available resources of each node in the second cache block are less than the first resource, it means that each node in the second cache block is an unschedulable node. When scheduling resources for the second job, the scheduling of the second cache block can be batch skipped, reducing system overhead and improving scheduling efficiency.

[0011] In an alternative manner, before sequentially traversing the available resources of each node in each available cache block, the method further includes: when the first resource does not correspond to any cache block, determining a third cache block corresponding to a second resource, where the second resource is smaller than the first resource and the sizes of the second resource and the first resource are adjacent; and when the third cache block includes a node whose available resource is greater than or equal to the first resource, determining the third cache block as the available cache block.

[0012] In this application, by finding the cache block (i.e., the third cache block) corresponding to the second resource that is smaller than the first resource and closest to the first resource, the node set of the second resource closest to the first resource is found. The node set is necessarily a subset of the node set of the first resource. Therefore, it is only necessary to determine the available cache block in the cache block corresponding to the second resource, further reducing the calculation amount and improving the scheduling efficiency.

[0013] In an alternative manner, the method further includes: determining a fourth cache block corresponding to a third resource, where the third resource is greater than or equal to the second resource; and determining the fourth cache block as the available cache block.

[0014] In this application, through the second resource, the cache block (i.e., the fourth cache block) corresponding to the third resource greater than the second resource is determined, filtering out the cache blocks smaller than the second resource. It is only necessary to determine the available cache block in the fourth cache block corresponding to the third resource, further reducing the calculation amount and improving the scheduling efficiency.

[0015] In an alternative manner, the determining the first candidate node set in the first cache block includes: taking out a second candidate node set from the first cache block, where the available resources of each candidate node in the second candidate node set are greater than or equal to the first resource; and when it is determined that the available resources of each candidate node in the second candidate node set in the initialization node set are greater than or equal to the first resource, adding each candidate node to the first candidate node set, where the initialization node set is obtained by clustering each node in the resource pool according to the first job.

[0016] Take out the second candidate node set from the first cache block. The available resources of each node in the second candidate node set are greater than or equal to the first resource. At the same time, it is satisfied that the available resources of each node in the second candidate node set in the initialization node set are also greater than or equal to the first resource. Only then can the second candidate node set be determined as the first candidate node set.

[0017] In an alternative manner, the association relationship between the available resources stored in the cache block and the node set is represented by a bitmap, and the bitmap includes a first value and a second value, where the first value indicates that the available resources are less than the first resource, and the second value indicates that the available resources are greater than or equal to the first resource; specifically, adding the candidate nodes to the first candidate node set is as follows: determining the second bitmap of the second candidate node set according to the first bitmap stored in the first cache block; determining the third bitmap corresponding to the second candidate node set in the initialization node set according to the identifiers of the nodes in the second candidate node set; performing a logical AND operation on the second bitmap and the third bitmap to obtain the nodes representing the second value, and adding them to the first candidate node set.

[0018] In this application, representing the association relationship between the available resources and the node set by a bitmap can effectively reduce memory occupancy; in addition, using two bitmaps for logical AND operation greatly reduces the amount of calculation and improves the scheduling efficiency.

[0019] In some design solutions, the method further includes: when the first resource corresponds to the first cache block, batch-updating the association relationship between the available resources stored in the first cache block and the node set.

[0020] In this application, after finding the first cache block corresponding to the first resource, it means that the resources of each node in the first cache block are called. At this time, updating the association relationship in the first cache block enables the association relationship to be updated in real time, improving the accuracy of the available resources of each node.

[0021] In an alternative manner, the association relationship between the available resources and the cache block is represented by a node view. Before determining the cache block corresponding to the first resource, it further includes: clustering the nodes according to the available resources of the nodes in the resource pool to obtain each node set, and each node set is included in each cache block; creating the node view according to the available resources, each cache block, the association relationship between the available resources and each cache block.

[0022] In this application, clustering the nodes according to the available resources of each node facilitates subsequent utilization of the association between nodes, enables fast searching for callable nodes and fast batch filtering of nodes, thereby achieving efficient resource scheduling.

[0023] In an alternative manner, the method further includes: clustering the jobs according to the required resources of the jobs to obtain each job set; specifically, creating the node view is as follows: creating the node view according to the available resources, each cache block, the association relationship between the available resources and each cache block, the required resources, and the association relationship between each job set and each cache block.

[0024] In this application, by clustering each job according to the same resource requirements, it is convenient to utilize the association relationships between jobs and between nodes and jobs subsequently. When performing resource scheduling for other jobs with the same resource requirements, it is possible to batch skip the filtered nodes using the existing node view, thereby achieving efficient resource scheduling.

[0025] In an alternative manner, the creation of the node view is specifically as follows: According to the available resources, each cache block, the association relationship between the available resources and each cache block, and the restriction information, the node view is created; the restriction information is used to restrict the depth and / or width of the node view.

[0026] In an alternative manner, the creation of the node view is specifically as follows: The arrangement levels of the available resources in the node view satisfy a first threshold, and the number of nodes in each level of cache blocks of the node view satisfies a second threshold. According to the available resources, each cache block, the association relationship between the available resources and each cache block, the first threshold, and the second threshold, the node view is created. In this application, by restricting the node view, the memory occupation by the node view is effectively prevented, and memory is saved.

[0027] In an alternative manner, the node view includes a multi-way search tree or a binary tree.

[0028] In a second aspect, a resource scheduling device is provided. The device includes: a receiving module, a determining module, and a scheduling module. Among them, the receiving module is used to receive a first job, and the first job includes a first resource required by the first job. The determining module is used to determine the cache block corresponding to the first resource according to the first resource and the association relationship between the available resources and the cache blocks. The cache block includes a node set, available resources, and the association relationship between the available resources and the node set; when the first resource corresponds to a first cache block, a first candidate node set in the first cache block is determined. The available resources of each candidate node in the first candidate node set are greater than or equal to the first resource, and the nodes in the first candidate node set are schedulable nodes. The scheduling module is used to schedule the resources of the schedulable nodes for the first job to use.

[0029] In an alternative manner, the device further includes: a traversal module. The traversal module is configured to, when the first resource does not correspond to any cache block, sequentially traverse the available resources of each node in each available cache block, where the available cache blocks include nodes whose available resources are greater than or equal to the first resource. The determination module is configured to determine a first node set in each of the available cache blocks, where the available resources of each node in the first node set are greater than or equal to the first resource; and when the available resources of each specified node in the first node set in the initialization node set are greater than or equal to the first resource, determine each of the specified nodes as schedulable nodes; the initialization node set is obtained by clustering each node in the resource pool according to the first job.

[0030] In an alternative manner, the device further includes: a filtering module. The determination module is configured to determine a second node set in each of the available cache blocks, where the available resources of each node in the second node set are less than the first resource. The filtering module is configured to batch filter the second node set, and the nodes in the second node set are unschedulable nodes.

[0031] In an alternative manner, the cache block further stores the association relationship between the node set and the job set. The determination module is configured to determine a second cache block in each of the available cache blocks, where the second cache block includes the nodes other than each specified node in the first node set and / or the nodes in the second node set. The scheduling module is configured to, when scheduling resources for a second job, batch skip the resource scheduling of each node in the second cache block to schedule the resources of the nodes in the cache blocks other than the second cache block, and the second job and the first job belong to the same job set.

[0032] In an alternative manner, the determination module is configured to: when the first resource does not correspond to any cache block, determine a third cache block corresponding to a second resource, where the second resource is less than the first resource and the magnitudes of the second resource and the first resource are adjacent; and when the third cache block includes nodes whose available resources are greater than or equal to the first resource, determine the third cache block as the available cache block.

[0033] In an alternative manner, the determination module is configured to: determine a fourth cache block corresponding to a third resource, where the third resource is greater than or equal to the second resource; and determine the fourth cache block as the available cache block.

[0034] In an alternative manner, the determining module is configured to: retrieve a second candidate node set from the first cache block, where the available resources of each candidate node in the second candidate node set are greater than or equal to the first resource; and add each candidate node to the first candidate node set when it is determined that the available resources of each candidate node in the second candidate node set in the initialization node set are greater than or equal to the first resource, where the initialization node set is obtained by clustering each node in the resource pool according to the first job.

[0035] In an alternative manner, the association relationship between the available resources stored in the cache block and the node set is represented by a bitmap, where the bitmap includes a first value and a second value. The first value indicates that the available resources are less than the first resource, and the second value indicates that the available resources are greater than or equal to the first resource. The determining module is configured to: determine a second bitmap of the second candidate node set according to the first bitmap stored in the first cache block; determine a third bitmap corresponding to the second candidate node set in the initialization node set according to the identifiers of the nodes in the second candidate node set; perform a logical AND operation on the second bitmap and the third bitmap to obtain the nodes representing the second value, and add them to the first candidate node set.

[0036] In an alternative manner, the apparatus further includes an updating module, where the updating module is configured to batch update the association relationship between the available resources stored in the first cache block and the node set when the first resource corresponds to the first cache block.

[0037] In an alternative manner, the association relationship between the available resources and the cache block is represented by a node view, and the apparatus further includes a clustering module and a creating module. The clustering module is configured to cluster each node according to the available resources of each node in the resource pool to obtain each node set, and each node set is included in each cache block. The creating module is configured to create the node view according to the available resources, each cache block, and the association relationship between the available resources and each cache block.

[0038] In an alternative manner, the clustering module is configured to cluster each job according to the required resources of the job to obtain each job set. The creating module is configured to: create the node view according to the available resources, each cache block, the association relationship between the available resources and each cache block, the required resources, and the association relationship between each job set and each cache block.

[0039] In an alternative manner, the creating module is configured to: create the node view according to the available resources, each cache block, the association relationship between the available resources and each cache block, and the restriction information; the restriction information is used to restrict the depth and / or width of the node view.

[0040] In an alternative manner, the creation module is configured to: the arrangement levels of the available resources in the node view satisfy a first threshold, and the number of nodes in each cache block at each level of the node view satisfies a second threshold. According to the available resources, the cache blocks, the association relationship between the available resources and the cache blocks, the first threshold, and the second threshold, create the node view.

[0041] In an alternative manner, the node view includes a multi-way search tree or a binary tree.

[0042] In a third aspect, an electronic device is provided, the device includes a memory and a processor; at least one instruction is stored in the memory, and the at least one instruction is loaded and executed by the processor so that the electronic device implements the methods in the above aspects.

[0043] In a fourth aspect, a computer-readable storage medium is provided, in which at least one instruction is stored, and the instruction is loaded and executed by a processor to implement the methods in the above aspects.

[0044] In a fifth aspect, a computer program (product) is provided, the computer program (product) includes: computer program code, when the computer program code is run by a computer, the computer is caused to execute the methods in the above aspects.

[0045] In a sixth aspect, a chip is provided, including a processor, configured to call and run the instruction stored in the memory from the memory, so that a communication device installed with the chip executes the methods in the above aspects.

[0046] In a seventh aspect, another chip is provided, including: an input interface, an output interface, a processor, and a memory, the input interface, the output interface, the processor, and the memory are connected through an internal connection path, and the processor is configured to execute the code in the memory, and when the code is executed, the processor is configured to execute the methods in the above aspects.

[0047] It should be understood that the beneficial effects obtained by the technical solutions of the second to seventh aspects of the present application and the corresponding possible implementation manners can refer to the technical effects of the first aspect and its corresponding possible implementation manners above, and will not be elaborated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 A schematic diagram of the scheduling process of the resource scheduling method provided for the related art;

[0049] Figure 2 A flowchart of a resource scheduling method provided by an embodiment of the present application;

[0050] Figure 3 It is a schematic flowchart of another resource scheduling method provided by an embodiment of the present application;

[0051] Figure 4 It is a schematic structural diagram of a B+ tree provided by an embodiment of the present application;

[0052] Figure 5 It is a schematic flowchart of yet another resource scheduling method provided by an embodiment of the present application;

[0053] Figure 6 It is a schematic flowchart of still another resource scheduling method provided by an embodiment of the present application;

[0054] Figure 7 It is an application schematic diagram of the B+ tree in an application scenario provided by an embodiment of the present application;

[0055] Figure 8 It is an application schematic diagram of the B+ tree in another application scenario provided by an embodiment of the present application;

[0056] Figure 9 An embodiment of the present application also provides a schematic structural diagram of a resource scheduling device. Detailed implementation manners

[0057] The terms used in the implementation manners part of the present application are only used to explain the specific embodiments of the present application, rather than to limit the present application.

[0058] The scheduler is the brain of a distributed computing cluster and is used to match resources for a job in the computing cluster according to the job submitted by a user. Therefore, the resource allocation and scheduling efficiency of the scheduler play a key role in the resource utilization rate and throughput of the computing cluster. Exemplarily, Figure 1 The figure shows a schematic diagram of the resource scheduling method of the scheduler provided by the related art. As Figure 1 shown, in the cloud computing business scenario, taking the scheduler as an example of an open-source container orchestration platform (Kubernetes), the computing cluster may include Node 1, Node 2, Node 3,..., Node n, where n is a positive integer greater than 3. The available resources of Node 1 may include 20-core central processing unit (CPU) and 128G of memory; the available resources of Node 2 may include 64-core CPU and 256G of memory; the available resources of Node 3 may include 48-core CPU and 32G of memory;... the available resources of Node n may include 128-core CPU and 512G of memory.

[0059] The scheduler receives Job 1, and the resources required by Job 1 are 32-core CPUs and 64GB of memory. When the scheduler selects Job 1 for scheduling, the scheduler sequentially traverses all nodes in the computing cluster and verifies the resources required by Job 1 against the available resources of each node. As Figure 1 shown, the scheduler needs to sequentially traverse each node to find a node that meets the requirement of 32-core CPUs, and then sequentially traverse each node to find a node that meets the requirement of 64GB. It can be seen that within each scheduling cycle, the scheduler needs to perform "the number of nodes * the number of active jobs * the number of resource dimensions" resource verifications, that is, the scheduler needs to perform n * 1 * 2 resource verifications. Finally, the scheduler can determine that the resources required by Job 1 match the available resources of Node 2 and Node n. This means that the scheduler can allocate the resources of Node 2 or Node n for Job 1.

[0060] It can be seen that when the scheduler schedules Job 1, in order to check whether the available resources of each node meet the job requirements of Job 1, the scheduler needs to sequentially traverse the resources of each node in the computing cluster, and each scheduling cycle requires "the number of nodes * the number of active jobs * the number of resource dimensions" resource verifications, resulting in a large amount of computation and low scheduling efficiency.

[0061] To solve the above technical problems, an embodiment of the present application provides a resource scheduling method. Exemplarily, the execution subject of the method can be a scheduler. The method may include: The scheduler receives a first job, and the first job includes first resources required by the first job. The scheduler determines a cache block corresponding to the first resources according to the first resources and the association relationship between the available resources and the cache block. The cache block includes a node set, available resources, and the association relationship between the available resources and the node set. When the first resources correspond to a first cache block, the scheduler determines a first candidate node set in the first cache block, and the available resources of each candidate node in the first candidate node set are greater than or equal to the first resources. The nodes in the first candidate node set are schedulable nodes. The scheduler schedules the resources of the schedulable nodes for the first job to use. In this way, the scheduler uses the first resources as an index to find the association relationship between the resources and the cache block. Through the association relationship, it can quickly find the candidate node set that meets the first resources, without having to traverse each node multiple times to find the nodes that match the first resources, reducing the amount of computation and improving the scheduling efficiency.

[0062] Among them, the association relationship between the available resources and the cache block can be represented by a node view. In the embodiment of the present application, the association relationship between the available resources and the cache block is represented by a node view.

[0063] The above scheduler may include, but is not limited to: the commercial scheduler (load sharing facility, LSF) in the high performance computing (HPC) scenario, the commercial scheduler (simple linux utility for resource management, SLURM) in the HPC scenario, the commercial scheduler (yet another resource negotiator, YARN) in the big data scenario, and Kubernetes. No specific limitation is made in the embodiments of this application.

[0064] The resource scheduling method provided in this application can be applied to distributed cluster computing scenarios, such as HPC scenarios, artificial intelligence (AI) scenarios, big data scenarios, cloud computing scheduling system scenarios, and so on. Exemplarily, the resource scheduling method can be applied to network scheduling scenarios, computing power network element scheduling scenarios, Linux process scheduling, etc. No specific limitation is made in the embodiments of this application.

[0065] In some embodiments, Figure 2 It is a schematic flowchart of a resource scheduling method provided in an embodiment of this application. In Figure 2 a system architecture in the above distributed cluster computing scenario is shown. As Figure 2 shown, the system architecture may include a management node and computing nodes. Among them, the management node is used to schedule the resources of each node in the computing cluster. The computing nodes can be understood as the nodes in the computing cluster.

[0066] A scheduler is configured on the management node. The scheduler may include a resource management interface, a resource management module, a job submission interface, and a job scheduling module. Among them, the resource management interface is used to receive the configuration information submitted by the administrator for configuring the cluster resources. The resource management module is used to configure the cluster resource pool according to the configuration information. In addition, the resource management module is further used to create a node view according to the node information of each node in the cluster resource pool and the required resources of the job, and synchronize the node view to the job scheduling module. The job submission interface is used to receive the jobs submitted by users. The job scheduling module is used to determine the schedulable resources for the job according to the node view. After the resources are determined, the job is sent to the corresponding computing node for running.

[0067] A scheduler may also be configured on the computing node, which works in cooperation with the scheduler on the management node. Among them, the scheduler on the computing node may include a resource monitoring module and a job control module. Among them, the resource monitoring module is used to collect node information from the operating system of the computing node, process the node information, and then report the processed node information to the resource management module. The job control module is used to receive the jobs sent by the job scheduling module and send the jobs to the operating system of the computing node. The operating system of the computing node runs the jobs.

[0068] Based on the above system architecture, a resource scheduling method provided by an embodiment of the present application is described in detail in the following two stages, specifically:

[0069] The first stage is the creation stage of the node view.

[0070] Figure 3 A flowchart of a resource scheduling method provided by an embodiment of the present application is shown in Figure 3 As shown, the method may include S301-S304 (some of the steps are optional).

[0071] S301. The scheduler receives the configuration information submitted by the administrator, and the configuration information is used to configure the cluster resource pool.

[0072] The scheduler can be understood as the scheduler on the management node side. The scheduler is used to schedule the resources of the nodes on the computing node side for a specified job to use. Among them, there are multiple nodes on the computing node side, and these nodes can form multiple node sets. And the configuration information can be used to configure the multiple nodes on the computing node side into a cluster resource pool.

[0073] In one example, the configuration information may include at least one of the basic information of the computer, the physical information of the computer, and the logical information of the computer. In one example, the configuration information may further include the following limitation conditions of the node view, and the limitation conditions can refer to the relevant description below and will not be elaborated here. In one example, the configuration information may include the value range of the required resources of the job, for example, CPU_RANGE = 1-256.

[0074] In one example, the basic information may include an identifier and a name. The physical information may include the rack model, the cabinet model, and the server model. The logical information may include the resource amount of the computer and the department to which the computer belongs. Among them, the resources may include the number of CPU cores, the memory occupancy, and the number of graphics processing unit (GPU) cards, etc. Of course, the embodiments of the present application are not limited to the above enumeration.

[0075] Exemplarily, the configuration information may include basic information of the computer, and the basic information may include the name of the computer. The administrator can submit the names of the computers, and subsequently, the scheduler can obtain other information of each computer according to the names of the computers, such as the physical information and logical information of the computer.

[0076] In one implementable manner, after the computer cluster is started, the administrator can submit the configuration information through a configuration item file provided by the scheduler, an application programming interface (API), a command-line tool, or a visual operation interface.

[0077] Exemplarily, Figure 2 FIG. is a schematic diagram of a creation process for creating a node view based on the above system architecture. As Figure 2 shown, S301 can be implemented as follows: Step ① The administrator submits the configuration information to the resource management through the resource management interface of the scheduler. Correspondingly, the scheduler can receive the configuration information submitted by the administrator through the resource management interface.

[0078] S302. The scheduler configures the cluster resource pool according to the configuration information and in accordance with the first preset rule.

[0079] Among them, the first preset rule can be defined manually or preset in advance. Exemplarily, the first preset rule may include any one of the following: Computers belonging to the same department are configured in one resource pool; or, computers of the same model are configured in one resource pool; or, computers with the same resources are configured in one resource pool.

[0080] Among them, the same model can be understood as: the same rack model, the same cabinet model, or the same server model. The same resources can be understood as: the same total resources or the same available resources.

[0081] Of course, the first preset rule may also include other rules, and the other rules may also be a combination of several rules, such as: Computers belonging to the same department and of the same model are configured in one resource pool; or, computers of the same model and with the same resources are configured in one resource pool. These are not listed one by one in the embodiments of the present application.

[0082] Continuing with the above example, the scheduler can obtain other information of each computer according to the names of the computers, such as the resources of the computer. Then, the scheduler configures the computers with the same resources in one resource pool.

[0083] In one implementable manner, as Figure 2As shown in the figure, S302 can be implemented as follows: In step ②, the resource management module of the scheduler receives the configuration information received by the resource management interface, and configures the cluster resource pool according to the configuration information and the first preset rule.

[0084] S303. The scheduler receives the node information reported by each node in the cluster resource pool. The node information includes the total resources, resource usage, and resource utilization rate of the node.

[0085] Among them, the resource utilization rate of each node can be calculated by each node according to the total resources and resource usage of the node. Each node in the cluster resource pool can periodically collect the original information of each node, and the original information records the total resources and resource usage of the node.

[0086] In one implementation, a corresponding scheduler is also configured on each node. As Figure 2 shown in the figure, S303 can be implemented as follows: In step ③, the resource monitoring module of the scheduler of each node periodically collects the total resources and resource usage of the node from the operating system on each node. And, the resource monitoring module calculates the resource utilization rate according to the total resources and resource usage of the node. After that, in step ④, the resource monitoring module reports the total resources, resource usage, and resource utilization rate of each node to the resource management module of the scheduler on the management node side. Correspondingly, the scheduler receives the total resources, resource usage, and resource utilization rate of each node.

[0087] S304. The scheduler creates a node view according to the node information of each node and the required resources of each job scheduled to each node. The node view is used to represent the association relationship between available resources and cache blocks, and the cache blocks store the association relationship between available resources and node sets.

[0088] Among them, the resource requirements of each job scheduled to each node can be understood as: the required resources of each job on the specified node. Available resources can be understood as resources that can be used. Resources that can still be used on the node. Since resources can have multiple properties. For example, resources representing quantity, such as memory; for another example, resources representing attributes, such as the number of CPU cores, the number of GPU cards, etc. Therefore, for resources of different properties, the determination of the available resources of the node can include the following situations, specifically:

[0089] In one situation, the available resources of the node can refer to the resources representing quantity of the node, and the available resources can be determined according to the total resources of the node and the resource utilization rate of the node.

[0090] Exemplarily, if the available resource of a node is the available memory of the node, S304 can be implemented as follows: The scheduler can calculate the available memory of each node based on the total memory and memory utilization rate of each node. The scheduler establishes the association relationship between each node and each job, and the association relationship between each node and the available memory according to the available memory and the required memory of each job scheduled to each node, so as to establish a node view.

[0091] In one implementable manner, as Figure 2 shown, S304 can be: Step ⑥ The resource management module of the scheduler calculates the available resources of the node according to the total resources of the node and the job requirements of the node. The resource management module of the scheduler creates a node view according to the available resources of each node and the job requirements of each node. Moreover, the resource management module synchronizes the node view to the job scheduling module.

[0092] In another case, the available resource of a node can also refer to the resource used to characterize the attributes of the node. For example, if the resource is the number of CPU cores, the available resource of the node is the number of CPU cores of the node.

[0093] Exemplarily, if the available resource of a node is the number of CPU cores of the node, S304 can be implemented as follows: The scheduler can establish the association relationship between each node and each job, and the association relationship between each node and the available memory according to the number of CPU cores of each node and the number of CPU cores required by each job scheduled to each node, so as to establish a node view.

[0094] In some embodiments, the node view may include a multi-way search tree or a binary tree. Exemplarily, taking the node view as a multi-way search tree, that is, a B+ tree, as an example. The above available resource can be the index of the B+ tree, or the keyword of the tree node of the B+ tree. The tree node of the B+ tree is composed of an available resource and a cache block.

[0095] In the embodiments of the present application, one-dimensional resource corresponds to one B+ tree. Exemplarily, the number of CPU cores corresponds to one B+ tree, the occupied memory corresponds to one B+ tree, and the number of GPU cards corresponds to one B+ tree. In subsequent applications, when the scheduler finds the corresponding candidate node on the B+ tree corresponding to each dimension resource, the candidate nodes can be logically ANDed according to different dimension resources to determine whether the candidate node is a schedulable node. If the candidate node is a schedulable node, the scheduler adds the node to the set of schedulable nodes. In this way, the scheduler can schedule the resources of the nodes in the set of schedulable nodes for the jobs to use.

[0096] Of course, in some other embodiments, multiple dimension resources can also correspond to one B+ tree. Exemplarily, the number of CPU cores and the occupied memory jointly correspond to one B+ tree.

[0097] In one implementable manner, S304 may include S3041 - S3042.

[0098] S3041. The scheduler clusters each node according to the available resources of each node in the resource pool to obtain each node set, and each node set is included in each cache block.

[0099] Exemplarily, the cluster resource pool may include Node 1 - Node 162. Among them, the available resources of each node may be as follows:

[0100] Example 1, Node 1 - Node 9: The available resource of Node 1 is a 4-core CPU, the available resources of Node 2 - Node 3 are both 1-core CPUs, the available resource of Node 4 is an 8-core CPU, the available resources of Node 5 - Node 6 are 1-core, the available resources of Node 7 - Node 8 are 8-core CPUs, and the available resource of Node 9 is a 1-core CPU.

[0101] Example 2, Node 55 - Node 63: The available resources of Node 55 - Node 57 are both 20-core CPUs, the available resources of Node 58 - Node 59 are both 28-core CPUs, the available resources of Node 60 - Node 61 are both 20-core CPUs, and the available resources of Node 62 - Node 63 are both 28-core CPUs.

[0102] Example 3, Node 64 - Node 72: The available resources of Node 64 - Node 67 are both 28-core CPUs, the available resources of Node 68 - Node 70 are both 42-core CPUs, the available resource of Node 71 is 28-core, and the available resource of Node 72 is 32-core.

[0103] The available resources of Node 10 - Node 54 and Node 73 - Node 162 are not listed one by one.

[0104] Then, the available resources may include 1-core CPU, 4-core CPU, 8-core CPU, 12-core CPU, 14-core CPU, 16-core CPU, 20-core CPU, 28-core CPU, 32-core CPU, 36-core CPU, 42-core CPU, 48-core CPU, 52-core CPU, 58-core CPU, 64-core CPU, 76-core CPU, 85-core CPU, and 96-core CPU.

[0105] Then, the scheduler may cluster some of the nodes 1 - 162 according to the available resources. Exemplarily, the scheduler may cluster the above nodes 1 - 9 according to 1-core CPU to obtain cache block 1. Similarly, the scheduler may cluster the above nodes 55 - 63 according to 20-core CPU to obtain cache block 2, and the node set that meets 4-core CUP is aggregated in cache block 2.... The scheduler may cluster the above nodes 64 - 72 according to 28-core CPU to obtain cache block 3.

[0106] In the embodiments of the present application, the scheduler may cluster each node according to the available resources of each node, which is convenient for subsequent utilization of the correlation between nodes to achieve rapid search for callable nodes and rapid batch filtering of nodes, thereby realizing efficient resource scheduling.

[0107] In some embodiments, the administrator / user may submit commands for displaying node clustering information and job clustering information through means such as a configuration item file, API, command line tool, or visual operation interface provided by the scheduler. The scheduler may display the node clustering information and job clustering information according to the commands, enabling the administrator / user to intuitively view the situation of each node and each job, improving usability and the functional completeness of the scheduler.

[0108] In some embodiments, since the available resources of not all nodes in each node set are the resources corresponding to the node set, the scheduler establishes the association relationship between each node in each node set and the available resources. The association relationship includes two cases. The first case is that the available resources of the node are the resources corresponding to the node set to which the node belongs; the second case is that the available resources of the node are not the resources corresponding to the node set to which the node belongs.

[0109] In an example, the scheduler may represent the association relationship between each node in each node set and the available resources in the form of a bitmap. The association relationship may be represented by a bitmap, and the bitmap may include 0 and 1. Among them, 0 indicates that the available resources of the node do not meet 1-core CPU, and 1 indicates that the available resources of the node meet 1-core CPU.

[0110] Therefore, the scheduler may obtain the association relationship between each of the above node sets and their corresponding available resources. Exemplarily:

[0111] Among nodes 1 - 9, the available resources of nodes 1, 4, 7 - 8 are occupied. Then, the association relationship between 1-core CPU and nodes 1 - 9 can be represented by the bitmap as 011011001.

[0112] Among nodes 55 - 63, the available resources of nodes 58 - 59 and nodes 62 - 63 are occupied. Then, the association relationship between 20-core CPU and nodes 55 - 63 can be represented by the bitmap as 111001100.

[0113] Among nodes 64 - 72, the available resources of nodes 68 - 70 and node 72 are occupied. Then, the association relationship between 28-core CPU and nodes 64 - 72 can be represented by the bitmap as 111100010.

[0114] S3042. The scheduler creates a node view according to the available resources, each cache block, and the association relationship between the available resources and each cache block.

[0115] In some embodiments, to prevent the node view from occupying too much memory, S3042 can be implemented as follows: The scheduler creates a node view based on the available resources, each cache block, the association relationship between the available resources and each cache block, and the limit information.

[0116] Among them, the limit information is used to limit the depth and / or width of the node view to prevent the node view from occupying too much memory. Among them, the depth of the node view can be understood as the level or order of the node view. The width of the node view can be understood as the number of child nodes included in the nodes in the node view. The limit information can be included in the above configuration information.

[0117] In one implementation, the limit information may include a first threshold and a second threshold. Among them, the first threshold is used to limit the arrangement level of each available resource in the node view; the second threshold is used to limit the number of nodes in each level of cache blocks in the node view. Exemplarily, the first threshold can be 3, and the second threshold can be 9.

[0118] Exemplarily, taking the node view as a B+ tree as an example, the order of the B+ tree is 3, and the number of child nodes included in the tree nodes is less than the order. The available resources serve as the indexes of the B+ tree, and the available resources and their corresponding cache blocks serve as the tree nodes of the B+ tree.

[0119] Continuing with the above example, the available resources may include: 1-core CPU, 4-core CPU, 8-core CPU, 12-core CPU, 14-core CPU, 16-core CPU, 20-core CPU, 28-core CPU, 32-core CPU, 36-core CPU, 42-core CPU, 48-core CPU, 52-core CPU, 58-core CPU, 64-core CPU, 76-core CPU, 85-core CPU, and 96-core CPU.

[0120] Then, the scheduler can arrange the above available resources in ascending order. After that, the scheduler moves the indexes that are integer multiples of the order to the parent nodes, and according to the number of child nodes being less than the order, obtains the B+ tree. Exemplarily, the order of the B+ tree is 3. The scheduler moves the 3rd index (i.e., 8-core CPU) to be the parent node of 1-core CPU, 4-core CPU, and 14-core CPU, moves the 6th index (i.e., 16-core CPU) to be the parent node of 14-core CPU, 20-core CPU, and 28-core CPU, moves the 12th index (i.e., 48-core CPU) to be the parent node of 36-core CPU, 42-core CPU, and 52-core CPU, and moves the 15th index (i.e., 64-core CPU) to be the parent node of 58-core CPU, 76-core CPU, and 85-core CPU.

[0121] After that, the scheduler moves the indexes that are integer multiples of the order in the parent node up to the parent node. Exemplarily, the scheduler moves the 9th index (i.e., 32-core CPU) up to be the parent node of the 3rd index (i.e., 8-core CPU) and the 6th index (i.e., 16-core CPU), and moves the 18th index (i.e., 96-core CPU) up to be the parent node of the 12th index (i.e., 48-core CPU) and the 15th index (i.e., 64-core CPU).

[0122] The scheduler can configure identifiers for the cache blocks of each tree node according to the hierarchy and the order of the index values of each tree node. Exemplarily, in the first level, the identifier of the cache block of the 32-core CPU is B1,... In the second level, the identifier of the cache block of the 8-core CPU is B3, and the identifier of the cache block of the 16-core CPU is B4,... In the third level, the identifier of the cache block of the 1-core CPU is B7, and the identifier of the cache block of the 4-core CPU is B8,... The identifier of the cache block of the 76-core CPU is B17, and the identifier of the cache block of the 85-core CPU is B18. Finally, the scheduler can obtain the structure schematic diagram of the B+ tree as shown Figure 4 shown.

[0123] In another implementable manner, on the basis of S3041-S3042, S304 may further include S3043-S3044.

[0124] S3043: The scheduler clusters each job according to the required resources of the job to obtain each job set.

[0125] Example 1: The required resources of Job 1 are 20-core CPU, and the required resources of Job 2 are 20-core CPU. The number of CPU cores of the first node is 20. Then, both Job 1 and Job 2 can be scheduled to the first node. In this way, the required resources of Job 1 and Job 2 on the first node are both 20-core CPU. For another example, the number of CPU cores of the second node is also 20. Then, both Job 1 and Job 2 can be scheduled to the first node and the second node. In this way, the required resources of Job 1 and Job 2 on the first node and the second node are both 20-core CPU. Therefore, Job 1 and Job 2 belong to the same job set.

[0126] Example 2: The required resources of Job 1 are 64G of memory, and the required resources of Job 2 are 128G of memory. The memory of the first node is 128G. Then, both Job 1 and Job 2 can be scheduled to the first node. In this way, the required resources of Job 1 on the first node are 64G, and the required resources of Job 2 on the first node are 128G. Therefore, Job 1 and Job 2 belong to the same job set.

[0127] In the embodiments of the present application, the scheduler may cluster each job according to the same resource requirements, which facilitates subsequent utilization of the associations between jobs and between nodes and jobs. When performing resource scheduling for other jobs with the same resource requirements, the scheduler can use the existing node view to batch skip the filtered nodes, thereby achieving efficient resource scheduling.

[0128] S3044. The scheduler creates a node view based on the available resources, each cache block, the association relationship between the available resources and each cache block, the required resources, and the association relationship between each job set and each cache block.

[0129] Exemplarily, each job may include Job 1 - Job 3, and the required resources of Job 1 - Job 3 are the same.

[0130] In addition, the jobs that the scheduler currently needs to process may include Job 1, Job 2, and Job 3, and the resources scheduled for Job 1, Job 2, and Job 3 are the same. Therefore, the scheduler may construct Job 1, Job 2, and Job 3 into a job set and establish a corresponding relationship between the job set and each node set. The corresponding relationship between the job set and each node set is stored in the above-mentioned each cache block. Exemplarily, if the required resources of Job 1, Job 2, and Job 3 are 1-core CPU, then Job 1, Job 2, and Job 3 correspond to Node 1 - Node 9. Similarly, if the required resources of Job 1, Job 2, and Job 3 are 20-core CPU, then Job 2 and Job 3 correspond to Node 55 - Node 63.

[0131] In the embodiments of the present application, by constructing a node view, such as a B+ tree, the association relationships between the available resources and nodes, between nodes, between jobs, and between job sets and node sets can be established. When using the node view to query schedulable nodes subsequently, the schedulable nodes can be quickly queried, improving the scheduling efficiency and the cache hit rate.

[0132] In some embodiments, in order to make the node view more accurate, the above-mentioned node view may be updated in real time. The update timing of the node view may include the following:

[0133] Timing 1. When the resources of the nodes in a certain cache block are scheduled, update the node view.

[0134] In one example, in the case where the scheduler schedules the first job, the scheduler determines the set of schedulable nodes for the job. The scheduler selects a target node from the set of schedulable nodes to run the first job. When the first job is running on the target node, the available resources of the target node will change. At this time, the scheduler updates the available resources of the target node, and the cache block related to the target node is also updated accordingly, thereby updating the node view.

[0135] Exemplarily, assume that before the target node runs the first job, the available memory of the target node is 64G. The required memory for the first job is 32G. After the first job runs on the target node, the available memory of the target node becomes 32G. In this way, the available resources of the target node change from 64G to 32G. At this time, the scheduler can change the association relationship in the cache block corresponding to the available resources of the target node, that is, change it to that the available resources of the target node do not meet 64G. Or, the scheduler can also change the cache block corresponding to the target node. Exemplarily, the cache block corresponding to the target node is 64G, and after the change, the cache block corresponding to the target node is 32G.

[0136] Timing two, when the user submits a new job, update the node view.

[0137] In one example, when the scheduler receives a new job (such as the second job), the scheduler can dynamically establish an index value according to the second resources required by the second job and add the index value to the node view to update the node view.

[0138] The second stage is the usage stage of the node view.

[0139] Figure 6 Another process schematic diagram of a resource scheduling method is also provided for the embodiments of the present application, as Figure 6 shown, the method may include: S601 - S614 (where some steps are optional).

[0140] S601. The scheduler receives the first job, and the first job may carry the first resources required by the first job.

[0141] The first resources may include at least one of the number of CPU cores, the memory occupancy, and the number of GPU cards.

[0142] The first job may include multiple sub - jobs, such as sub - job 1, sub - job 2, and sub - job 3. Of course, the first job may also refer to a single job. The embodiments of the present application do not make specific limitations.

[0143] In one implementable manner, Figure 5 Another process schematic diagram of a resource scheduling method provided for the embodiments of the present application, or: Figure 5 A usage process schematic diagram of using the node view based on the above - mentioned system architecture. As Figure 5 shown, S601 can be implemented as: Step ① The user can pass through the API interface of the scheduler (such as Figure 5Submit the job through the job submission interface, command-line tool, or visual operation interface shown in the figure. At this time, the job submission interface of the scheduler receives the job submitted by the user. Step ② The job submission interface sends the job to the job scheduling module of the scheduler. Correspondingly, the job scheduling module of the scheduler receives the job.

[0144] Example 1, the user can submit the first job through the command-line tool of the scheduler. The code of the first job can be:

[0145] / / The user submits a job that requires 20-core CPUs, 64G of memory, 1 GPU data card, and other resources

[0146] job_cli –-resource ‘cpu=20;mem=64G;gpu=1’ myjob.sh

[0147] S602. The scheduler initializes each node set.

[0148] After the scheduler receives the job in S601, the scheduler can schedule the resources required by the job according to the scheduling principle. Among them, the scheduling principle can include the first-in, first-out principle. Exemplarily, after the job scheduling module of the scheduler receives the job submitted by the user, the scheduler can schedule the first resource required by the first job according to the first-in, first-out principle.

[0149] In one implementable manner, the scheduler can determine to initialize each node set according to the node information of each node. Exemplarily, the scheduler can determine each initialized node set according to the total resources of each node. The available resources of each node in the initialized node set are determined according to the resources of each node when the scheduler selects to schedule the first job.

[0150] In some embodiments, S602 is executed before the scheduler executes S601, and the embodiments of the present application do not make specific limitations.

[0151] S603. The scheduler queries the node view according to the first resource to determine whether there is a cache block corresponding to the first resource; in the case of finding the first cache block corresponding to the first resource, the scheduler sequentially executes S604 and S605; in the case of not finding the first cache block corresponding to the first resource, the scheduler executes S606.

[0152] The node view is used to represent the association relationship between available resources and cache blocks, and the association relationship between available resources and node sets is stored in the cache block. The node view can be created in the first stage, and the creation process can refer to the above relevant description and will not be elaborated here.

[0153] Exemplarily, the node view is Figure 4The B+ tree shown in []. When the first resource is a 20-core CPU, the scheduler looks up the B+ tree using the 20-core CPU as the index. As Figure 7 shown, the lookup process can be as follows: Lookup ① The scheduler looks up the parent nodes (32-core CPU and 96-core CPU) of the B+ tree using the 20-core CPU as the index and finds the parent node 32-core CPU. Lookup ② The scheduler looks up the child nodes of the parent node 32-core CPU (i.e., 8-core CPU and 16-core CPU). Lookup ③ The scheduler looks up the child nodes of the parent node 16-core CPU (i.e., 20-core CPU and 28-core CPU). Finally, the scheduler finds the cache block B11 corresponding to the 20-core CPU.

[0154] In the embodiments of the present application, the scheduler can find the target with only three lookups according to the index link in the node view, can quickly find the resources, does not need to traverse each node in turn, saves a large amount of calculations, and improves the scheduling efficiency of the scheduler.

[0155] S604. The scheduler determines a first candidate node set in the first cache block, and the available resources of each candidate node in the first candidate node set are greater than or equal to the first resource.

[0156] When the scheduler finds the first cache block corresponding to the first resource, the available resources of each node in the first cache block may not meet the first resource. Therefore, the scheduler takes out each candidate node in the first cache block whose available resources meet the first resource, and these candidate nodes form the first candidate node set. Specifically, the scheduler determines the first candidate node set in the first cache block. Exemplarily, taking the Figure 4 B+ tree in [] as an example, the first cache block is cache block B11. The bitmap of cache block B11 is 111001100. As described above, the bitmap represents the association relationship between nodes 55 - node 63 and available resources. Then the scheduler can determine that the available resources of nodes 55, node 56, node 57, node 60, and node 61 meet the first resource. In this way, the scheduler forms the first candidate node set with nodes 55, node 56, node 57, node 60, and node 61.

[0157] To ensure that the available resources of each node in the first candidate node set meet the first resource, in one implementable manner, S604 can be implemented as:

[0158] S6041. The scheduler takes out a second candidate node set from the first cache block, and the available resources of each candidate node in the second candidate node set are greater than or equal to the first resource.

[0159] Continuing with the above example, the first cache block is cache block B11, and the association relationship between each node and available resources is stored in cache block B11. According to the association relationship, nodes that meet the 20-core CPU can be determined. Exemplarily, the available resources of nodes 55 - 57 and nodes 60 - 61 both meet the 20-core CPU. Then, nodes 55 - 57 and nodes 60 - 61 are nodes in the second candidate node set.

[0160] S6042. When it is determined that the available resources of each candidate node in the second candidate node set in the initialization node set are greater than or equal to the first resource, the scheduler adds each candidate node to the first candidate node set.

[0161] The initialization node set is obtained by clustering each node in the resource pool according to the first job. When the scheduler schedules the first job, the scheduler clusters each node in the resource pool according to the required resources of the first job to obtain the initialization node set. The scheduler combines the available resources of each node in the initialization node set to determine whether the available resources of each candidate node in the second candidate node set still meet the first resource. Exemplarily, assume that in the initialization node set, the available resources of nodes 55 - 57 meet the 20-core CPU, and the available resources of nodes 60 - 61 do not meet the 20-core CPU. Then, the scheduler adds nodes 55 - 57 to the first candidate node set.

[0162] In an implementable manner, the association relationship between the available resources stored in the cache block and the node set is represented by a bitmap. The bitmap includes a first value and a second value. Among them, the first value indicates that the available resources are less than the first resource, and the second value indicates that the available resources are greater than or equal to the first resource. S6042 can be implemented as:

[0163] Step a. The scheduler determines the second bitmap of the second candidate node set according to the first bitmap of the first cache block.

[0164] Continuing with the above example, the first value is represented as 0, the second value is represented as 1, the first cache block can be cache block B11, and the first bitmap of cache block B11 can be 111001100. Then, the scheduler extracts the nodes with the digit "1" to obtain the second candidate node set. The second candidate node set is nodes 55 - 57 and nodes 60 - 61. The second bitmap of the second candidate node set is 11111.

[0165] Step b. The scheduler determines the third bitmap corresponding to the second candidate node set in the initialization node set according to the identifiers of each node in the second candidate node set.

[0166] Continuing with the above example, in the initialized node set, the available resources of nodes 55 - 57 meet the requirement of 20-core CPUs, while the available resources of nodes 60 - 61 do not meet the requirement of 20-core CPUs. Then, the scheduler can determine that the third bitmap corresponding to the second candidate node set in the initialized node set is 11100.

[0167] Step c: The scheduler performs a logical AND operation on the second bitmap and the third bitmap to obtain the nodes represented by 1, and adds them to the first candidate node set.

[0168] Continuing with the above example, the scheduler performs a logical AND operation on the second bitmap and the third bitmap, i.e., "11111" & "11100", to obtain "11100", which indicates that the available resources of nodes 55 - 57 meet the requirement of 20-core CPUs. At this time, the scheduler adds nodes 55 - 57 to the first candidate node set.

[0169] In some embodiments, after finding the first cache block corresponding to the first resource, the method of the embodiments of the present application further includes: the scheduler batch-updates the association relationship between the available resources stored in the first cache block and the node set. Exemplarily, the association relationship between the available resources and each node stored in the first cache block is updated to: the available resources of nodes 55 - 57 meet the requirement of 20-core CPUs, and the available resources of nodes 58 - 63 do not meet the requirement of 20-core CPUs. That is, the bitmap of the first cache block is changed to: 111000000.

[0170] In some embodiments, if the available resources of the nodes in the first cache block meet the first resource, then the nodes in other cache blocks with an index value greater than the index value of the first cache block will necessarily also meet the first resource. Therefore, the scheduler can batch-update the nodes in other cache blocks with an index value greater than the index value of the first cache block. Exemplarily, the scheduler can use the above method to determine the nodes in other cache blocks whose available resources meet the first resource, and add the nodes to the first schedulable node set.

[0171] Continuing with the above example, since the node set corresponding to 20-core CPUs meets the resources required by the first job, then the available resources greater than 20-core CPUs (such as 28-core CPUs - 96-core CPUs) will necessarily also meet the resources required by the first job. Therefore, the scheduler can use the above method to determine the nodes in the candidate node sets corresponding to 28-core CPUs - 96-core CPUs that meet the resources required by the first job, and add the nodes to the first schedulable node set. Therefore, in the embodiments of the present application, when the candidate nodes in a certain cache block are updated, the nodes with an index value greater than the cache block can be batch-updated, so as to update the node view in real time and improve the accuracy of the node view.

[0172] In the embodiments of the present application, by using bitmaps to represent the clustering information of each node and each job, the memory occupancy can be reduced. In addition, bitwise operations of the bitmaps can be used to achieve accelerated calculations to quickly determine schedulable nodes and improve scheduling efficiency.

[0173] S605. The scheduler determines the nodes in the first candidate node set as schedulable nodes.

[0174] S606. The scheduler sequentially traverses the available resources of each node in each available cache block, and the available cache block includes nodes whose available resources are greater than or equal to the first resource.

[0175] As long as the available resources of one node in the cache block meet the first resource, the cache block is an available cache block.

[0176] Exemplarily, the node view is the B+ tree corresponding to the number of CPU cores shown in Figure 4 The first resource is a 24-core CPU. The scheduler searches the B+ tree with the 24-core CPU as the index. As shown in Figure 8 , the search process can be as follows: Search ① The scheduler searches the parent nodes (32-core CPU and 96-core CPU) of the B+ tree with the 24-core CPU as the index and finds the parent node 32-core CPU. Search ② The scheduler searches the child nodes of the parent node 32-core CPU (i.e., 8-core CPU and 16-core CPU). Search ③ The scheduler searches the child nodes of the parent node 16-core CPU (i.e., 20-core CPU and 28-core CPU). However, the scheduler does not find the tree node corresponding to the 24-core CPU. At this time, the scheduler sequentially traverses the available cache blocks.

[0177] In the embodiments of the present application, the scheduler defaults to filtering out the node sets with available resources less than the first resource in the cache block. In this way, the scheduler initially filters out some nodes, avoiding a part of invalid scheduling and reducing part of the calculation amount.

[0178] In some embodiments, since the tree nodes with smaller index values must be subsets of the tree nodes with larger index values. Therefore, the scheduler can purposefully search for available cache blocks. Then S606 may include S6061 and S6062.

[0179] S6061. The scheduler determines the third cache block corresponding to the second resource, the second resource is less than the first resource, and the second resource is adjacent to the first resource.

[0180] Continuing with the above example, the first resource is a 24-core CPU. As can be seen from the B+ tree described in Figure 4 , since the node set corresponding to the 20-core CPU must be a subset of the node set corresponding to the 24-core CPU. Therefore, the scheduler can determine the 20-core CPU (i.e., the above-mentioned second resource) and the cache block B11 corresponding to the 20-core CPU (i.e., the third cache block).

[0181] S6062. When there is a node in the third cache block whose available resources are greater than or equal to the first resource, the scheduler determines that the third cache block is an available cache block.

[0182] As long as there is one node in the third cache block whose available resources meet the first resource, the third cache block is an available cache block.

[0183] In some embodiments, since the node set corresponding to the 20-core CPU meets the resources required by the first job, then the available resources greater than the 20-core CPU (such as 28-core CPU - 96-core CPU) will surely also meet the resources required by the first job. The above S606 may further include: S6063 - S6064.

[0184] S6063. The scheduler determines the fourth cache block corresponding to the third resource in the node view, where the third resource is greater than the second resource.

[0185] Continuing with the above example, the first resource is a 24-core CPU. As can be seen from the B+ tree as described Figure 4 above, the scheduler can determine the fourth cache block (i.e., B1, B2, B5, B6, and B12 - B18) corresponding to the 28-core CPU - 96-core CPU (i.e., the third resource).

[0186] S6064. The scheduler determines that the fourth cache block is an available cache block.

[0187] As long as there is one node in the fourth cache block whose available resources meet the first resource, the fourth cache block is an available cache block.

[0188] S607. The scheduler determines whether the available resources of the node sets in each available cache block are less than the first resource. When the available resources of the second node set in each available cache block are less than the first resource, the scheduler sequentially executes S608; when the available resources of the first node set in each available cache block are greater than or equal to the first resource, the scheduler executes S609.

[0189] S608. When the available resources of the second node set in each available cache block are less than the first resource, batch filter the second node set.

[0190] The scheduler filters out each node in each available cache block whose available resources do not meet the first resource.

[0191] In the embodiments of the present application, the scheduler further filters out the node set in the available cache block whose available resources are less than the first resource. In this way, the scheduler further filters out some nodes, avoiding ineffective scheduling and further reducing the amount of computation.

[0192] In one example, to avoid ineffective scheduling and waste of computing resources, in some embodiments, after the scheduler executes S608, the method provided by the embodiments of the present application may further include: S609 - S612.

[0193] S609. The scheduler determines a first node set in each available cache block, and the available resources of each node in the first node set are greater than or equal to a first resource.

[0194] S610. When the available resources of each designated node in the first node set in the initialization node set are greater than or equal to the first resource, the scheduler determines each designated node as a schedulable node.

[0195] S611. The scheduler filters out the nodes in the first node set other than each designated node.

[0196] After the scheduler determines the first node set in each available cache block, the scheduler further determines the relationship between the available resources of each node in the first node set in the initialization node set and the first resource. If the available resources of each designated node in the first node set meet the first resource, the scheduler determines each designated node as a schedulable node, and the scheduler filters out the nodes in the first node set other than each designated node.

[0197] In the embodiments of the present application, the scheduler further filters the first node set with available resources greater than the first resource in the available cache block. If the available resources of each node in the first node set in the initialization node set are less than the first resource, these nodes are filtered out. In this way, the scheduler further filters out some nodes, avoiding ineffective scheduling and further reducing the amount of computation.

[0198] After filtering these nodes, the scheduler sequentially traverses each available cache block and determines whether there are still nodes with available resources greater than or equal to the first resource in each available cache block. If there are nodes with available resources greater than or equal to the first resource in the available cache block, the scheduler executes S610; if there are no nodes with available resources greater than the first resource in the available cache block, the scheduler executes S612 and S613. Specifically as follows:

[0199] S612. The scheduler determines a second cache block in each available cache block, and the second cache block includes the nodes other than each designated node in the first node set and / or the nodes in the second node set.

[0200] It can be understood that each node in the second cache block may include: nodes other than each specified node in the first node set; or, nodes in the second node set; or, nodes other than each specified node in the first node set and nodes in the second node set. This means that the available resources of each node in the second cache block are all less than the first resource. Exemplarily, the bitmap of the second cache block may be represented as 000000000. This also means that the scheduler cannot schedule the resources of the nodes in the second cache block for the first job.

[0201] S613. When the scheduler schedules resources for the second job, the scheduler batch skips the resource scheduling for each node in the second cache block, and the second job and the first job belong to the same job set.

[0202] When the scheduler receives the second job, the second job and the first job belong to the same job set, and the node sets available for the second job and the first job are the same. In S612, the scheduler determines that the first job cannot use the resources of the nodes in the second cache block. Then, when the scheduler schedules the second job, similarly, the second job also cannot use the resources of the nodes in the second cache block. Then, the scheduler batch skips the resource scheduling for each node in the second cache block.

[0203] In the embodiments of the present application, since the required resources of the second job are the same as those of the first job, when the scheduler schedules resources for the second job, only the resource check result when scheduling the first job needs to be utilized, without the need for a large number of repeated resource checks, reducing the computational amount and system overhead, and avoiding redundant calculations.

[0204] In an implementable manner, the above S602 - S613 may be implemented by Figure 5 the job scheduling module shown, which will not be elaborated here.

[0205] S614. The scheduler schedules the resources of schedulable nodes for the first job to use.

[0206] In an implementable manner, after Figure 5 the job scheduling module shown determines the schedulable nodes, in step ③, the job scheduling module selects a target computing node from the set of schedulable nodes and sends the first job to the job management and control module of the target computing node. In step ④, the job management and control module sends the job to the operating system of the target computing node, and the operating system runs the job.

[0207] As Figure 9 shown, the embodiments of the present application further provide a schematic structural diagram of a resource scheduling device. The device 900 may include: a receiving module 901, a determining module 902, and a scheduling module 903. Among them,

[0208] The receiving module 901 is used to receive a first job, and the first job includes a first resource required by the first job.

[0209] The determining module 902 is used to determine, according to the first resource and the association relationship between available resources and cache blocks, the cache block corresponding to the first resource. The cache block includes a node set, available resources, and the association relationship between the available resources and the node set. When the first resource corresponds to the first cache block, determine the first candidate node set in the first cache block. The available resources of each candidate node in the first candidate node set are greater than or equal to the first resource, and the nodes in the first candidate node set are schedulable nodes.

[0210] The scheduling module 903 is used to schedule the resources of schedulable nodes for the first job to use.

[0211] In this application, by using the first resource as an index to find the association relationship between available resources and cache blocks, through the association relationship, the candidate node set that meets the first resource can be quickly found, without having to traverse each node multiple times to find the node that matches the first resource, reducing the amount of calculation and improving the scheduling efficiency.

[0212] In an alternative manner, the device 900 further includes: a traversing module 904. Among them,

[0213] The traversing module 904 is used to, when the first resource does not correspond to any cache block, sequentially traverse the available resources of each node in each available cache block. The available cache block includes nodes whose available resources are greater than or equal to the first resource.

[0214] The determining module 902 is used to determine the first node set in each of the available cache blocks. The available resources of each node in the first node set are greater than or equal to the first resource. When the available resources of each specified node in the first node set in the initialization node set are greater than or equal to the first resource, determine each of the specified nodes as a schedulable node. The initialization node set is obtained by clustering each node in the resource pool according to the first job.

[0215] In this application, by screening out available cache blocks and traversing the available resources of each node in the available cache blocks, the traversal of unavailable cache blocks is saved. In addition, in the available cache blocks, determine the first node set whose available resources are greater than or equal to the first resource, and then further judge the resources of the nodes in the first node set in the initialization node set, so as to find the nodes that meet the first resource and improve the scheduling efficiency.

[0216] In an alternative manner, the device 900 further includes: a filtering module 905. Among them,

[0217] The determination module 902 is used to determine a second node set in each of the available cache blocks, and the available resources of each node in the second node set are less than the first resource.

[0218] The filtering module 905 is used to batch-filter the second node set, and the nodes in the second node set are unschedulable nodes.

[0219] In the present application, by screening out the available cache blocks and traversing the available resources of each node in the available cache blocks, the traversal of unavailable cache blocks is saved. In addition, in the available cache blocks, the node set with available resources less than the first resource is batch-filtered, so that these nodes do not need to be considered when performing resource scheduling, improving the scheduling efficiency.

[0220] In an alternative manner, the cache block also stores the association relationship between the node set and the job set;

[0221] The determination module 902 is used to determine a second cache block in each of the available cache blocks, and the second cache block includes the nodes in the first node set except for each specified node and / or the nodes in the second node set.

[0222] The scheduling module 902 is used to batch-skip the resource scheduling of each node in the second cache block when scheduling resources for the second job, so as to schedule the resources of the nodes in the cache blocks other than the second cache block, and the second job and the first job belong to the same job set.

[0223] In the present application, by determining the second cache block in the available cache blocks, and the available resources of each node in the second cache block are less than the first resource, it means that each node in the second cache block is an unschedulable node. When performing resource scheduling for the second job, the scheduling of the second cache block can be batch-skipped, reducing the system overhead and improving the scheduling efficiency.

[0224] In an alternative manner, the determination module 902 is used to determine a third node set in each of the available cache blocks, and the available resources of each node in the third node set are greater than or equal to the first resource, and the nodes in the third node set are schedulable nodes.

[0225] In the present application, by determining the node set with available resources greater than or equal to the first resource in each of the available cache blocks as the third node set, the nodes in the third node set are all schedulable nodes. Subsequently, the scheduler only needs to schedule the resources of the schedulable nodes, without traversing each node, reducing the calculation amount and improving the scheduling efficiency.

[0226] In an alternative manner, the determination module 902 is configured to: when the first resource does not correspond to any cache block, determine a third cache block corresponding to a second resource, where the second resource is smaller than the first resource and the sizes of the second resource and the first resource are adjacent; when the nodes with available resources greater than or equal to the first resource are included in the third cache block, determine the third cache block as an available cache block.

[0227] In this application, by finding the cache block (i.e., the third cache block) corresponding to the second resource that is smaller than the first resource and closest to the first resource, the node set of the second resource closest to the first resource is found. The node set must be a subset of the node set of the first resource. Therefore, it is only necessary to determine the available cache block in the cache block corresponding to the second resource, further reducing the amount of calculation and improving the scheduling efficiency.

[0228] In an alternative manner, the determination module 902 is configured to: determine a fourth cache block corresponding to a third resource, where the third resource is greater than or equal to the second resource; determine the fourth cache block as an available cache block.

[0229] In this application, through the second resource, the cache block (i.e., the fourth cache block) corresponding to the third resource greater than the second resource is determined, filtering out the cache blocks smaller than the second resource. It is only necessary to determine the available cache block in the fourth cache block corresponding to the third resource, further reducing the amount of calculation and improving the scheduling efficiency.

[0230] In an alternative manner, the determination module 902 is configured to: take out a second candidate node set from the first cache block, where the available resources of each candidate node in the second candidate node set are greater than or equal to the first resource; when it is determined that the available resources of each candidate node in the second candidate node set in the initialization node set are greater than or equal to the first resource, add each candidate node to the first candidate node set, and the initialization node set is obtained by clustering each node in the resource pool according to the first job.

[0231] In an alternative manner, the association relationship between the available resources stored in the cache block and the node set is represented by a bitmap, and the bitmap includes a first value and a second value. Among them, the first value represents that the available resources are smaller than the first resource, and the second value represents that the available resources are greater than or equal to the first resource.

[0232] The determination module 902 is configured to: determine the second bitmap of the second candidate node set according to the first bitmap stored in the first cache block; determine the third bitmap corresponding to the second candidate node set in the initialization node set according to the identifiers of the nodes in the second candidate node set; perform a logical AND operation on the second bitmap and the third bitmap to obtain the nodes representing the second value and add them to the first candidate node set.

[0233] In this application, representing the association relationship between available resources and the node set through bitmaps can effectively reduce memory occupancy; in addition, by performing a logical AND operation on two bitmaps, the computational amount is greatly reduced and the scheduling efficiency is improved.

[0234] In an alternative manner, the apparatus 900 further includes: an update module 906, which is configured to batch update the association relationship between the available resources stored in the first cache block when the first resource corresponds to the first cache block.

[0235] In this application, after finding the first cache block corresponding to the first resource, it means that the node resources of the first cache block are called. At this time, updating the association relationship in the first cache block enables the association relationship to be updated in real time, improving the accuracy of the available resources of each node.

[0236] In an alternative manner, the association relationship between the available resources and the cache block is represented by a node view, and the apparatus 900 further includes: a clustering module 907 and a creation module 908. Among them,

[0237] The clustering module 907 is configured to cluster each node according to the available resources of each node in the resource pool to obtain each node set, and each node set is included in each cache block.

[0238] The creation module 908 is configured to create a node view according to the available resources, each cache block, and the association relationship between the available resources and each cache block.

[0239] In this application, by clustering each node according to the available resources of each node, it is convenient to utilize the correlation between nodes subsequently, and it is possible to achieve quick search for callable nodes and quick batch filtering of nodes, thereby realizing efficient resource scheduling.

[0240] In an alternative manner, the clustering module 907 is configured to cluster each job according to the required resources of the job to obtain each job set.

[0241] The creation module 908 is configured to: create a node view according to the available resources, each cache block, the association relationship between the available resources and each cache block, the required resources, and the association relationship between each job set and each cache block.

[0242] In this application, by clustering each job according to the same resource requirements, it is convenient to utilize the correlation between jobs and between nodes and jobs subsequently. When performing resource scheduling for other jobs with the same resource requirements, it is possible to batch skip the filtered nodes by using the existing node view, thereby realizing efficient resource scheduling.

[0243] In an alternative manner, a creation module 908 is configured to: create a node view according to available resources, each cache block, the association relationship between the available resources and each cache block, and restriction information; the restriction information is used to restrict the depth and / or width of the node view.

[0244] In an alternative manner, the creation module 908 is configured to: create a node view according to available resources, each cache block, the association relationship between the available resources and each cache block, a first threshold, and a second threshold, where the arrangement levels of the available resources in the node view satisfy the first threshold, and the number of nodes in each level of cache blocks of the node view satisfies the second threshold.

[0245] In this application, by restricting the node view, the memory occupation of the node view is effectively prevented, and the memory is saved.

[0246] In an alternative manner, the node view includes a multi-way search tree or a binary tree.

[0247] It should be understood that when the above-mentioned Figure 9 device realizes its functions, only the division of the above-mentioned functional modules is used for illustration. In practical applications, the above-mentioned functions can be allocated to different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the device provided in the above embodiment and the method embodiment belong to the same concept. For the specific implementation process, please refer to the method embodiment, which will not be elaborated here.

[0248] An embodiment of this application further provides an electronic device, which includes a processor. The processor is configured to load and run at least one instruction so that the electronic device realizes the resource scheduling method provided by the embodiment of this application. Optionally, the device further includes a memory, and the memory is coupled to the processor and is configured to store at least one instruction.

[0249] It should be understood that the above-mentioned processor may be a central processing unit (CPU), or may also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc. It is worth noting that the processor may be a processor that supports the advanced RISC machines (ARM) architecture.

[0250] Further, in an alternative embodiment, the above-mentioned memory may include a read-only memory and a random access memory, and provide instructions and data to the processor. The memory may also include a non-volatile random access memory. For example, the memory may also store information about the device type.

[0251] The memory may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example but not limitation, many forms of RAM are available. For example, static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchlink DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0252] An embodiment of the present application also provides a computer-readable storage medium, in which at least one instruction is stored, and the instruction is loaded and executed by a processor to enable a computer to implement the resource scheduling method described in any one of the above.

[0253] An embodiment of the present application also provides a computer program (product), when the computer program is executed by a computer, it can cause the processor or the computer to execute the corresponding steps and / or processes in the above method embodiments.

[0254] An embodiment of the present application also provides a chip, the chip includes a processor, which is used to call and run the instructions stored in the memory from the memory, so that a communication device installed with the chip executes the resource scheduling method described in any one of the above.

[0255] Another chip is also provided in an embodiment of the present application, including: an input interface, an output interface, a processor, and a memory. The input interface, the output interface, the processor, and the memory are connected through an internal connection path. The processor is configured to execute the code in the memory. When the code is executed, the processor is configured to execute any one of the resource scheduling methods described above.

[0256] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions described in the present application are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from a website, a computer, a server, or a data center to another website, computer, server, or data center in a wired manner (such as coaxial cable, optical fiber, digital subscriber line) or a wireless manner (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or a data center that includes one or more available media integrated. The available medium can be a magnetic medium (such as a floppy disk, a hard disk, a magnetic tape), an optical medium (such as a DVD), or a semiconductor medium (such as a solid state disk).

[0257] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data for analysis, stored data, displayed data, etc.), and signals involved in the present application are all authorized by the user or fully authorized by all parties, and the collection, use, and processing of relevant data need to comply with the relevant laws, regulations, and standards of relevant countries and regions. For example, the setting results involved in the present application are obtained under sufficient authorization.

[0258] Those of ordinary skill in the art can realize that, in combination with the method steps and modules described in the embodiments disclosed herein, they can be implemented in software, hardware, firmware, or any combination thereof. To clearly illustrate the interchangeability of hardware and software, the steps and components of each embodiment have been generally described according to functions in the above description. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Those of ordinary skill in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of this application.

[0259] Those of ordinary skill in the art can understand that all or part of the steps for implementing the above embodiments can be completed by hardware or can be completed by a program instructing relevant hardware. The program can be stored in a computer-readable storage medium, and the above-mentioned storage medium can be a read-only memory, a disk, an optical disc, or the like.

[0260] When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer program instructions. As an example, the method of the embodiments of this application can be described in the context of machine-executable instructions, such as program modules executed in devices included in a target real or virtual processor. Generally speaking, program modules include routines, programs, libraries, objects, classes, components, data structures, etc., which perform specific tasks or implement specific abstract data structures. In each embodiment, the functions of the program modules can be merged or split among the described program modules. The machine-executable instructions for the program modules can be executed within local or distributed devices. In a distributed device, the program modules can be located in both local and remote storage media.

[0261] The computer program code for implementing the method of the embodiments of this application can be written in one or more programming languages. These computer program codes can be provided to the processor of a general-purpose computer, a special-purpose computer, or a determining device of other programmable agents, such that when the program code is executed by the computer or the determining device of other programmable agents, it causes the functions / operations specified in the flowchart and / or block diagram to be implemented. The program code can be executed entirely on the computer, partially on the computer, as an independent software package, partially on the computer and partially on a remote computer, or entirely on a remote computer or server.

[0262] In the context of the embodiments of this application, the computer program code or related data can be carried by any suitable carrier so that the device, apparatus, or processor can execute the various processes and operations described above. Examples of the carrier include signals, computer-readable media, and the like.

[0263] Examples of signals can include electrical, optical, radio, acoustic, or other forms of propagated signals, such as carrier waves, infrared signals, and the like.

[0264] A machine-readable medium can be any tangible medium that contains or stores a program for or relating to an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any suitable combination thereof. More specific examples of machine-readable storage media include electrical connections with one or more wires, portable computer disks, hard disks, random access memories (RAMs), read-only memories (ROMs), erasable programmable read-only memories (EPROMs or flash memories), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0265] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices, and modules described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be elaborated herein.

[0266] In several embodiments provided in the present application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the modules is only a logical functional division, and there can be other division methods in actual implementation. For example, multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Additionally, the displayed or discussed couplings, direct couplings, or communication connections to each other can be indirect couplings or communication connections through some interfaces, devices, or modules, and can also be electrical, mechanical, or other forms of connections.

[0267] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical modules, that is, they can be located in one place, or distributed to multiple network modules. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of the embodiments of the present application.

[0268] In addition, the functional modules in various embodiments of the present application can be integrated into one processing module, or each module can exist physically alone, or two or more modules can be integrated into one module. The above-mentioned integrated modules can be implemented in the form of hardware or in the form of software functional modules.

[0269] When the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods in various embodiments of this application. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical discs that can store program codes.

[0270] In this application, terms such as "first" and "second" are used to distinguish between identical or similar items with basically the same functions and effects. It should be understood that there is no logical or chronological dependence between "first", "second", and "nth", nor are the quantity and execution order limited. It should also be understood that although the following description uses terms such as first and second to describe various elements, these elements should not be limited by the terms. These terms are only used to distinguish one element from another. For example, without departing from the scope of various described examples, the first image can be referred to as the second image, and similarly, the second image can be referred to as the first image. Both the first image and the second image can be images, and in some cases, they can be separate and different images.

[0271] It should also be understood that in various embodiments of this application, the magnitude of the serial numbers of each process does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of this application.

[0272] In this application, the meaning of the term "at least one" refers to one or more, and the meaning of the term "multiple" refers to two or more. For example, multiple second messages refer to two or more second messages. In this article, the terms "system" and "network" are often used interchangeably.

[0273] It should be understood that the terms used in the description of various described examples in this article are only for describing specific examples and are not intended to be limiting. As used in the description of various described examples and the appended claims, the singular forms "a", "an", and "the" are also intended to include the plural forms unless the context clearly indicates otherwise.

[0274] It should also be understood that the term "and / or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items. The term "and / or" describes an associative relationship between associated objects and indicates that three relationships may exist. For example, A and / or B may mean: A exists alone, A and B exist simultaneously, and B exists alone. Additionally, the character " / " in this application generally indicates that the associated objects before and after are in an "or" relationship.

[0275] It should also be understood that the term "comprises" (also referred to as "includes", "including", "comprises" and / or "comprising") when used in this specification specifies the presence of the stated features, integers, steps, operations, elements, and / or components, but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or their groups.

[0276] It should also be understood that the terms "if" and "when" can be interpreted to mean "when" ("when" or "upon") or "in response to determining" or "in response to detecting". Similarly, depending on the context, the phrase "if it is determined..." or "if [the stated condition or event] is detected" can be interpreted to mean "when it is determined..." or "in response to determining..." or "when [the stated condition or event] is detected" or "in response to detecting [the stated condition or event]".

[0277] It should be understood that determining B based on A does not mean determining B solely based on A. B can also be determined based on A and / or other information.

[0278] It should also be understood that the "one embodiment", "an embodiment", and "a possible implementation" mentioned throughout the specification mean that specific features, structures, or characteristics related to the embodiment or implementation are included in at least one embodiment of this application. Therefore, the "in one embodiment" or "in an embodiment", "a possible implementation" that appear throughout the specification do not necessarily refer to the same embodiment. Additionally, these specific features, structures, or characteristics can be combined in one or more embodiments in any suitable manner.

Claims

1. A resource scheduling method, characterized in that, The method includes: Receiving a first job, where the first job includes a first resource for the first job requirement; Determining, according to the first resource and the association relationship between available resources and cache blocks, the cache block corresponding to the first resource, where the cache block includes a node set, available resources, and the association relationship between the available resources and the node set; When the first resource corresponds to a first cache block, determining a first candidate node set in the first cache block, where the available resources of each candidate node in the first candidate node set are greater than or equal to the first resource, and the nodes in the first candidate node set are schedulable nodes; Scheduling the resources of the schedulable nodes for the first job to use.

2. The method according to claim 1, characterized in that, The method further includes: When the first resource does not correspond to any cache block, sequentially traversing the available resources of each node in each available cache block, where the available cache blocks include nodes whose available resources are greater than or equal to the first resource; Determining a first node set in each of the available cache blocks, where the available resources of each node in the first node set are greater than or equal to the first resource; When the available resources of each specified node in the first node set in the initialization node set are greater than or equal to the first resource, determining each of the specified nodes as a schedulable node; the initialization node set is obtained by clustering each node in the resource pool according to the first job.

3. The method according to claim 2, characterized in that The method further includes: Determining a second node set in each of the available cache blocks, where the available resources of each node in the second node set are less than the first resource; Batch filtering the second node set, where the nodes in the second node set are non-schedulable nodes.

4. The method according to claim 3, characterized in that The association relationship between the node set and the job set is also stored in the cache block. After batch filtering the second node set, it further includes: Determining a second cache block in each of the available cache blocks, where the second cache block includes the nodes in the first node set except each specified node and / or the nodes in the second node set; When scheduling resources for a second job, batch skipping the resource scheduling of each node in the second cache block to schedule the resources of the nodes in the cache blocks other than the second cache block, where the second job and the first job belong to the same job set.

5. The method according to any one of claims 2-4, characterized in that, Before sequentially traversing the available resources of each node in each available cache block, it further includes: When the first resource does not correspond to any cache block, determining a third cache block corresponding to a second resource, where the second resource is less than the first resource and the size of the second resource is adjacent to that of the first resource; When the third cache block includes nodes whose available resources are greater than or equal to the first resource, determining the third cache block as the available cache block.

6. The method according to claim 5, characterized in that, The method further includes: Determining a fourth cache block corresponding to a third resource, where the third resource is greater than or equal to the second resource; Determining the fourth cache block as the available cache block.

7. The method according to claim 1, characterized in that The determining the first candidate node set in the first cache block includes: Taking out a second candidate node set from the first cache block, where the available resources of each candidate node in the second candidate node set are greater than or equal to the first resource; When it is determined that the available resources of each candidate node in the second candidate node set in the initialization node set are greater than or equal to the first resource, add each candidate node to the first candidate node set, where the initialization node set is obtained by clustering each node in the resource pool according to the first job.

8. The method according to claim 7, characterized in that The association relationship between the available resources stored in the cache block and the node set is represented by a bitmap, and the bitmap includes a first value and a second value, where the first value indicates that the available resources are less than the first resource, and the second value indicates that the available resources are greater than or equal to the first resource; The adding each candidate node to the first candidate node set includes: Determine the second bitmap of the second candidate node set according to the first bitmap stored in the first cache block; Determine the third bitmap corresponding to the second candidate node set in the initialization node set according to the identifiers of the nodes in the second candidate node set; Perform a logical AND operation on the second bitmap and the third bitmap to obtain the nodes representing the second value, and add them to the first candidate node set.

9. The method according to claim 1 or 7 or 8, characterized in that The method further includes: When the first resource corresponds to the first cache block, batch-update the association relationship between the available resources stored in the first cache block and the node set.

10. The method according to any one of claims 1-9, characterized in that, The association relationship between the available resources and the cache block is represented by a node view. Before determining the cache block corresponding to the first resource, it further includes: Cluster each node according to the available resources of each node in the resource pool to obtain each node set, and each node set is included in each cache block; Create the node view according to the available resources, each cache block, and the association relationship between the available resources and each cache block.

11. The method according to claim 10, characterized in that, The method further includes: Cluster each job according to the required resources of the job to obtain each job set; The creating the node view includes: Create the node view according to the available resources, each cache block, the association relationship between the available resources and each cache block, the required resources, and the association relationship between each job set and each cache block.

12. The method according to claim 10 or 11, characterized in that, The creating the node view includes: Create the node view according to the available resources, each cache block, the association relationship between the available resources and each cache block, and the restriction information; the restriction information is used to restrict the depth and / or width of the node view.

13. The method according to any one of claims 10 to 12, characterized in that, The node view includes a multi-way search tree or a binary tree.

14. A resource scheduling device, characterized in that, The apparatus executes the method according to any one of claims 1-13.

15. An electronic device, characterized in that, The device includes a memory and a processor; at least one instruction is stored in the memory, and the at least one instruction is loaded and executed by the processor to enable the electronic device to implement the method according to any one of claims 1-13.

16. A computer-readable storage medium, characterized in that, At least one instruction is stored in the computer-readable storage medium, and the instruction is loaded and executed by the processor to enable the computer to implement the method according to any one of claims 1-13.

Images