Load-aware scheduling system and method oriented to multi-dimensional resources

Abstract

The invention discloses a multi-dimensional resource-oriented load aware scheduling system, which is applied to a cluster platform, the cluster platform comprises a main node and a plurality of cluster nodes, the main node is provided with a service component, a scheduling component and a monitoring component, and each cluster node is provided with a monitoring component. According to the load-aware scheduling system and method oriented to the multi-dimensional resources, before container task scheduling, all container tasks and all nodes are sorted, and the optimal nodes are matched for all the container tasks according to the preset optimal matching principle; the unbalanced problem that most of container tasks are scheduled to the same node to run, so that resources of part of nodes are excessive, and loads of part of nodes are overloaded is solved. In the optimal matching principle, not only are resource requirements of the container task on three dimensions of a CPU, a disk and a memory met, but also 10% of resource margin is reserved for the container task to use.

Description

Technical Field

[0001] The present invention belongs to the technical field of resource scheduling management, and particularly relates to a load-aware scheduling system and method for multi-dimensional resources. Background Art

[0002] In today's rapidly developing cloud computing environment, container orchestration tools have achieved efficient resource utilization by significantly simplifying the resource management and scheduling distribution processes in cloud platforms, providing strong support for cloud computing. For the key issue of real-time response to resource requests in container clouds, scheduling algorithms have become a key research topic in the field of container orchestration. The current mainstream container orchestration tool, Kubernetes, provides default pre-selection and preference scheduling algorithms to meet the requirements for efficient resource scheduling in cloud computing environments.

[0003] There are two main problems with the pre-selection and preference scheduling algorithm strategies defaultly adopted by Kubernetes, which need to be further improved and optimized. First, the default scheduling strategy only considers two resources, CPU and memory, without considering the disk usage factor or the performance changes of the nodes themselves. In actual production environments, the performance of the nodes themselves is also a relatively important influencing factor. Second, when the preference and pre-selection strategies in the default scheduling strategy of the Kubernetes cluster perform node screening, they are both judged based on the resource capacity of the nodes themselves and the application amounts of the resource objects of the Pods to be scheduled. This static scheduling method cannot accurately reflect the real load situation of the cluster, and the real resource utilization rate of containers often varies greatly from the Request / Limit of the resources they apply for, which directly leads to the problem of uneven cluster load. Summary of the Invention

[0004] The purpose of the present invention is to propose a load-aware scheduling system and method for multi-dimensional resources to solve the problems proposed in the background art.

[0005] To achieve the above purpose, the technical solutions adopted by the present invention are as follows:

[0006] A load-aware scheduling system for multi-dimensional resources proposed by the present invention is applied to a cluster platform. The cluster platform includes a master node and multiple cluster nodes. A service component, a scheduling component, and a monitoring component are provided on the master node, and a monitoring component is provided on each cluster node, where:

[0007] Each of the monitoring components is used to periodically collect the resource load information, the online status of the node itself, and the resource usage during the operation of each container task on the node itself, and report them to the service component. At the same time, the resource usage during the operation of each container task on the node itself is reported to the scheduling component;

[0008] The service component is used to receive the issued container task list, store the configuration parameters of each container task in the container task list into the database, store the resource load information of each node, the in-position status of each node, and the resource usage during the running of container tasks on each node reported by each monitoring component into the database, and send task scheduling information to the scheduling component;

[0009] The scheduling component is used to retrieve the database after receiving the task scheduling information, sort each container task according to the configuration parameters to obtain a task list, sort each node according to the resource load information and the configuration parameters of each container task to obtain a node list corresponding to each container task, and according to the preset best matching principle, sequentially match the best nodes for the container tasks in the task list and run the container tasks on the best nodes. At the same time, it monitors in real time the resource usage during the running of container tasks on each node reported by the monitoring component, judges the relationship between the resource usage of the container task and the configuration parameters of the container task, and performs dynamic scheduling on each container task.

[0010] Preferably, the resource load information of each node includes the current CPU margin, the current disk margin, and the current memory margin, and the resource usage during the running of each container task includes the actual CPU usage, the actual disk usage, and the actual memory usage.

[0011] Preferably, the configuration parameters of each container task include a task name, an image name, resource requirements, weights of resource requirements, and priorities. Among them, the resource requirements include the estimated CPU usage, the estimated disk usage, and the estimated memory usage for running each container task. The weights of resource requirements include the CPU weight, the disk weight, and the memory weight. The priorities are divided into normal, important, and urgent, and the hierarchical relationship from high to low is urgent > important > normal. The image corresponding to the image name of the container task is stored in the image repository;

[0012] At the same time, the service component classifies each container task in the container task list according to the weights of resource requirements. When the CPU weight of the container task is the highest, the type of the container task is CPU-intensive. When the disk weight of the container task is the highest, the type of the container task is disk-intensive. When the memory weight of the container task is the highest, the type of the container task is memory-intensive.

[0013] Preferably, when the scheduling component is used to retrieve the database after receiving the task scheduling information, sort each container task according to the configuration parameters to obtain a task list, and sort each node according to the resource load information and the configuration parameters of each container task to obtain a node list corresponding to each container task, it operates in the following manner:

[0014] The scheduling component sorts each container task from high to low according to the descending order of priority in the configuration parameters of each container task in the database. When the priorities are the same, they are sorted from high to low according to the comprehensive task scores, where the comprehensive task score = estimated CPU usage * CPU weight + estimated disk usage * disk weight + estimated memory usage * memory weight. After sorting, a task list is obtained.

[0015] For each container task, the scheduling component calculates the comprehensive node score based on the resource load information and the configuration parameters of the container task, and sorts each node from high to low according to the comprehensive node score. After sorting, a node list corresponding to each container task is obtained, where the comprehensive node score = current CPU margin * CPU weight + current disk margin * disk weight + current memory margin * memory weight.

[0016] Preferably, when the scheduling component matches the best node for each container task according to the preset best matching principle, it operates as follows:

[0017] The conditions of the best matching principle are: the current CPU margin of the node is greater than or equal to 1.1 times the estimated CPU usage of the current container task, the current disk margin of the node is greater than or equal to 1.1 times the estimated disk usage of the current container task, the current memory margin of the node is greater than or equal to 1.1 times the estimated memory usage of the current container task, and the comprehensive node score is greater than or equal to the comprehensive task score.

[0018] For each container task, it is sequentially determined whether each node meets the conditions of the best matching principle according to the order of the corresponding node list, and the node that first meets the conditions of the best matching principle is used as the best node of the current container task.

[0019] If there is no node in the corresponding node list that can meet the conditions of the best matching principle for the current container task, the container tasks on each node are sequentially paused according to the order of the corresponding node list. When pausing the container tasks on the node, the container tasks with general priority and the same type as the current container task are paused in the order in which the container tasks are matched to the node until there is a node that can meet the conditions of the best matching principle.

[0020] If the scheduling component retrieves that the matched best node is in an out-of-position state, a new best node is matched for the container task on the out-of-position best node according to the preset best matching principle.

[0021] Preferably, the scheduling component monitors in real time the resource usage during the running of container tasks on each node reported by the monitoring component, judges the relationship between the resource usage of the container tasks and the configuration parameters of the container tasks, and when dynamically scheduling each container task, operates in the following manner:

[0022] For the CPU usage and memory usage of the container tasks on a node, when at least one of the actual usages exceeds the first preset range of the estimated usage, and at the same time the corresponding current margin in the resource load information of the node exceeds the second preset value of the estimated usage, the scheduling component expands the estimated usage corresponding to the container task to the corresponding actual usage;

[0023] For the CPU usage, disk usage and memory usage of the container tasks on a node, when at least one of the actual usages exceeds the first preset range of the estimated usage, and at the same time the corresponding current margin in the resource load information of the node is less than the third preset value, or when at least one of the actual usages exceeds the second preset value of the estimated usage, the scheduling component pauses the running of the container task, and matches a new best node for the container task according to the preset best matching principle. After the container task runs on the new best node, the container task on the original best node is deleted.

[0024] Preferably, when the scheduling component runs a container task on the best node, it operates in the following manner:

[0025] If there is no mirror of the container task to be run on the current best node, the scheduling component first obtains the mirror name corresponding to the container task from the database, then obtains the corresponding mirror from the mirror repository to the current best node according to the mirror name, and creates and runs the container task according to the mirror of the container task;

[0026] If there is a mirror of the container task to be run on the current best node, the scheduling component directly creates and runs the container task according to the mirror of the container task.

[0027] Preferably, when the scheduling component runs a container task on the best node, if there is already a container task with the same task name on other nodes, before creating the container task on this best node, the scheduling component pauses the running of the container task with the same task name on other existing nodes until after creating and running the container task on this best node, the container task with the same task name on other existing nodes is deleted.

[0028] Preferably, the scheduling component also feeds back the scheduling result to the service component, and if the container task runs successfully, the scheduling result is scheduling success; if the container task runs fails or the container task creation fails, the scheduling result is scheduling failure, and the service component updates the scheduling status of each container task in the database according to the fed-back scheduling result.

[0029] A load-aware scheduling method for multi-dimensional resources, which is applied to a load-aware scheduling system for multi-dimensional resources;

[0030] The load-aware scheduling method for multi-dimensional resources includes:

[0031] Each of the monitoring components periodically collects the resource load information and the on-site status of its own node, and reports them to the service component;

[0032] The service component receives the issuance of the container task list, stores the configuration parameters of each container task in the container task list in the database, and at the same time stores the resource load information and the on-site status of each node reported by each monitoring component in the database, and sends task scheduling information to the scheduling component;

[0033] After receiving the task scheduling information, the scheduling component retrieves the database, sorts each container task according to the configuration parameters to obtain a task list, sorts each node according to the resource load information and the configuration parameters of each container task to obtain a node list corresponding to each container task, and according to the preset best matching principle, sequentially matches the best nodes for the container tasks in the task list, and runs the container tasks on the best nodes;

[0034] Each of the monitoring components periodically collects the resource usage during the running of each container task on its own node, and reports it to the service component and the scheduling component, and the service component stores and updates the resource usage during the running of the container tasks on each node in the database;

[0035] The scheduling component monitors in real time the resource usage during the running of the container tasks on each node reported by the monitoring component, judges the relationship between the resource usage of the container task and the configuration parameters of the container task, and performs dynamic scheduling on each container task.

[0036] Compared with the prior art, the beneficial effects of the present invention are:

[0037] 1. Before scheduling the container tasks, the load-aware scheduling system and method for multi-dimensional resources sort each container task and each node, and according to the preset best matching principle, match the best nodes for each container task, solving the unbalanced problem that most of the container tasks are scheduled to run on the same node, resulting in resource surplus on some nodes and heavy load on some nodes;

[0038] 2. In the best matching principle, not only just meet the resource requirements of the container task for the three dimensions of CPU, disk and memory, but also reserve 10% of the resource margin for the container task to use;

[0039] 3. After the estimated resource requirements of the container task match the best node, it will be dynamically scheduled according to the actual resource usage required when the container task is running. This not only solves the problem of insufficient actual resource usage required when the container task is running due to the estimated resource requirements of the container task, but also ensures that the container task has a sufficient operating environment. It also solves the problem of excessive node load caused by excessive resources required by the container task. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 It is a module block diagram of a cluster platform in a load-aware scheduling system and method for multi-dimensional resources of the present invention;

[0041] Figure 2 The present invention is a flow chart of a load-aware scheduling method for multi-dimensional resources. DETAILED DESCRIPTION

[0042] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0043] In one embodiment, Figure 1 As shown in FIG. 1 , a load-aware scheduling system for multi-dimensional resources is provided, which is applied to a cluster platform. The cluster platform includes a master node and multiple cluster nodes (each node is interconnected through a network). The master node is provided with a service component, a scheduling component and a monitoring component. Each cluster node is provided with a monitoring component. Figure 1 As shown, where:

[0044] Each monitoring component is used to periodically collect resource load information of the node (i.e., the node to which the monitoring component belongs), the status of the node, and the resource usage of each container task on the node during operation, and report it to the service component. At the same time, the resource usage of each container task on the node during operation is reported to the scheduling component.

[0045] In this embodiment, the resource load information of each node includes the current CPU margin (i.e., the number of cores remaining in the current CPU), the current disk margin (i.e., the current remaining disk size, in MB), and the current memory margin (i.e., the current remaining memory size, in MB). The resource usage during the operation of each container task includes the actual CPU usage, actual disk usage, and actual memory usage.

[0046] A service component is used to receive the issued container task list (where the container task list contains multiple container tasks and the container task list is issued by the user to the service component), store the configuration parameters of each container task in the container task list in the database, and store the resource load information of each node reported by each monitoring component, the presence status of each node, and the resource usage during the running of container tasks on each node in the database (where the resource usage during the running of container tasks is updated in real time to the database in the subsequent process), and send task scheduling information to the scheduling component (where the scheduling information refers to receiving the container task list issued by the user);

[0047] In this embodiment, the configuration parameters of each container task include task name, image name, resource requirements, weights of resource requirements, and priorities. Among them, the resource requirements include the estimated CPU usage, estimated disk usage, and estimated memory usage for running each container task. The weights of resource requirements include CPU weight, disk weight, and memory weight (where the sum of the CPU weight, disk weight, and memory weight is 10). The priorities are divided into normal, important, and urgent, and the hierarchical relationship from high to low is urgent > important > normal. The image corresponding to the image name of the container task is stored in the image repository (the image repository is used to store the images of container tasks. In this embodiment, the image repository is set on the master node); where each CPU usage is the number of CPU cores required, each disk usage is the required disk size, and each memory usage is the required memory size;

[0048] At the same time, the service component classifies each container task in the container task list according to the weights of resource requirements. When the CPU weight of a container task is the highest, the type of this container task is CPU-intensive. When the disk weight of a container task is the highest, the type of this container task is disk-intensive. When the memory weight of a container task is the highest, the type of this container task is memory-intensive (and the type of each container task is saved to the database. When a certain resource weight is higher, it means that this container task requires this kind of resource more).

[0049] A scheduling component is used to, after receiving the task scheduling information, retrieve the database, sort each container task according to the configuration parameters to obtain a task list, sort each node according to the resource load information and the configuration parameters of each container task to obtain a node list corresponding to each container task, and, according to the preset best matching principle, sequentially match the best node for the container tasks in the task list and run the container tasks on the best nodes. At the same time, it monitors in real time the resource usage during the running of container tasks on each node reported by the monitoring component, judges the relationship between the resource usage of the container tasks and the configuration parameters of the container tasks, and performs dynamic scheduling on each container task.

[0050] In this embodiment, the scheduling component is used to retrieve the database after receiving task scheduling information, sort each container task according to the configuration parameters to obtain a task list, and sort each node according to the resource load information and the configuration parameters of each container task to obtain a node list corresponding to each container task. The operations are as follows:

[0051] The scheduling component sorts each container task from high to low according to the descending order of priority in the configuration parameters of each container task in the database. When the priorities are the same, they are sorted from high to low according to the task comprehensive score. After sorting, a task list is obtained, where the task comprehensive score = estimated CPU usage * CPU weight + estimated disk usage * disk weight + estimated memory usage * memory weight;

[0052] For each container task, the scheduling component calculates the node comprehensive score according to the resource load information and the configuration parameters of the container task, and sorts each node from high to low according to the node comprehensive score. After sorting, a node list corresponding to each container task is obtained (that is, each container task has its own corresponding ordered node list), where the node comprehensive score = current CPU margin * CPU weight + current disk margin * disk weight + current memory margin * memory weight.

[0053] In this embodiment, when the scheduling component matches the best node for each container task according to the preset best matching principle, the operations are as follows:

[0054] The conditions of the best matching principle are: the current CPU margin of the node is greater than or equal to 1.1 times the estimated CPU usage of the current container task, and the current disk margin of the node is greater than or equal to 1.1 times the estimated disk usage of the current container task, and the current memory margin of the node is greater than or equal to 1.1 times the estimated memory usage of the current container task, and the node comprehensive score is greater than or equal to the task comprehensive score (when there are nodes that meet the above four conditions at the same time, the conditions of the best matching principle are met);

[0055] For each container task, it is sequentially determined whether each node meets the conditions of the best matching principle according to the order of the corresponding node list (since each container task has a corresponding node list, for each container task, each node is sequentially determined according to the order of its corresponding node list), and the node that first meets the conditions of the best matching principle is used as the best node of the current container task;

[0056] If, for the current container task, there is no node in the corresponding node list that can meet the conditions of the best matching principle, then the container tasks on each node are paused in sequence according to the order of the corresponding node list. When pausing the container tasks on a node, according to the order in which the container tasks are matched to the node, the container tasks with general priority and the same type as the current container task are paused (that is, in the order in which the container tasks are matched to the node, traverse each container task in turn to check whether it is a container task with general priority and the same type as the current container task. The container task that is matched to the node first is traversed first, and the container task that is matched to the node later is traversed later. When a container task is matched to a node, it means that the node is used as the best node for the container task) until there is a node that can meet the conditions of the best matching principle;

[0057] If the scheduling component retrieves that the best node that is matched is in an out-of-position state, then a new best node is matched for the container task on the out-of-position best node according to the preset best matching principle; for the container task after it is matched to the best node, it is stored in the database (that is, the best node corresponding to the container task is stored, and when the best node is switched later, it is updated in real time in the database).

[0058] In this embodiment, the scheduling component monitors in real time the resource usage during the operation of the container tasks on each node reported by the monitoring component, judges the relationship between the resource usage of the container tasks and the configuration parameters of the container tasks, and when dynamically scheduling each container task, operates in the following manner:

[0059] For the CPU usage and memory usage of container tasks on a node, when at least one of the actual usages exceeds the first preset range of the estimated usage, and at the same time the corresponding current margin in the resource load information of the node exceeds the second preset value of the estimated usage, the scheduling component expands the estimated usage corresponding to the container task to the corresponding actual usage (that is, it should be noted that when the actual CPU usage of the container task on the node exceeds the first preset range of the estimated CPU usage (or the actual memory usage exceeds the first preset range of the estimated memory usage), and at the same time the current CPU margin in the resource load information of the node exceeds the second preset value of the estimated CPU usage (or the current memory margin exceeds the second preset value of the estimated memory usage), the scheduling component expands the estimated CPU usage (or estimated memory usage) of the container task to the corresponding actual usage. The specific expansion operation for the container task is as follows: Pause the original container task, create a new container task, and the new container task is configured with the actual CPU usage (or actual memory usage) of the original container task. After successful creation, delete the original container task, and name the task name of the new container task as that of the original container task, and then start the new container task); in this embodiment, the first preset range is 10% - 30%, and the second preset value is 30%.

[0060] For the CPU usage, disk usage, and memory usage of container tasks on a node, when at least one of the actual usages exceeds the first preset range of the estimated usage, and at the same time the corresponding current margin in the resource load information of the node is less than the third preset value, or when at least one of the actual usages exceeds the second preset value of the estimated usage, the scheduling component pauses the operation of the container task (that is, it should be noted that when the actual CPU usage of the container task on the node exceeds the first preset range of the estimated CPU usage (or the actual memory usage exceeds the first preset range of the estimated memory usage, or the actual disk usage exceeds the first preset range of the estimated disk usage), and at the same time each margin in the resource load information of the node is less than the third preset value; or, when the actual CPU usage of the container task on the node exceeds the second preset value of the estimated CPU usage (or when the actual disk usage of the container task on the node exceeds the second preset value of the estimated disk usage, when the actual memory usage of the container task on the node exceeds the second preset value of the estimated memory usage), the scheduling component pauses the operation of the container task), and matches a new best node for the container task according to the preset best matching principle. After the container task runs on the new best node, delete the container task on the original best node; in this embodiment, the third preset value is 10%.

[0061] In this embodiment, when the scheduling component runs container tasks on the best node, it operates in the following manner:

[0062] If there is no image for the container task to be run on the current best node, the scheduling component first obtains the image name corresponding to the container task from the database, then obtains the corresponding image from the image repository to the current best node according to the image name, and creates and runs the container task according to the image of the container task;

[0063] If there is an image for the container task to be run on the current best node, the scheduling component directly creates and runs the container task according to the image of the container task.

[0064] In this embodiment, when the scheduling component runs the container task on the best node, if there is already a container task with the same task name on other nodes, before creating the container task on this best node, the scheduling component pauses the running of the container task with the same task name on other existing nodes, and does not resume until after creating and running the container task on this best node, and then deletes the container task with the same task name on other existing nodes.

[0065] In this embodiment, the scheduling component also feeds back the scheduling result to the service component. If the container task runs successfully, the scheduling result is scheduling success; if the container task runs fails or the container task creation fails, the scheduling result is scheduling failure, and the service component updates the scheduling status of each container task (where the scheduling status includes scheduling success, scheduling failure, and unscheduled) in the database according to the fed-back scheduling result.

[0066] In another embodiment, as Figure 2 shown, based on a load-aware scheduling system for multi-dimensional resources, a load-aware scheduling method for multi-dimensional resources is proposed, and the load-aware scheduling method for multi-dimensional resources is applied to the load-aware scheduling system for multi-dimensional resources;

[0067] The load-aware scheduling method for multi-dimensional resources includes:

[0068] Each monitoring component periodically collects the resource load information and the on-site status of its own node, and reports them to the service component;

[0069] The service component receives the issuance of the container task list, stores the configuration parameters of each container task in the container task list in the database, and at the same time stores the resource load information and the on-site status of each node reported by each monitoring component in the database, and sends task scheduling information to the scheduling component;

[0070] After receiving the task scheduling information, the scheduling component searches the database, sorts each container task according to the configuration parameters to obtain a task list, and sorts each node according to the resource load information and the configuration parameters of each container task to obtain a node list corresponding to each container task. In addition, according to the preset best matching principle, the best nodes are matched to the container tasks in the task list in turn, and the container tasks are run on the best nodes.

[0071] Each monitoring component periodically collects the resource usage of each container task on the node during operation and reports it to the service component and scheduling component. The service component stores the resource usage of the container task on each node during operation in the database.

[0072] The scheduling component monitors the resource usage of container tasks on each node reported by the monitoring component in real time, determines the relationship between the resource usage of container tasks and the configuration parameters of container tasks, and dynamically schedules each container task.

[0073] For the specific limitations on the load-aware scheduling method for multi-dimensional resources, please refer to the limitations on the load-aware scheduling system for multi-dimensional resources mentioned above, which will not be repeated here.

[0074] The load-aware scheduling system and method for multi-dimensional resources sorts each container task and each node before scheduling the container task, and matches the best node for each container task according to the preset best matching principle, thereby solving the imbalance problem that most of the container tasks are scheduled to the same node for operation, resulting in excess resources on some nodes and overload on some nodes. In the best matching principle, not only does it just meet the resource requirements of the container task in the three dimensions of CPU, disk and memory, but it also reserves 10% of the resource margin for the container task. After the estimated resource requirements of the container task match the best node, it will also be dynamically scheduled according to the actual resource usage required when the container task is running, which not only solves the problem that the estimated resource requirements of the container task are too small, resulting in insufficient actual resource usage required when the container task is running, and ensures that the container task has a sufficient operating environment, but also solves the problem that the actual resources required by the container task exceed the estimated resource requirements by too much, resulting in overload on the node.

[0075] It should be understood that although Figure 2 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 2At least a part of the steps therein may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed and completed at the same moment, but can be executed at different moments, and the execution order of these sub-steps or stages is not necessarily sequential, but can be executed alternately or in turn with at least a part of other steps or sub-steps or stages of other steps.

[0076] The above-described embodiments only express several implementation manners of the present application, and their descriptions are relatively specific and detailed, but should not be construed as limiting the scope of the invention patent. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several deformations and improvements can still be made, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the appended claims.

Claims

1. A load-aware scheduling system for multi-dimensional resources, characterized by: The load-aware scheduling system for multi-dimensional resources is applied to a cluster platform, which includes a master node and multiple cluster nodes. The master node is provided with a service component, a scheduling component and a monitoring component, and each cluster node is provided with a monitoring component, wherein: Each of the monitoring components is used to periodically collect resource load information of the node, the in-place status of the node, and the resource usage of each container task on the node during operation, and report it to the service component, and at the same time report the resource usage of each container task on the node during operation to the scheduling component; The service component is used to receive the issued container task list, store the configuration parameters of each container task in the container task list to the database, store the resource load information of each node reported by each monitoring component, the in-place status of each node and the resource usage of the container task on each node during operation to the database, and send the task scheduling information to the scheduling component; The scheduling component is used to search the database after receiving the task scheduling information, sort each container task according to the configuration parameters to obtain a task list, and sort each node according to the resource load information and the configuration parameters of each container task to obtain a node list corresponding to each container task, and according to the preset best matching principle, match the best node for the container task in the task list in turn, and run the container task on the best node, and at the same time monitor the resource usage of the container task on each node reported by the monitoring component in real time during the operation, judge the relationship between the resource usage of the container task and the configuration parameters of the container task, and dynamically schedule each container task.

2. The load-aware scheduling system for multi-dimensional resources according to claim 1, characterized in that: The resource load information of each of the nodes includes the current CPU margin, the current disk margin and the current memory margin, and the resource usage during the operation of each of the container tasks includes the actual CPU usage, the actual disk usage and the actual memory usage.

3. The load-aware scheduling system for multi-dimensional resources according to claim 2, characterized in that: The configuration parameters of each container task include a task name, an image name, resource requirements, a weight of resource requirements and a priority, wherein the resource requirements include an estimated CPU usage, an estimated disk usage and an estimated memory usage for running each container task, wherein the weight of resource requirements includes a CPU weight, a disk weight and a memory weight, wherein the priority is divided into general, important and urgent, and the priority is ranked from high to low as urgent>important>general, wherein the image of the container task corresponding to the image name is stored in the image repository; At the same time, the service component classifies the container tasks in the container task list according to the weight of resource requirements. When the CPU weight of the container task is the highest, the type of the container task is CPU intensive. When the disk weight of the container task is the highest, the type of the container task is disk intensive. When the memory weight of the container task is the highest, the type of the container task is memory intensive.

4. The load-aware scheduling system for multi-dimensional resources as claimed in claim 3, characterized in that: The scheduling component is used to retrieve the database after receiving the task scheduling information, sort each container task according to the configuration parameters to obtain a task list, and sort each node according to the resource load information and the configuration parameters of each container task. When the node list corresponding to each container task is obtained, it operates as follows: The scheduling component sorts the container tasks from high to low according to the priority order from high to low in the configuration parameters of each container task in the database. When the priorities are the same, the tasks are sorted from high to low according to their comprehensive scores. After sorting, a task list is obtained, where the comprehensive score of the task = estimated CPU usage * CPU weight + estimated disk usage * disk weight + estimated memory usage * memory weight; For each container task, the scheduling component calculates the node comprehensive score according to the resource load information and the configuration parameters of the container task, and sorts the nodes from high to low according to the node comprehensive score, and obtains the node list corresponding to each container task after sorting, where the node comprehensive score = current CPU margin * CPU weight + current disk margin * disk weight + current memory margin * memory weight.

5. The load-aware scheduling system for multi-dimensional resources according to claim 3, characterized in that: The scheduling component matches the best node for each container task according to the preset best matching principle, and operates as follows: The conditions of the best matching principle are: the current CPU margin of the node is greater than or equal to 1.1 times the estimated CPU usage of the current container task, and the current disk margin of the node is greater than or equal to 1.1 times the estimated disk usage of the current container task, and the current memory margin of the node is greater than or equal to 1.1 times the estimated memory usage of the current container task, and the node comprehensive score is greater than or equal to the task comprehensive score; For each container task, determine whether each node meets the best matching principle in the order of the corresponding node list, and take the node that first meets the best matching principle as the best node for the current container task; If there is no node in the corresponding node list that can meet the best matching principle conditions for the current container task, the container tasks on each node are suspended in sequence according to the order of the corresponding node list. When suspending the container tasks on the nodes, the container tasks with general priority and consistent with the current container task type are suspended in the order in which the container tasks are matched to the nodes until there is a node that can meet the best matching principle conditions; If the scheduling component retrieves the best matching node that is not in place, a new best node is matched for the container task on the best node that is not in place according to the preset best matching principle.

6. The load-aware scheduling system for multi-dimensional resources according to claim 3, characterized in that: The scheduling component monitors the resource usage of the container tasks on each node reported by the monitoring component in real time during operation, determines the relationship between the resource usage of the container tasks and the configuration parameters of the container tasks, and performs dynamic scheduling of each container task in the following manner: For the CPU usage and memory usage of the container task on the node, when at least one of the actual usage exceeds the first preset range of the estimated usage, and the corresponding current margin in the resource load information of the node exceeds the second preset value of the estimated usage, the scheduling component expands the estimated usage corresponding to the container task to the corresponding actual usage; For the CPU usage, disk usage and memory usage of the container task on the node, when at least one of the actual usage exceeds the first preset range of the estimated usage, and the corresponding current margin in the resource load information of the node is less than the third preset value, or when at least one of the actual usage exceeds the second preset value of the estimated usage, the scheduling component suspends the operation of the container task and matches a new best node for the container task according to the preset best matching principle. After the container task runs on the new best node, the container task on the original best node is deleted.

7. The load-aware scheduling system for multi-dimensional resources according to claim 1, characterized in that: When the scheduling component runs the container task on the best node, it operates as follows: If the image of the container task to be run does not exist on the current best node, the scheduling component first obtains the image name corresponding to the container task from the database, then obtains the corresponding image from the image repository to the current best node based on the image name, and creates and runs the container task based on the image of the container task; If the image of the container task to be run exists on the current best node, the scheduling component directly creates and runs the container task based on the image of the container task.

8. The load-aware scheduling system for multi-dimensional resources according to claim 7, characterized in that: When the scheduling component runs a container task on the best node, if a container task with the same task name already exists on other nodes, the scheduling component will suspend the running of the container task with the same task name already existing on other nodes before creating the container task on the best node, and delete the container task with the same task name already existing on other nodes after the container task is created and run on the best node.

9. The load-aware scheduling system for multi-dimensional resources according to claim 7, characterized in that: The scheduling component also feeds back the scheduling result to the service component, and if the container task runs successfully, the scheduling result is scheduling success; if the container task fails to run or the container task creation fails, the scheduling result is scheduling failure, and the service component updates the scheduling status of each container task in the database according to the feedback scheduling result.

10. A load-aware scheduling method for multi-dimensional resources, characterized in that: The load-aware scheduling method for multi-dimensional resources is applied to the load-aware scheduling system for multi-dimensional resources described in any one of claims 1 to 9; The load-aware scheduling method for multi-dimensional resources includes: Each of the monitoring components periodically collects resource load information of the node and the in-place status of the node, and reports it to the service component; The service component receives the container task list, stores the configuration parameters of each container task in the container task list to the database, stores the resource load information of each node and the in-place status of each node reported by each monitoring component to the database, and sends the task scheduling information to the scheduling component; After receiving the task scheduling information, the scheduling component searches the database, sorts each container task according to the configuration parameters to obtain a task list, and sorts each node according to the resource load information and the configuration parameters of each container task to obtain a node list corresponding to each container task, and matches the best node for the container tasks in the task list in turn according to the preset best matching principle, and runs the container task on the best node; Each of the monitoring components periodically collects resource usage during the operation of each container task on the node, and reports it to the service component and the scheduling component, and the service component stores and updates the resource usage during the operation of the container task on each node in the database; The scheduling component monitors the resource usage during the operation of the container tasks on each node reported by the monitoring component in real time, determines the relationship between the resource usage of the container tasks and the configuration parameters of the container tasks, and dynamically schedules each container task.

Images