Multi-tenant resource scheduling method and system based on logic cluster

Abstract

The invention relates to a multi-tenant resource scheduling method and system based on a logic cluster, and the method comprises the steps: dividing a physical cluster: dividing the physical cluster into a plurality of logic clusters, each logic cluster having different resource characteristics and related scheduling strategies; defining a multi-tenant client, and making task type labels, affiliation relationships, dependency relationships and task ID marks for different tasks submitted by tenants by using the multi-tenant client; task scheduling; comprising the steps that after a plurality of tasks are sent to a certain logic cluster, all the tasks are queued to wait for resource allocation, and a priority scheduling strategy is adopted in the waiting stage; and resource allocation: allocating a task to be processed to a corresponding node in the logic cluster for processing. According to the method, logic clusters of different emphasized tasks are introduced, so that various tasks are planned to be allocated to required nodes, and the resource utilization rate and the task allocation efficiency are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of cloud computing and resource scheduling, and in particular to a multi-tenant resource scheduling method and system based on a logical cluster. Background Art

[0002] Currently, Kubernetes (K8s) has become a standard tool for container orchestration and is widely used in various cloud computing and data center environments. With the popularity of multi-tenant environments, how to efficiently and securely manage and schedule resources has become a major challenge.

[0003] The traditional K8s architecture will cause uneven resource allocation and resource grabbing problems in the case of multi-tenant and multi-task, resulting in low resource utilization, and cannot fully meet the resource allocation needs of different tenants and different tasks. In the traditional K8s architecture, resource management in a multi-tenant environment is often based on the physical cluster or virtual machine level, which has problems such as low resource utilization, fierce resource competition, and low security. Summary of the invention

[0004] In view of the shortcomings of the existing technology, the present invention proposes a multi-tenant resource scheduling method based on logical clusters. By introducing the concept of logical clusters in the K8S cluster, flexible resource scheduling and efficient management in a multi-tenant environment are achieved.

[0005] The present invention also proposes a multi-tenant resource scheduling system based on logical clusters.

[0006] Terminology explanation:

[0007] Logical cluster: A virtual cluster created by dividing the physical cluster through software-defined methods. These virtual clusters run independently and have specific resources and configurations to meet different computing and storage requirements, providing more flexible resource management and isolation capabilities. At the same time, they can dynamically expand and shrink capacity by accepting and transferring nodes according to their own load conditions, and ensure that their own logical clusters always hold a certain amount of required resources.

[0008] In order to achieve the above object, the present invention is implemented through the following technical solutions:

[0009] A first aspect of the present invention provides a multi-tenant resource scheduling method based on a logical cluster;

[0010] The multi-tenant resource scheduling method based on logical clusters includes:

[0011] Physical cluster division: divide the physical cluster into multiple logical clusters, each of which has different resource characteristics and related scheduling policies;

[0012] Define a multi-tenant client, and use the multi-tenant client to tag the task type, affiliation, dependency, and task ID for different tasks submitted by tenants; including: using the multi-tenant client to complete the registration of tenants and tag the tasks submitted by tenants with different type tags, dependency tags, affiliation tags, and task ID tags, and assign them to different logical clusters according to the tags;

[0013] Task scheduling: When several tasks are sent to a logical cluster, all tasks queue up and wait for resource allocation. The priority scheduling strategy is adopted in the waiting stage. Three queues of high priority, secondary priority and low priority are set. Each task enters the corresponding priority queue according to its own task ID.

[0014] Resource allocation: Allocate the tasks to be processed to the corresponding nodes in the logical cluster for processing; including: using the load formula to calculate the load of each node; for each task, select the node with the lowest load for resource allocation.

[0015] Preferably, according to the present invention, a physical cluster is divided into a plurality of logical clusters, each logical cluster having different resource characteristics and related scheduling strategies; including:

[0016] Divide the physical cluster into four logical clusters for different task types at the logical level, including CPU-tight logical cluster, memory-tight logical cluster, storage-tight logical cluster, and balanced logical cluster. Each logical cluster handles different types of tasks.

[0017] Each logical cluster has a master node and several worker nodes.

[0018] The master node is responsible for task mobilization and resource allocation within the cluster, and periodically monitors the load of all working nodes;

[0019] Work nodes are divided into three categories, including fixed work nodes, mobile work nodes and emergency nodes;

[0020] Each logical cluster has several fixed working nodes and labels indicating their affiliation. The working nodes always belong to the logical cluster and process task requests on the logical cluster.

[0021] At the same time, each logical cluster sets up several mobile work nodes and sets labels to indicate their affiliation. The mobile work nodes process the task requests of the logical cluster. If the utilization rate of a certain resource of a mobile work node is greater than the sum of the resource utilization rates of the fixed work nodes and the emergency nodes, and the utilization rate of the resource exceeds 70%, the node will be adopted to the logical cluster with the lowest resource utilization rate, and the label of the node will be changed.

[0022] At the same time, each logical cluster is also equipped with several emergency nodes. When the load in a logical cluster increases suddenly, the emergency node is put into use and returns to the original logical cluster after completion.

[0023] Further preferably, the node label is set as follows:

[0024] The label includes a 4-digit identifier. The first digit indicates whether it is an emergency node. 0 indicates a normal working node, and 1 indicates an emergency node. The second digit indicates whether it is a fixed working node. 0 indicates a fixed working node, and 1 indicates a mobile working node. The third and fourth digits indicate which logical cluster it currently belongs to. 00 indicates a balanced logical cluster, 01 indicates a CPU-tight logical cluster, 10 indicates a memory-tight logical cluster, and 11 indicates a storage-tight logical cluster.

[0025] Preferably, according to the present invention, a multi-tenant client is used to complete the registration of tenants and tasks submitted by tenants are tagged with different types of tags and dependency tags, and are assigned to different logical clusters according to the tags; including:

[0026] First, the multi-tenant client completes tenant registration for different tenants;

[0027] Then, the tenant submits the task on the multi-tenant client. The submitted task information includes the size of the CPU, memory, and storage resources required for the task, as well as the dependencies between the task and other tasks.

[0028] Finally, the type of the task is calculated and a task type label is added to the task. The task type label also includes which tenant the task belongs to, whether the task has any dependent predecessor tasks, and the task ID.

[0029] Further preferably, the task type label is defined as follows:

[0030] Each task type label consists of 8 digits. The first and second digits indicate the type of task the task belongs to, where 00 indicates balanced type, 01 indicates CPU-intensive type, 10 indicates memory-intensive type, and 11 indicates storage-intensive type. The third and fourth digits indicate the tenant, indicating which tenant the task comes from. The fifth and sixth digits indicate the dependency of the task, where 00 indicates no dependency and is executed immediately, 01 indicates dependency on task 00 and requires the tenant's task 00 to be completed before continuing execution, 10 indicates dependency on task 01 and 11 indicates dependency on task 10. The seventh and eighth digits indicate the task ID.

[0031] Preferably, according to the present invention, each task enters a corresponding priority queue according to its own task identifier; including:

[0032] The dependency relationship is 00, that is, the tasks whose fifth and sixth digits of the task type label are 00 are in the high priority queue, the dependency relationship is 01, that is, the tasks whose fifth and sixth digits of the task type label are 01 are in the secondary priority queue, and the dependencies are 10 and 11, that is, the tasks whose fifth and sixth digits of the task type label are 10 and 11 are in the low priority queue.

[0033] Further preferably, the master node periodically calculates the load value of all nodes in the cluster, that is, the load condition Load, and the calculation formula of the load value Load is as follows:

[0034] Load=loadcpu*W1+load MEM *W2+load STOR *W3;

[0035] loadcpu=CPUused / CPUtotal;

[0036] load MEM =MEMused / MEMtotal;

[0037] load STOR =STORused / STORtotal;

[0038] Among them, loadcpu refers to the CPU load in the node, CPUused refers to the used resources of the node CPU, and CPUtotal refers to the total CPU resources of the node; similarly, load MEM It is the load of the node's memory resources, MEMused is the memory resources used by the node, and MEMtotal is the total memory resources of the node; load STOR It is the load of the node's storage resources, STORused is the storage resources used by the node, and STORtotal is the total storage resources of the node.

[0039] Further preferably, in a CPU tight logic cluster, W1 is 0.6, W2 is 0.2, and W3 is 0.2.

[0040] In the memory-intensive logical cluster, W1 is 0.2, W2 is 0.6, and W3 is 0.2.

[0041] In a storage-intensive logical cluster, W1 is 0.2, W2 is 0.2, and W3 is 0.6.

[0042] In a balanced logical cluster, W1 is 1 / 3, W2 is 1 / 3, and W3 is 1 / 3.

[0043] Further preferably, if the load conditions of the fixed working node and the mobile working node are the same, the mobile working node is used preferentially.

[0044] A second aspect of the present invention provides a multi-tenant resource scheduling system based on logical clusters;

[0045] The multi-tenant resource scheduling system based on logical clusters includes:

[0046] The physical cluster partitioning module is configured to: partition the physical cluster into a plurality of logical clusters, each logical cluster having different resource characteristics and related scheduling strategies;

[0047] The multi-tenant client module is configured to: define a multi-tenant client, and use the multi-tenant client to tag the task type, subordination, dependency and task ID for different tasks submitted by tenants; including: using the multi-tenant client to complete the registration of tenants and tag the tasks submitted by tenants with different type tags, dependency tags, subordination tags and task ID tags, and assign them to different logical clusters according to the tags;

[0048] The task scheduling module is configured as follows: when a number of tasks are sent to a certain logical cluster, all tasks queue up to wait for resource allocation, and a priority scheduling strategy is adopted in the waiting stage, and three queues of high priority, secondary priority and low priority are set. Each task enters the corresponding priority queue according to its own task identifier;

[0049] The resource allocation module is configured to: allocate the tasks to be processed to the corresponding nodes in the logical cluster for processing; including: using the load formula to calculate the load of each node; and for each task, selecting the node with the lowest load for resource allocation.

[0050] The beneficial effects of the present invention are:

[0051] 1. The present invention introduces logical clusters with different focuses on tasks, so that various tasks can be allocated to required nodes in a planned manner, thereby improving resource utilization and task allocation efficiency.

[0052] 2. The present invention labels different nodes and tasks respectively, so that various tasks can be accurately allocated to required nodes to avoid waste of resources.

[0053] 3. The present invention allows working nodes to flow at the logical cluster level, which can improve the resource utilization rate of the nodes and avoid the situation where a certain type of resource is heavily loaded but other resources are underutilized.

[0054] 4. The present invention is provided with a mobile emergency node, which can handle sudden mission peak situations. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 It is a structural block diagram of the multi-tenant resource scheduling system based on logical clusters of the present invention;

[0056] Figure 2 It is a flowchart of a multi-tenant resource scheduling method based on a logical cluster of the present invention;

[0057] Figure 3 It is a flowchart of the multi-tenant client of the present invention;

[0058] Figure 4 Schematic diagram of a logical cluster of the present invention. DETAILED DESCRIPTION

[0059] The present invention will be further defined below in conjunction with the accompanying drawings and embodiments, but is not limited thereto.

[0060] Example 1

[0061] Multi-tenant resource scheduling methods based on logical clusters, such as Figure 2 As shown, including:

[0062] Physical cluster division: divide the physical cluster into multiple logical clusters, each of which has different resource characteristics and related scheduling policies;

[0063] On the other hand, a multi-tenant client is defined for the system, and the multi-tenant client is used to tag the task type, affiliation, dependency and task ID for different tasks submitted by tenants; including: using the multi-tenant client to complete the registration of tenants and tag the tasks submitted by tenants with different type tags, dependency tags, affiliation tags and task ID tags, and assigning them to different logical clusters according to the tags;

[0064] Task scheduling: When several tasks are sent to a logical cluster with a certain focus, all tasks queue up and wait for resource allocation. The priority scheduling strategy is adopted in the waiting stage, and three queues of high priority, secondary priority and low priority are set. Each task enters the corresponding priority queue according to its own task ID.

[0065] Resource allocation: Allocate the tasks to be processed to the corresponding nodes in the logical cluster for processing; including: using the load formula to calculate the load of each node; for each task, select the node with the lowest load for resource allocation.

[0066] Example 2

[0067] The difference between the multi-tenant resource scheduling method based on logical clusters described in Example 1 is that:

[0068] The physical cluster is divided into multiple logical clusters, each with different resource characteristics and related scheduling strategies; including: Specifically, a mechanism for logically dividing clusters is designed to divide the original physical cluster into four logical clusters for different task types at the logical level; including: CPU-tight logical cluster, memory-tight logical cluster, storage-tight logical cluster and balanced logical cluster, each of which handles different types of tasks;

[0069] Because a cluster is composed of several nodes, the division of logical clusters is essentially a logical division of nodes, which does not change the physical location of the nodes, but only affects task allocation and resource allocation.

[0070] like Figure 4 As shown in the figure, each logical cluster has a master node and several worker nodes. The master node is responsible for task mobilization and resource allocation within the cluster, and periodically monitors the load of all worker nodes. The worker nodes are divided into three categories, including fixed worker nodes, mobile worker nodes, and emergency nodes.

[0071] Each logical cluster has several fixed working nodes and labels indicating their affiliation. The working nodes always belong to the logical cluster and process task requests on the logical cluster.

[0072] At the same time, each logical cluster sets up several mobile work nodes and sets labels to indicate their subordinates. The mobile work nodes process the task requests of the logical cluster. If the utilization rate of a certain resource of a mobile work node is greater than the sum of the resource utilization rates of the fixed work nodes and the emergency nodes, and the utilization rate of the resource exceeds 70%, the node will be adopted to the logical cluster with the smallest resource utilization rate, and the label of the node will be changed. For example, a mobile work node a was originally a work node of a CPU-intensive logical cluster. During a period of time, CPU-intensive tasks were assigned to it, making its CPU resource utilization rate far greater than memory and storage resources. At this time, the master node calculated that the memory utilization rate was the smallest among the three resource utilization rates, so the node was adopted to the memory-intensive logical cluster to prepare to accept memory-intensive tasks.

[0073] Assume that a node is originally labeled 0101, which is a non-emergency mobile CPU-intensive node. Assume that its CPU usage is 80%, memory usage is 50%, and storage usage is 20%. At this time, the node is overloaded and should not be assigned new tasks. The master node changes its label to 0111 and transfers it to the storage-intensive logical cluster. At this time, a new load balancing calculation formula (the load balancing calculation formula is given later) is obtained to accept new types of tasks, ensuring the effective use of resources.

[0074] Both mobile worker nodes and fixed worker nodes process task requests from logical clusters, but they differ in their working meanings. Mobile worker nodes are designed to quickly absorb more tasks so that they can reach the load threshold and be dispatched to other logical clusters, thereby improving utilization. Fixed worker nodes are designed to prevent all nodes in the cluster from being dispatched out. Their role is to ensure that there are always worker nodes in a logical cluster.

[0075] At the same time, each logical cluster is also equipped with several emergency nodes, which are not normally assigned tasks. When the load in a logical cluster suddenly increases, the emergency nodes are put into use and return to the original logical cluster after completion.

[0076] The node labels are set as follows:

[0077] The label includes a 4-digit identifier. The first digit indicates whether it is an emergency node, 0 indicates a normal working node, and 1 indicates an emergency node; the second digit indicates whether it is a fixed working node, 0 indicates a fixed working node, and 1 indicates a mobile working node; the third and fourth digits indicate which logical cluster it currently belongs to, 00 indicates a balanced logical cluster, 01 indicates a CPU-tight logical cluster, 10 indicates a memory-tight logical cluster, and 11 indicates a storage-tight logical cluster. In this way, different nodes can be clearly divided into different logical clusters, and nodes can be seconded between different clusters, which helps to improve resource utilization and handle emergencies.

[0078] Use the multi-tenant client to complete the registration of tenants and label the tasks submitted by tenants with different types of tags and dependency tags, and assign them to different logical clusters according to the tags; Figure 3 As shown, including:

[0079] Specifically, first, the multi-tenant client completes tenant registration for different tenants;

[0080] Then, the tenant submits the task on the multi-tenant client. The submitted task information includes the size of the CPU, memory, and storage resources required for the task, as well as the dependencies between the task and other tasks.

[0081] Finally, the type of the task is calculated and a task type label is added to the task. The task type label also includes which tenant the task belongs to, whether the task has any dependent predecessor tasks, and the task ID.

[0082] The task type tags are defined as follows:

[0083] Each task type label consists of 8 digits. The first and second digits indicate the type of task the task belongs to, where 00 indicates balanced type, 01 indicates CPU-intensive type, 10 indicates memory-intensive type, and 11 indicates storage-intensive type. The third and fourth digits indicate the tenant, indicating which tenant the task comes from. The fifth and sixth digits indicate the dependency of the task, 00 indicates no dependency and immediate execution, 01 indicates dependency 00 completion and requires the tenant's 00 task to be completed before continuing execution, 10 indicates dependency 01 completion, and 11 indicates dependency 10 completion. The seventh and eighth digits indicate the task ID. Prevent tasks from having the same label. Different types of tasks are sent to different logical clusters with different emphasis types to wait for the next task scheduling and resource allocation. Different types of tasks are sent to different logical clusters with different emphasis types to wait for the next task scheduling and resource allocation. Assume that tenant 01 submits three tasks A, B, and C. All three tasks are CPU-intensive, and both B and C depend on the completion of A. Therefore, the label of A is set to 01010001, the label of B is set to 01010102, and the label of C is set to 01010103. Through this design, we can clearly know the type of task, the tenant ID of the task, the dependency relationship between tasks, and clearly locate each task ID, which is convenient for subsequent task scheduling.

[0084] The task type is calculated as follows:

[0085] The resource configuration of the common general-purpose node Microsoft Azure (D_v5 series) on the market is used as the standard, and its resource configuration is {cpu: memory: storage} = {4 cores: 16G: 64G}. The resource ratio of the task request and the standard is calculated. When the resource ratio of a type a is not less than the sum of the ratios of the other two resources bc, the task is classified as a type a task.

[0086] For example:

[0087] The standard configuration is shown in Table 1:

[0088] Table 1

[0089]

[0090] Task A is shown in Table 2:

[0091] Table 2

[0092]

[0093] At this time, the CPU accounts for 75%, the memory accounts for 12.5%, and the storage accounts for 15.6%. The CPU accounts for more than the sum of the memory and storage accounts, so task A is a CPU-intensive task. In particular, if a task does not meet the requirement that the proportion of one type of resource is not less than the sum of the other two types of resources, the task is classified as a balanced task.

[0094] Each task enters the corresponding priority queue according to its own task ID; including:

[0095] The task scheduling stage adopts a multi-priority classification scheduling strategy. Specifically, different types of tasks are assigned to different types of logical clusters according to their labels, and each logical cluster contains multiple priority sequences, including high priority, secondary priority and low priority. Each task is assigned to a different priority sequence according to the dependency sequence number of its own label. The task with a dependency of 00, that is, the fifth and sixth digits of the task type label are 00, is in the high priority queue because there is no predecessor task that can be executed immediately. The task with a dependency of 01, that is, the fifth and sixth digits of the task type label are 01, is in the secondary priority queue, and the tasks with dependencies of 10 and 11, that is, the fifth and sixth digits of the task type label are 10 and 11, are in the low priority queue. If a tenant's task with a dependency of 00 is executed, the tenant's task with a dependency of 01 will enter the high priority queue, and the task with a dependency of 10 will enter the secondary priority queue, and so on, until all tasks are scheduled.

[0096] Assume that a tenant 02 has three tasks, namely Task A: 01020001; Task B: 02020002; Task C: 01020103. Therefore, tasks A, B, and C are respectively in the high priority queue of the CPU-tight logical cluster, the high priority queue of the memory-tight logical cluster, and the sub-priority queue of the CPU-tight type. The master node will monitor all tasks of tenant 02. Only when all 00-dependent tasks of tenant 02 are completed, the sub-priority tasks will be placed in the high priority queue waiting for resource allocation, and all teams in the high priority queue will be allocated resources in order of priority.

[0097] Resource allocation focuses on which node to assign the task to. For each logical cluster, the master node in the cluster calculates the load value of all working nodes except the emergency node, and uses the node with the smallest load value to process the task. If the fixed working nodes and the mobile working nodes in the cluster have the same load value, the mobile working nodes are used first to facilitate their maximum load value to be transferred to other logical clusters, thereby improving their resource utilization. If the remaining nodes are unable to handle the excess tasks or the entire logical cluster reaches the load threshold, the master node can put its own emergency nodes and the emergency nodes of other logical clusters into use, and release the emergency nodes after the task peak period passes, thereby preventing the logical cluster from collapsing when facing sudden peak tasks.

[0098] Load balancing is used to calculate the load values ​​of nodes and logical clusters and determine whether they are overloaded. The master node periodically calculates the load values ​​of all nodes in the cluster, i.e., the load status Load, and dispatches tasks based on this. The calculation formula for the load value Load is as follows:

[0099] Load=loadcpu*W1+load MEM *W2+load STOR *W3;

[0100] loadcpu=CPUused / CPUtotal;

[0101] load MEM =MEMused / MEMtotal;

[0102] load STOR =STORused / STORtotal;

[0103] Among them, loadcpu refers to the CPU load in the node, CPUused refers to the used resources of the node CPU, and CPUtotal refers to the total CPU resources of the node; similarly, load MEM It is the load of the node's memory resources, MEMused is the memory resources used by the node, and MEMtotal is the total memory resources of the node; load STOR It is the load of the node's storage resources, STORused is the storage resources used by the node, and STORtotal is the total storage resources of the node.

[0104] In the CPU-tight logical cluster, W1 is 0.6, W2 is 0.2, and W3 is 0.2.

[0105] In the memory-intensive logical cluster, W1 is 0.2, W2 is 0.6, and W3 is 0.2.

[0106] In a storage-intensive logical cluster, W1 is 0.2, W2 is 0.2, and W3 is 0.6.

[0107] In a balanced logical cluster, W1 is 1 / 3, W2 is 1 / 3, and W3 is 1 / 3.

[0108] Through this calculation method with different weights, nodes that reach the load threshold in a certain logical cluster can be adopted to another logical cluster to make full use of their unused resources, greatly improving resource utilization. On the other hand, in addition to calculating the load value of each working node, the master node also calculates the load value of the entire cluster. When the cluster load value reaches the preset threshold, it will call its own emergency node and apply to call the emergency nodes of other logical clusters, thereby solving the problem of high task concurrency.

[0109] The weights of resources in different logical clusters are different. If the loads of fixed working nodes and mobile working nodes are the same, the mobile working nodes are used first. This ensures the reasonable allocation and efficient use of resources and balances the loads of each node.

[0110] Through this calculation method with different weights, nodes that reach the load threshold in a certain logical cluster can be adopted to another logical cluster to make full use of their unused resources, greatly improving resource utilization. On the other hand, in addition to calculating the load value of each working node, the master node also calculates the load value of the entire cluster. When the cluster load value reaches the preset threshold, it will call its own emergency node and apply to call the emergency nodes of other logical clusters, thereby solving the problem of high task concurrency.

[0111] Example 3

[0112] Multi-tenant resource scheduling system based on logical clusters, such as Figure 1 As shown, including:

[0113] The physical cluster partitioning module is configured to: partition the physical cluster into a plurality of logical clusters, each logical cluster having different resource characteristics and related scheduling strategies;

[0114] The multi-tenant client module is configured to: define a multi-tenant client, and use the multi-tenant client to tag the task type, subordination, dependency and task ID for different tasks submitted by tenants; including: using the multi-tenant client to complete the registration of tenants and tag the tasks submitted by tenants with different type tags, dependency tags, subordination tags and task ID tags, and assign them to different logical clusters according to the tags;

[0115] The task scheduling module is configured as follows: when a number of tasks are sent to a certain logical cluster, all tasks queue up to wait for resource allocation, and a priority scheduling strategy is adopted in the waiting stage, and three queues of high priority, secondary priority and low priority are set. Each task enters the corresponding priority queue according to its own task identifier;

[0116] The resource allocation module is configured to: allocate the tasks to be processed to the corresponding nodes in the logical cluster for processing; including: using the load formula to calculate the load of each node; and for each task, selecting the node with the lowest load for resource allocation.

[0117] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used in the present invention have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.

[0118] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

Claims

1. A multi-tenant resource scheduling method based on a logical cluster, characterized in that: include: Physical cluster division: divide the physical cluster into multiple logical clusters, each of which has different resource characteristics and related scheduling policies; Define a multi-tenant client, and use the multi-tenant client to tag the task type, affiliation, dependency, and task ID for different tasks submitted by tenants; including: using the multi-tenant client to complete the registration of tenants and tag the tasks submitted by tenants with different type tags, dependency tags, affiliation tags, and task ID tags, and assign them to different logical clusters according to the tags; Task scheduling: When several tasks are sent to a logical cluster, all tasks queue up and wait for resource allocation. The priority scheduling strategy is adopted in the waiting stage. Three queues of high priority, secondary priority and low priority are set. Each task enters the corresponding priority queue according to its own task ID. Resource allocation: Allocate the tasks to be processed to the corresponding nodes in the logical cluster for processing; including: using the load formula to calculate the load of each node; for each task, select the node with the lowest load for resource allocation.

2. The multi-tenant resource scheduling method based on logical cluster according to claim 1, characterized in that: Divide the physical cluster into multiple logical clusters, each with different resource characteristics and related scheduling policies; including: Divide the physical cluster into four logical clusters for different task types at the logical level, including CPU-tight logical cluster, memory-tight logical cluster, storage-tight logical cluster, and balanced logical cluster. Each logical cluster handles different types of tasks. Each logical cluster has a master node and several worker nodes. The master node is responsible for task mobilization and resource allocation within the cluster, and periodically monitors the load of all working nodes; Work nodes are divided into three categories, including fixed work nodes, mobile work nodes and emergency nodes; Each logical cluster has several fixed working nodes and labels indicating their affiliation. The working nodes always belong to the logical cluster and process task requests on the logical cluster. At the same time, each logical cluster sets up several mobile work nodes and sets labels to indicate their subordinates. The mobile work nodes process the task requests of the logical cluster. If the utilization rate of a certain resource of a mobile work node is greater than the sum of the resource utilization rates of the fixed work nodes and the emergency nodes, and the utilization rate of the resource exceeds 70%, the node will be adopted to the logical cluster with the lowest resource utilization rate, and the label of the node will be changed; Each logical cluster is also equipped with several emergency nodes. When the load in a logical cluster increases suddenly, the emergency nodes are put into use and return to the original logical cluster after completion.

3. The multi-tenant resource scheduling method based on logical cluster according to claim 1, characterized in that: The node labels are set as follows: The label includes a 4-digit identifier. The first digit indicates whether it is an emergency node. 0 indicates a normal working node, and 1 indicates an emergency node. The second digit indicates whether it is a fixed working node. 0 indicates a fixed working node, and 1 indicates a mobile working node. The third and fourth digits indicate which logical cluster it currently belongs to. 00 indicates a balanced logical cluster, 01 indicates a CPU-tight logical cluster, 10 indicates a memory-tight logical cluster, and 11 indicates a storage-tight logical cluster.

4. The multi-tenant resource scheduling method based on logical cluster according to claim 1, characterized in that: Use the multi-tenant client to register tenants and label the tasks submitted by tenants with different types of tags and dependency tags, and assign them to different logical clusters according to the tags; including: First, the multi-tenant client completes tenant registration for different tenants; Then, the tenant submits the task on the multi-tenant client. The submitted task information includes the size of the CPU, memory, and storage resources required for the task, as well as the dependencies between the task and other tasks. Finally, the type of the task is calculated and a task type label is added to the task. The task type label also includes which tenant the task belongs to, whether the task has any dependent predecessor tasks, and the task ID.

5. The multi-tenant resource scheduling method based on logical cluster according to claim 4, characterized in that: The task type tags are defined as follows: Each task type label consists of 8 digits. The first and second digits indicate the type of task the task belongs to, where 00 indicates balanced type, 01 indicates CPU-intensive type, 10 indicates memory-intensive type, and 11 indicates storage-intensive type. The third and fourth digits indicate the tenant, indicating which tenant the task comes from. The fifth and sixth digits indicate the dependency of the task, where 00 indicates no dependency and is executed immediately, 01 indicates dependency on task 00 and requires the tenant's task 00 to be completed before continuing execution, 10 indicates dependency on task 01 and 11 indicates dependency on task 10. The seventh and eighth digits indicate the task ID.

6. The multi-tenant resource scheduling method based on logical cluster according to claim 1, characterized in that: Each task enters the corresponding priority queue according to its own task ID; including: The dependency relationship is 00, that is, the tasks whose fifth and sixth digits of the task type label are 00 are in the high priority queue, the dependency relationship is 01, that is, the tasks whose fifth and sixth digits of the task type label are 01 are in the secondary priority queue, and the dependencies are 10 and 11, that is, the tasks whose fifth and sixth digits of the task type label are 10 and 11 are in the low priority queue.

7. The multi-tenant resource scheduling method based on logical cluster according to any one of claims 1 to 6, characterized in that: The master node periodically calculates the load value of all nodes in the cluster, that is, the load condition Load. The calculation formula of the load value Load is as follows: Load=loadcpu*W1+load MEM *W2+load STOR *W3; loadcpu=CPUused / CPUtotal; load MEM =MEMused / MEMtotal; load STOR =STORused / STORtotal; Among them, loadcpu refers to the CPU load in the node, CPUused refers to the used resources of the node CPU, and CPUtotal refers to the total CPU resources of the node; similarly, load MEM It is the load of the node's memory resources, MEMused is the memory resources used by the node, and MEMtotal is the total memory resources of the node; load STOR It is the load of the node's storage resources, STORused is the storage resources used by the node, and STORtotal is the total storage resources of the node.

8. The multi-tenant resource scheduling method based on logical cluster according to claim 7, characterized in that: In the CPU-tight logic cluster, W1 is 0.6, W2 is 0.2, and W3 is 0.2; In the memory-intensive logical cluster, W1 is 0.2, W2 is 0.6, and W3 is 0.2; In the storage-intensive logical cluster, W1 is 0.2, W2 is 0.2, and W3 is 0.6; In a balanced logical cluster, W1 is 1 / 3, W2 is 1 / 3, and W3 is 1 / 3.

9. The multi-tenant resource scheduling method based on logical cluster according to claim 7, characterized in that: If the load conditions of the fixed working node and the mobile working node are the same, the mobile working node will be used first.

10. A multi-tenant resource scheduling system based on logical clusters, characterized in that: include: The physical cluster partitioning module is configured to: partition the physical cluster into a plurality of logical clusters, each logical cluster having different resource characteristics and related scheduling strategies; The multi-tenant client module is configured to: define a multi-tenant client, and use the multi-tenant client to tag the task type, subordination, dependency and task ID for different tasks submitted by tenants; including: using the multi-tenant client to complete the registration of tenants and tag the tasks submitted by tenants with different type tags, dependency tags, subordination tags and task ID tags, and assign them to different logical clusters according to the tags; The task scheduling module is configured as follows: when a number of tasks are sent to a certain logical cluster, all tasks queue up to wait for resource allocation, and a priority scheduling strategy is adopted in the waiting stage, and three queues of high priority, secondary priority and low priority are set. Each task enters the corresponding priority queue according to its own task identifier; The resource allocation module is configured to: allocate the tasks to be processed to the corresponding nodes in the logical cluster for processing; including: using the load formula to calculate the load of each node; and for each task, selecting the node with the lowest load for resource allocation.

Images