Method, device, equipment and storage medium for container cluster scheduling

Abstract

The present application discloses a method, apparatus, device and storage medium for container cluster scheduling, and relates to the field of computer technology. The method includes: creating multiple PODs according to the resource requirements of the POD indicated by the received container scheduling request, grouping the created PODs according to the resource requirements of the PODs and the upper limit of the resources that the node group can provide, and then assigning a corresponding node group to each POD group from the node group included in the container cluster, and for each POD included in each POD group, determining the corresponding target node from the node group assigned to the POD group, and binding each POD to the corresponding target node. The container cluster scheduling method adopted in the embodiment of the present application improves the scheduling efficiency of the container cluster on the basis of satisfying the node load balancing, reduces the load pressure of the native scheduler of the container cluster, and makes the resource utilization rate of the node higher.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method, apparatus, device and storage medium for scheduling a container cluster. Background Art

[0002] In current cloud computing products, container cluster scheduling algorithms usually use container resource utilization, node load balancing, and disaster recovery strategies as the main indicators for evaluating cluster performance, and the rationality of container scheduling algorithms also highly affects cluster performance. For example, the Kubernetes container orchestration scheduling technology, which is widely used in the field of cloud computing, fully considers the above issues. On the basis of meeting the personalized resource allocation needs of users, it uses certain filtering algorithms and scoring algorithms to select the most suitable node-bound PODs. Therefore, the Kubernetes container orchestration scheduling technology has largely solved the problem of container resource utilization and has great advantages in achieving load balancing and disaster recovery. However, with the expansion of cluster scale and the increase in the number of cloud computing platform users, the Kubernetes container orchestration scheduling technology needs to allocate a large number of PODs to nodes one by one, and the efficiency of container scheduling is too low. In addition, the sequential repetitive scoring and screening of nodes will also lead to reduced efficiency. Therefore, improving the efficiency of container scheduling has become a problem that needs to be solved urgently. On the other hand, the native scheduler of Kubernetes is responsible for both node filtering and scoring, and also takes into account the load balancing and disaster recovery of nodes, resulting in excessive load pressure on the scheduler. Summary of the invention

[0003] In view of the above problems, the embodiments of the present application provide a method, apparatus, device and storage medium for scheduling a container cluster, so as to solve the technical problems that the scheduling efficiency of the container cluster is low and the load pressure of the native scheduler is too large.

[0004] On the one hand, a container cluster scheduling method is provided, the method comprising:

[0005] Creating a plurality of PODs based on the received container scheduling request, wherein the scheduling request is used to indicate resource requirements of the plurality of PODs;

[0006] Based on the resource requirements of each POD and the upper limit of the amount of resources that can be provided by each node group, the multiple PODs are grouped to obtain multiple POD groups, each POD group including at least one POD;

[0007] Assign a corresponding node group to each POD group from the node groups included in the container cluster;

[0008] For each POD included in each POD group, a corresponding target node is determined from the node group allocated to the POD group, and each POD is bound to the corresponding target node.

[0009] On the one hand, a container cluster scheduling device is provided, including:

[0010] A creating unit, configured to create a plurality of PODs based on a received container scheduling request, wherein the scheduling request is used to indicate resource requirements of the plurality of PODs;

[0011] A grouping unit, configured to group the plurality of PODs based on resource requirements of the respective PODs and an upper limit value of resource amounts that can be provided by each node group, to obtain a plurality of POD groups, each POD group including at least one POD;

[0012] The allocation unit is used to allocate a corresponding node group to each POD group from the node groups included in the container cluster;

[0013] The binding unit is used to determine, for each POD included in each POD group, a corresponding target node from the node group allocated to the POD group, and bind each POD to the corresponding target node.

[0014] Optionally, the grouping unit is specifically used to:

[0015] Based on node attributes of each node included in the container cluster, group the nodes to obtain multiple node groups;

[0016] The node attributes between multiple nodes in each node group have one or more of the following relationships:

[0017] The resource attributes are the same;

[0018] The difference between the resource utilization rates is not greater than a first preset difference threshold;

[0019] The container cluster includes nodes in at least two topological structures.

[0020] Optionally, the grouping unit is specifically used to:

[0021] Based on the nodes with specific resources in the container cluster, obtaining at least one special resource node group;

[0022] Based on the resource utilization rate of each node, clustering is performed on common resource nodes in the container cluster except nodes with specific resources, and the number of groups N is determined according to the clustering result;

[0023] The common resource nodes in each topological structure in the container cluster are sorted in ascending order of resource utilization to obtain a common resource node queue corresponding to each topological structure;

[0024] Each common resource node queue is divided into N subqueues respectively, the order of common resource nodes in each subqueue is consistent with the corresponding common resource node queue, and the minimum resource utilization rate in one of any two subqueues is greater than the maximum resource utilization rate in the other subqueue;

[0025] In the N sub-queues corresponding to each common resource node queue, the nodes included in the sub-queues corresponding to the positions are combined to obtain N common resource node groups.

[0026] Optionally, the grouping unit is specifically used to:

[0027] For each node group in the multiple node groups, respectively, a resource utilization set of each node group is obtained, wherein the resource utilization set includes the resource utilization of each node in the corresponding node group;

[0028] For each resource utilization set, if the difference between the maximum value and the minimum value included therein is greater than a second preset difference threshold, the multiple node groups are grouped and adjusted to obtain multiple adjusted node groups.

[0029] Optionally, the device further comprises a migration unit, specifically used for:

[0030] When the load of the container cluster is lower than a preset load threshold, determining whether the activity of each POD is lower than a preset activity threshold; the activity represents the probability of resource demand of the POD, or the amount of resources when the resource demand is expanded;

[0031] When the activity of a target POD is lower than the preset activity threshold, the target POD is migrated to a node whose available resource amount is lower than the preset resource amount threshold.

[0032] Optionally, the grouping unit is specifically used to:

[0033] The multiple PODs are grouped to obtain multiple POD groups, where the multiple POD groups satisfy one or more of the following conditions:

[0034] Each POD group has the same resource requirements;

[0035] The total resource demand of each POD group is less than or equal to the upper limit of the resource amount, and the difference between the total resource demand and the upper limit of the resource amount does not exceed a third preset difference threshold;

[0036] The correlation between the PODs in each POD group is not less than a preset correlation threshold.

[0037] Optionally, the allocation unit is specifically used to:

[0038] Allocating the special resource node group included in the container cluster to a POD group having specific resource requirements among the multiple POD groups;

[0039] Allocating the common resource node group included in the container cluster to other POD groups among the multiple POD groups except the POD group with specific resource requirements;

[0040] If there are other POD groups that are not allocated to corresponding common resource node groups, the special resource node group is allocated to the other unallocated POD groups.

[0041] On the one hand, a computer device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of any one of the above methods when executing the computer program.

[0042] In one aspect, a computer storage medium is provided, on which computer program instructions are stored, and when the computer program instructions are executed by a processor, the steps of any of the above methods are implemented.

[0043] In one aspect, a computer program product or a computer program is provided, the computer program product or the computer program comprising computer instructions, the computer instructions being stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the steps of any of the above methods.

[0044] The beneficial effects of the embodiments of the present application are as follows:

[0045] In the embodiment of the present application, multiple PODs are created according to the resource requirements of the POD indicated by the received container scheduling request, and the created PODs are grouped according to the resource requirements of the POD and the upper limit of the resources that the node group can provide, and then a corresponding node group is assigned to each POD group from the node group included in the container cluster, and for each POD included in each POD group, the corresponding target node is determined from the node group assigned to the POD group, and each POD is bound to the corresponding target node. Among them, the container cluster scheduling method adopted in the embodiment of the present application, by grouping PODs and nodes respectively, and then using a hierarchical scheduling mechanism of large-grained batch group scheduling from POD group to node group and POD scheduling within node group, on the basis of satisfying node load balancing, improves the scheduling efficiency of the container cluster through hierarchical scheduling, and at the same time reduces the load pressure of the native scheduler of the container cluster, so that the resource utilization rate of the node is higher. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related technologies, the drawings required for use in the embodiments or the related technical descriptions are briefly introduced below. Obviously, the drawings described below are only the embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0047] Figure 1 A schematic diagram of an application scenario provided for an embodiment of the present application;

[0048] Figure 2 A system architecture diagram of a container cluster scheduling device provided in an embodiment of the present application;

[0049] Figure 3 A flowchart of a container cluster scheduling method provided in an embodiment of the present application;

[0050] Figure 4 A schematic diagram of a two-level scheduling module for POD scheduling provided in an embodiment of the present application;

[0051] Figure 5 A schematic diagram of a node grouping process provided in an embodiment of the present application;

[0052] Figure 6 A schematic diagram of grouping nodes in a cluster topology provided in an embodiment of the present application;

[0053] Figure 7 A grouping flow chart of a node grouping module provided in an embodiment of the present application;

[0054] Figure 8 A flowchart of a container cluster scheduling process provided in an embodiment of the present application;

[0055] Fig. 9 A schematic diagram of the structure of a container cluster scheduling device provided in an embodiment of the present application;

[0056] Fig.10 A schematic diagram of the composition structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0057] In order to make the purpose, technical scheme and advantages of the present application clearer, the technical scheme in the embodiment of the present application will be clearly and completely described below in conjunction with the drawings in the embodiment of the present application. Obviously, the described embodiment is only a part of the embodiment of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application. In the absence of conflict, the embodiments in the present application and the features in the embodiments can be combined with each other arbitrarily. In addition, although the logical order is shown in the flow chart, in some cases, the steps shown or described can be performed in an order different from that here.

[0058] To facilitate understanding of the technical solutions provided in the embodiments of the present application, some key terms used in the embodiments of the present application are explained here:

[0059] POD: It is the most basic resource type and the smallest scheduling unit in container cluster technology. A POD is used to store one or more containers that implement the same business function and share the same network and storage. All containers in the POD are uniformly scheduled and bound to the same node following the POD and are inseparable. The containers in the POD have two characteristics: shared network and shared storage. Shared network means that all containers in the POD share the same network namespace, including IP address and network port. Shared storage means that all containers in the POD can access shared storage volumes and allow containers to share data.

[0060] Container: Package the application code, tool library, operating environment, and the settings and dependencies required for operation into a container to achieve the advantage of portability. Different containers can run independently in different environments without affecting each other.

[0061] The following is a brief introduction to the design concept of the embodiment of the present application:

[0062] In the embodiment of the present application, multiple PODs are created according to the resource requirements of the POD indicated by the received container scheduling request, and the created PODs are grouped according to the resource requirements of the POD and the upper limit of the resources that the node group can provide, and then a corresponding node group is assigned to each POD group from the node group included in the container cluster, and for each POD included in each POD group, the corresponding target node is determined from the node group assigned to the POD group, and each POD is bound to the corresponding target node. Among them, the container cluster scheduling method adopted in the embodiment of the present application, by grouping PODs and nodes respectively, and then using a hierarchical scheduling mechanism of large-grained batch group scheduling from POD group to node group and POD scheduling within node group, on the basis of satisfying node load balancing, improves the scheduling efficiency of the container cluster through hierarchical scheduling, and at the same time reduces the load pressure of the native scheduler of the container cluster, so that the resource utilization rate of the node is higher.

[0063] In addition, in order to ensure node load balancing while further improving the resource utilization, high availability of nodes and scheduling efficiency of containers, the embodiment of the present application divides the nodes with specific resources in the container cluster into special resource node groups according to the resource attributes, resource utilization and topology of the container cluster to which they belong, and divides the common resource nodes with similar resource utilization and from different topologies into one group through clustering and grouping algorithms for other common resource nodes, and adjusts the divided node groups according to the resource utilization of each node group after the node group is running. Among them, the method enables nodes with special resources to be fully utilized, further improves the resource utilization of nodes, divides common resource nodes with similar resource utilization into one group, and dynamically adjusts the node groups according to the resource utilization, ensures that the resource utilization of each node in the node group is always similar, so that when performing POD scheduling in each node group, it is not necessary to spend more computing resources to achieve load balancing, ensure node load balancing, and further improve the scheduling efficiency of the container, and ensure the high availability of the nodes in the group by allowing the common resource node group to contain nodes from different topologies.

[0064] Finally, when the embodiment of the present application groups the PODs, the PODs with the same resource requirements are divided into a group, and the total resource requirement of each POD group is less than and closest to the upper limit of the node group resource amount, and there is a high correlation between each POD in each POD group. When performing large-granularity batch group scheduling from POD group to node group, the special resource node group is assigned to the POD group with specific resource requirements, and the ordinary resource node group is assigned to other POD groups. When there are other POD groups that are not assigned to the ordinary resource node group, the special resource node group is assigned to other unassigned POD groups. And when the container cluster load is too low, the POD with low activity is migrated to the node with low available resources. This method groups the PODs with the same resource requirements into a group, realizes the adaptability allocation corresponding to the special resource node group and the ordinary resource node group, and is convenient for saving computing resources to realize the load balancing of POD scheduling in each node group. The total resource requirement of each POD group is less than and closest to the upper limit of the node group, so that the resource requirements of the POD group match the resource amount of the node group, avoiding the situation where the node group resources are insufficient and cannot be scheduled. There is a high degree of correlation between the PODs in each POD group, so that PODs in the same group may come from the same application or have dependencies on each other. When scheduling PODs in the node group, affinity or anti-affinity strategies can be used more conveniently to improve the scheduling efficiency of containers. Allocating special resource node groups to other unallocated POD groups and migrating nodes based on POD activity can both efficiently utilize node fragmentation resources and improve node resource utilization without affecting POD operation.

[0065] The following briefly introduces the application scenarios to which the technical solutions of the embodiments of the present application can be applied. It should be noted that the application scenarios introduced below are only used to illustrate the embodiments of the present application and are not limited. In the specific implementation process, the technical solutions provided by the embodiments of the present application can be flexibly applied according to actual needs.

[0066] The technical solution provided in the embodiment of the present application can be applied to a container cluster, such as a container cloud platform based on the Kubernetes framework. Figure 1 As shown, it is a schematic diagram of an application scenario provided by an embodiment of the present application. In this scenario, it may include a user terminal 101, a container cluster scheduling device 102, and a cluster node device 103.

[0067] The user terminal 101 can be any device that can connect to a server and provide local services to users, such as a mobile phone, a personal computer (PC), a tablet computer (PAD), a laptop computer, a desktop computer, a mobile Internet device (MID), etc., and this embodiment does not make any specific limitation.

[0068] The container cluster scheduling device 102 may be a management device in the container cluster, used to implement functions such as deployment, management, and monitoring of containers.

[0069] The cluster node device 103 provides the necessary operating environment for the scheduled POD through the agent program of the container cluster. It can be an independent physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, i.e., content delivery networks (CDNs), as well as big data and artificial intelligence platforms, but is not limited to these.

[0070] It should be noted that Figure 1 The above is just an example. In fact, the number of cluster node devices, user terminals, and cluster node devices is not limited, and is not specifically limited in the embodiments of the present application. Figure 1 The components and structures shown are merely exemplary and non-limiting. In actual scenarios, other components and structures may be provided as required.

[0071] like Figure 2 As shown, it is a system architecture diagram of the container cluster scheduling device 102 provided in the embodiment of the present application. In this architecture, a control module 1021, a cache module 1022, a node grouping module 1023, a POD grouping module 1024, a primary scheduling module 1025, and a secondary scheduling module 1026 may be included. It should be noted that Figure 1 The components and structure of the container cluster scheduling device 102 shown are only exemplary and non-restrictive. In actual scenarios, other components and structures may be provided as required.

[0072] (1) The control module 1021 is used to receive the container scheduling request triggered by the user terminal, and obtain the container data information indicated by the request through the list monitoring mechanism to create the POD to be scheduled. The node information obtained from the cluster server and the created POD information to be scheduled can also be transmitted to the node grouping module 1023 and the POD grouping module 1024 respectively for subsequent scheduling process.

[0073] (2) A cache module 1022 is used to cache data of the entire container cluster scheduling device 102. The intermediate scheduling data generated by the above-mentioned other modules during the first-level and second-level scheduling processes will be temporarily stored in the cache module, such as the node group information generated by the node grouping module 1023, the POD group list information generated by the POD grouping module 1024, and the first-level scheduling result information generated by the first-level scheduling module 1025.

[0074] (3) The node grouping module 1023 is used to obtain node data information from the control module 1021 , dynamically group the nodes according to node attributes, and store the grouping results in the cache module 1022 .

[0075] (4) The POD grouping module 1024 is used to obtain data information of the POD to be scheduled from the control module 1021 , group the PODs according to a preset grouping rule, and store the generated POD group list in the cache module 1022 .

[0076] (5) The first-level scheduling module 1025 is used to obtain the node group information and POD group list information from the cache module 1022, assign a corresponding node group to each POD group, and bind the POD group batch scheduling to the node group.

[0077] (6) A secondary scheduling module 1026 is used to determine the corresponding target node from the node group bound to the POD group scheduling for each POD contained in each POD group after the primary scheduling module 1025 completes the large-grained batch scheduling, and bind each POD to the corresponding target node.

[0078] It should be noted that Figure 2 The above is just an example. In fact, the number of control modules, cache modules, node grouping modules, POD grouping modules, primary scheduling modules, and secondary scheduling modules is not limited, and is not specifically limited in the embodiments of the present application. Figure 2 The components and structures shown in the system architecture diagram are merely exemplary and non-restrictive. In actual scenarios, other components and structures may be provided as required.

[0079] In a possible implementation, the container cluster scheduling device uses a first-level scheduling module to be responsible for batch scheduling of POD groups bound to node groups, while the number of second-level scheduling modules is equal to and corresponds one-to-one to the number of node groups, and each second-level scheduling module is responsible for POD scheduling of the corresponding node group.

[0080] The following describes the container cluster scheduling method provided by the exemplary embodiment of the present application in combination with the application scenario described above and with reference to the accompanying drawings. It should be noted that the above application scenario is only shown to facilitate understanding of the spirit and principle of the present application, and the implementation of the present application is not limited in this respect.

[0081] See also Figure 3 As shown, it is a flow chart of the container cluster scheduling method provided in the embodiment of the present application. Here, the container cluster scheduling platform is used as an example for illustration. The specific implementation process of the method is as follows:

[0082] Step 301: Create multiple PODs based on a received container scheduling request, where the scheduling request is used to indicate resource requirements of the multiple PODs.

[0083] In an embodiment of the present application, when relevant business operations are performed on the user side, a scheduling request for multiple containers required to implement the business function will be triggered. The control module processes the received container scheduling request, analyzes the resources required for the multiple containers indicated by the request, and creates multiple PODs based on the container resource requirements, where one POD contains one or more containers that implement the same business function.

[0084] In a possible implementation, taking the Kubernetes container cluster as an example, the control module can follow the design of the native Kubernetes Informer notifier, where the Informer notifier is a core toolkit of Kubernetes. The Kubernetes component obtains the latest data information of the resource object through the list-watch function and get function provided by the Informer notifier. The control module obtains data information related to the scheduling request from the cluster server, such as the application code, tool library, operating environment, and settings and dependencies required for the scheduled container, and performs the corresponding POD creation process. The created POD can be stored in the cache module or directly used as the input of the POD grouping module.

[0085] Step 302: Based on the resource requirements of each POD and the upper limit of the amount of resources that each node group can provide, the multiple PODs are grouped to obtain multiple POD groups, each POD group including at least one POD.

[0086] In the embodiment of the present application, the POD grouping module groups the PODs according to the resource requirements of each POD and the upper limit of the resource amount that each node group can provide, so that the multiple POD groups obtained by grouping meet the following conditions:

[0087] (1) Each POD group has the same resource requirements.

[0088] Specifically, the resource requirements of POD may include but are not limited to the storage resources, CPU resources, disk resources, etc. required to run the POD container. The POD grouping module divides the multiple PODs created into multiple groups according to the same resource requirements so that each POD group has the same resource requirements. Through this method, PODs with specific resource requirements such as special GPUs and high-performance disks can be divided into a group, which is convenient for subsequent batch scheduling to node groups with special resources and fully utilizing nodes with special resources.

[0089] (2) The total resource demand of each POD group is less than or equal to the upper limit of the resource demand of the node group, and the difference between the total resource demand and the upper limit of the resource demand of the node group does not exceed a third preset difference threshold.

[0090] Specifically, the POD grouping module can obtain node group information from the control module, and calculate the total resource demand of the currently divided POD group in real time during grouping to compare the resource upper limit of the node group with the total resource demand of each POD, to ensure that the total resource demand of each POD group is less than or equal to the resource upper limit of the node group, and the difference between the total resource demand and the resource upper limit of the node group does not exceed the third preset difference threshold. In order to further improve the resource utilization of the node, it is usually ensured that the total resource demand is closest to the resource upper limit of the node group.

[0091] (3) The correlation between each POD in each POD group is not less than a preset correlation threshold.

[0092] Specifically, the correlation between two PODs represents the possibility that the two PODs come from the same application or have dependencies on each other. The POD grouping module ensures that the PODs in a group of PODs have a high correlation with each other by ensuring that the correlation between the PODs in each POD group is not less than a preset correlation threshold during grouping. This ensures that the PODs in the same group come from the same application or have dependencies on each other as much as possible. In this way, affinity or anti-affinity strategies can be used more conveniently when scheduling PODs in a node group, that is, PODs with high correlation are called to the same node, and the containers of the PODs can cooperate with each other to improve the efficiency of container calls. At the same time, it is avoided that PODs with low correlation are called to the same node, which causes the related operating environments or tool components to exclude each other when the containers in the PODs are running, resulting in conflicts and container crashes.

[0093] In one possible implementation, the node grouping module may group nodes by adding labels, for example, adding a serial number label of the corresponding node group to all nodes in the same node group, and both the primary and secondary scheduling modules may use the label to identify the node group to which the node belongs.

[0094] Step 303: Allocate a corresponding node group to each POD group from the node groups included in the container cluster.

[0095] In the embodiment of the present application, before the primary scheduling module assigns a corresponding node group to each POD, the node grouping module groups the nodes according to the node attributes of each node so that the multiple node groups obtained by grouping meet the following conditions:

[0096] (1) The resource attributes of multiple nodes in each node group are the same.

[0097] Specifically, the resource attributes of a node may include but are not limited to the storage resources, CPU resources, disk resources, etc. of the cluster server corresponding to the node. The control module obtains the resource attribute information of the server corresponding to each node from the cluster node device and sends it to the grouping module. The node grouping module can group nodes with the same resource attributes into one group based on the received resource attribute information. Through this method, specific resource nodes with special GPUs, high-performance disks, etc. can be divided into multiple special resource node groups, which is convenient for batch provision of resources for multiple PODs with special resource requirements.

[0098] (2) The difference in resource utilization between any two nodes in each node group is not greater than a first preset difference threshold.

[0099] Specifically, the control module can use the list-watch mechanism to implement real-time monitoring of the resource utilization of each node, and can send the resource utilization information of each node at a certain moment to the node grouping module. When grouping, the node grouping module makes the difference between the resource utilization of any two nodes not greater than the first preset difference threshold according to the resource utilization of the nodes, that is, the nodes with similar resource utilization are grouped together for subsequent POD scheduling within the node group. The container cluster scheduling device no longer needs to spend more computing resources to load balance the nodes, thereby improving the efficiency of container scheduling.

[0100] (3) Each node group includes nodes in at least two topologies in the container cluster.

[0101] Specifically, when each server forms a server cluster according to the topological structure, the control module can obtain the topological structure information of the server corresponding to each node and send it to the node grouping module. The POD grouping module uses the cluster topological information of the server corresponding to the node to group the nodes, so that each node group contains nodes belonging to different topological structures of the server cluster, so that when a node of a certain topology of the node group goes down, the nodes of other topologies can still continue to provide services, ensuring the high availability of the node group.

[0102] In a possible implementation, the node grouping module periodically obtains resource utilization sets of each node group from the control module, and the resource utilization set of each node group includes the resource utilization of each node in the corresponding node group. When the node grouping module determines that the difference between the maximum value and the minimum value contained in the resource utilization set of the node group is greater than the second preset difference threshold, the second preset difference threshold can be 10% of the maximum value, the node grouping module will adjust the divided node groups, and use the adjusted node groups as the latest node groups for use by other modules.

[0103] Specifically, the grouping adjustment can be to divide all nodes in the node group whose difference between the maximum and minimum values ​​in the resource utilization collection exceeds a preset threshold into other node groups according to the original grouping rules, or to regroup all node groups in the current container cluster according to the original grouping rules.

[0104] In an embodiment of the present application, the node-level scheduling module assigns a corresponding node group to each POD group based on the node group information and POD group information obtained from the cache module, and binds the POD group's large-grained batch scheduling to the node group.

[0105] Specifically, the first-level scheduling module will first allocate the node group with special resources to the POD group with specific resource requirements among multiple POD groups, and then allocate other common resource node groups to other POD groups except those with specific resource requirements. If other POD groups are not allocated to the corresponding common resource node groups, the first-level scheduling module will allocate the special resource node group to other unallocated POD groups to ensure that all pods can run normally, and efficiently utilize the node's fragmented resources to improve the node's resource utilization.

[0106] Step 304: for each POD included in each POD group, determine a corresponding target node from the node group allocated to the POD group, and bind each POD to the corresponding target node.

[0107] In the embodiment of the present application, after the primary scheduling module completes the process of allocating node groups to POD groups, the secondary scheduling module is used to implement the process of binding each POD in each POD group to the corresponding target node.

[0108] In one possible implementation, the secondary scheduling module can use the native scheduler of the container cluster. For example, in a Kubernetes container cluster, the secondary scheduling module can use the native scheduler of Kubernetes to schedule and bind PODs within the node group based on considerations such as node load balancing, resource utilization, and high availability.

[0109] refer to Figure 4 As shown in the figure, it is a schematic diagram of POD scheduling by the two-level scheduling module, wherein the first-level scheduling module allocates the POD groups with common resource requirements and the POD groups with special resource requirements to the corresponding common resource node groups and special resource node groups in batches according to the resource requirement attributes of the POD groups obtained from the cache module, and the second-level scheduling module binds the PODs in each node group to the specified node by considering factors such as load balancing and disaster recovery strategies.

[0110] In one possible implementation, when the workload of the container cluster server is lower than the preset load threshold, the control module will obtain the current activity of each POD through the list-watch mechanism, and determine whether it is lower than the preset activity threshold, where the activity represents the probability of the resource demand of the POD, or the amount of resources when the resource demand is expanded. When the control module determines that the activity of the target POD is lower than the threshold, the secondary scheduling module will migrate the target POD to a node with available resources lower than the preset resource threshold. Specifically, the control module can periodically obtain its workload from the cluster server, or it can obtain it through event triggering, such as when receiving feedback from the user end that the server is responding too slowly.

[0111] When the number of user access requests for business functions corresponding to the POD container increases, the possibility of expanding the POD also increases, and the probability of representing the POD resource demand or the activity of the resource quantity when the resource demand is expanded also increases. The activity of the POD with no expansion demand or low expansion demand resources is also low. At this time, the node with low available resources can meet the resource needs of the low-activity node. Therefore, in order to ensure the normal operation of the POD and efficiently utilize the fragmented resources of the node, the low-activity POD can be migrated to the node with low available resources through the secondary scheduling module.

[0112] In a possible implementation, in order to ensure node load balancing and further improve node resource utilization, high availability, and container scheduling efficiency, the node grouping module can also group nodes based on their resource attributes, resource utilization, and the topology of the container cluster they belong to. Figure 5 FIG. 1 is a schematic diagram of a process of node grouping provided in an embodiment of the present application. The specific implementation process of the method is as follows:

[0113] Step 501: Based on the nodes with specific resources in the container cluster, at least one special resource node group is obtained.

[0114] In the embodiment of the present application, the node grouping module directly divides the nodes with specific resources in each cluster topology into special resource node groups, so as to provide resources in batches for multiple pods with specific resource requirements.

[0115] Step 502: Based on the resource utilization of each node, cluster the common resource nodes in the container cluster except the nodes with specific resources, and determine the number of groups N according to the clustering result.

[0116] In the embodiment of the present application, the node grouping module performs clustering processing on the common resource nodes in the container cluster except the nodes with specific resources, and uses the obtained clustering result as the group number of the common resource node group.

[0117] In a possible implementation, the node grouping module may adopt the elbow method in the clustering algorithm, obtain the number of common resource nodes in the container cluster and the resource utilization rate corresponding to each common resource node from the control unit, and determine the number of node groups by the following formula:

[0118]

[0119] Among them, q is the number of classes of common resource nodes, p is the total number of common resource nodes in the container cluster, and x ij is the resource utilization of the jth node in the i-th type of nodes, y i is the average resource utilization of all nodes in the i-th class, and dist is the sum of the differences between the resource utilization of each node in the container cluster and the average resource utilization of its class. The corresponding elbow curve is obtained according to the above formula. The number of classes q corresponding to the inflection point of the elbow curve is the final number of node classes obtained by the clustering algorithm. Assuming that the number of container cluster topologies is t, the number of node groups m finally determined is:

[0120] m=max{2,t,q}

[0121] The number of node groups m is determined by comparing the final number of node classes q, the number of container cluster topologies t, and the value 2, and the maximum value is determined as the final number of common resource node groups.

[0122] Step 503: sorting the common resource nodes in each topological structure in the container cluster in ascending order of resource utilization, and obtaining a common resource node queue corresponding to each topological structure.

[0123] In the embodiment of the present application, in order to ensure that the difference between the resource utilization rates of the common resource node groups is not greater than the first preset difference threshold, and that the nodes in at least two topological structures in the container cluster are included, the node grouping module arranges the common resource node groups in each topological structure in the order of resource utilization rate from small to large as follows: Figure 6 The common resource node queue shown is used to facilitate the subsequent grouping process.

[0124] Step 504: Divide each common resource node queue into N subqueues respectively, the order of common resource nodes in each subqueue is consistent with the corresponding common resource node queue, and the minimum resource utilization in one of any two subqueues is greater than the maximum resource utilization in the other subqueue.

[0125] In the present application examples, reference Figure 5As shown, the node grouping module divides the common resource node queue of each topology into N sub-queues in the same order as the common resource node queue according to the determined node grouping number N, thereby ensuring that the difference in resource utilization between each sub-queue is not greater than the first preset difference threshold.

[0126] In a possible implementation, the node grouping module can also extract a certain number of nodes from each common resource node queue in the order of the common resource node queue to form a node group, wherein the number of extracted nodes is determined by the ratio of the total number of nodes in each common resource node queue to the determined number of node groups. This can ensure that each node group contains nodes from different topologies as much as possible, and ensure that the node resource utilization in each node group is similar, so that when performing POD scheduling within the node group, the high availability of the node can be guaranteed to meet the load balancing requirements.

[0127] Step 505: In the N sub-queues corresponding to each common resource node queue, the nodes included in the sub-queues corresponding to the positions are combined to obtain N common resource node groups.

[0128] In the present application examples, reference Figure 5 As shown, the node grouping module combines the nodes included in the subqueues corresponding to the positions in the N subqueues corresponding to the common resource node queues of each topology to obtain N common resource node groups, so that each node group contains nodes from different topologies, and ensures that the resource utilization of the nodes in each node group is similar, thereby ensuring high availability of the nodes and meeting the load balancing requirements of POD scheduling within the node group.

[0129] In a possible implementation, the node grouping module may further extract a certain number of nodes from each common resource node queue in the order of the common resource node queue to form a node group, and finally obtain N common resource node groups. The number of extracted nodes is determined by the ratio of the total number of nodes in each common resource node queue to the determined number of node groups.

[0130] Specifically, refer to Figure 6 As shown, it is a schematic diagram of grouping nodes in a cluster topology, wherein the example cluster includes three topological structures, wherein topology 1 includes four common resource nodes and one node with specific resources, topology 2 includes three common resource nodes and one node with specific resources, and topology 3 includes two common resource nodes and one node with specific resources. The black area in the rectangles of the two types of nodes in the figure represents the resource utilization of the nodes.

[0131] Taking the determined number of node groups equal to 2 as an example, the node grouping module will directly divide the nodes with specific resources in the three topologies into a special resource node group. For the other common resource nodes in each topology, the node grouping module will arrange them in order of resource utilization from small to large, and obtain the three corresponding common resource node queues in the three topological structures. Then, according to the order of the common resource node queues, the three common resource node queues are divided into two sub-queues respectively, and the nodes included in the sub-queues corresponding to the positions in the three topologies are combined according to the order of the common resource node queues to obtain two common resource node groups. In one possible implementation, refer to Figure 7 The figure shows a grouping flow chart of a node grouping module. The specific implementation steps of the method are as follows:

[0132] Step 701: Determine the number of groups into which all nodes in the container cluster need to be divided.

[0133] Step 702: Based on the resource utilization of the nodes, the nodes in each cluster topology are sorted from small to large according to the resource utilization of the nodes, and a node queue corresponding to each cluster topology is obtained.

[0134] Step 703: extract a certain number of nodes from the node queue corresponding to each cluster topology in proportion to form a node group, and obtain multiple node groups, wherein the number of extracted nodes is determined by the ratio of the total number of nodes to the number of groups.

[0135] Step 704: Add a node group serial number label to each node group, and add corresponding node group serial number labels to all nodes in the same node group.

[0136] Step 705: Determine whether a new node is generated in the container cluster, if yes, execute steps 706-707. If no, execute step 708.

[0137] Step 706: Obtain the resource utilization of the new node, and compare it with the resource utilization of the original nodes in the cluster to determine the original node with the smallest difference in resource utilization with the new node.

[0138] Step 707: query the node group serial number label of the node, and classify the new node into the node group corresponding to the serial number label.

[0139] Step 708: Determine whether there is a node group in which the difference between the maximum value and the minimum value of the node resource utilization in the node group is greater than the second preset difference threshold; if so, execute step 701.

[0140] In one possible implementation, reference Figure 8 The figure shows a flowchart of a container cluster scheduling process. The specific implementation steps of the method are as follows:

[0141] Step 801: Group all nodes in the container cluster to obtain multiple node groups.

[0142] Step 802: Determine whether a container scheduling request is received. If so, execute step 803; if not, end the scheduling.

[0143] Step 803: Create multiple PODs according to the container scheduling request.

[0144] Step 804: According to the resource requirements of each POD, multiple PODs are grouped to obtain multiple POD groups.

[0145] Step 805: Determine whether there is a POD group with special resource requirements. If so, execute step 806; if not, execute step 807.

[0146] Step 806: Allocate the node group with special resources to the POD group with special resource requirements.

[0147] Step 807: Allocate the common resource node group to the common resource demand POD group.

[0148] Step 808: Determine whether there is a common resource demand POD group that is not allocated to a common resource node group. If so, execute step 809; if not, execute step 810.

[0149] Step 809: Allocate the node group with special resources to the unallocated POD group with common resource requirements.

[0150] Step 810: Determine a corresponding target node from the node groups allocated to each POD group, and bind each POD to the corresponding target node.

[0151] Step 811: Determine whether the workload of the container cluster server is lower than a preset load threshold. If so, execute step 812; if not, execute step 815.

[0152] Step 812: Determine whether there is a POD group whose activity is lower than a preset activity threshold. If so, execute step 813; if not, execute step 815.

[0153] Step 813: Group multiple PODs according to the activity level of each POD to obtain multiple POD groups.

[0154] Step 814: Allocate the POD group whose activity is lower than the preset activity threshold to the node group whose available resource amount is lower than the preset resource amount threshold.

[0155] Step 815: Determine whether there is a node group in which the difference between the maximum value and the minimum value of the node resource utilization in the node group is greater than the second preset difference threshold. If so, execute step 801; if not, execute step 802.

[0156] In summary, the embodiment of the present application proposes a hierarchical pod scheduling method, adds a batch scheduling algorithm, and realizes pod batch scheduling, with the purpose of improving the scheduling efficiency of containers in the cluster on the basis of satisfying resource balance, dispersing the pressure of the central scheduler, and making resource utilization higher. Specifically, through the design of a two-level scheduler mechanism, hierarchical scheduling is used to improve scheduling efficiency and reduce scheduler pressure. The Node grouping algorithm comprehensively considers the node resource specificity and resource utilization rate, and adds node topology characteristics, and dynamically groups during operation. In addition, the Pod grouping algorithm is divided into two stages: grouping at creation time and grouping at runtime. The strategy of grouping according to pod activity is used, and cluster fragment resources are sorted during Pod operation, which further improves resource utilization.

[0157] See also Fig. 9 Based on the same inventive concept, the embodiment of the present application further provides a container cluster scheduling device 90, which includes:

[0158] A creating unit 901 is used to create multiple PODs based on the received container scheduling request, where the scheduling request is used to indicate resource requirements of the multiple PODs;

[0159] A grouping unit 902 is used to group multiple PODs based on the resource requirements of each POD and the upper limit of the resource amount that each node group can provide, to obtain multiple POD groups, each POD group including at least one POD;

[0160] The allocation unit 903 is used to allocate a corresponding node group to each POD group from the node groups included in the container cluster;

[0161] The binding unit 904 is used to determine, for each POD included in each POD group, a corresponding target node from the node group allocated to the POD group, and bind each POD to the corresponding target node.

[0162] Optionally, the grouping unit 902 is specifically configured to:

[0163] Based on the node attributes of each node included in the container cluster, each node is grouped to obtain multiple node groups;

[0164] The node attributes between multiple nodes in each node group have one or more of the following relationships:

[0165] The resource attributes are the same;

[0166] The difference between the resource utilization rates is not greater than a first preset difference threshold;

[0167] Contains nodes in at least two topologies in the container cluster.

[0168] Optionally, the grouping unit 902 is specifically configured to:

[0169] Based on the nodes with specific resources in the container cluster, obtaining at least one special resource node group;

[0170] Based on the resource utilization rate of each node, clustering is performed on common resource nodes in the container cluster except nodes with specific resources, and the number of groups N is determined according to the clustering result;

[0171] The common resource nodes in each topological structure in the container cluster are sorted in ascending order of resource utilization to obtain a common resource node queue corresponding to each topological structure;

[0172] Each common resource node queue is divided into N subqueues respectively, the order of common resource nodes in each subqueue is consistent with the corresponding common resource node queue, and the minimum resource utilization rate in one of any two subqueues is greater than the maximum resource utilization rate in the other subqueue;

[0173] In the N sub-queues corresponding to each common resource node queue, the nodes included in the sub-queues corresponding to the positions are combined to obtain N common resource node groups.

[0174] Optionally, the grouping unit 902 is specifically configured to:

[0175] For each node group in the multiple node groups, a resource utilization set of each node group is obtained, where the resource utilization set includes the resource utilization of each node in the corresponding node group;

[0176] For each resource utilization set, if the difference between the maximum value and the minimum value included therein is greater than a second preset difference threshold, the multiple node groups are grouped and adjusted to obtain multiple adjusted node groups.

[0177] Optionally, the grouping unit 902 is specifically configured to:

[0178] Multiple PODs are grouped to obtain multiple POD groups, where the multiple POD groups meet one or more of the following conditions:

[0179] Each POD group has the same resource requirements;

[0180] The total resource demand of each POD group is less than or equal to the upper limit of the resource quantity, and the difference between the total resource demand and the upper limit of the resource quantity does not exceed a third preset difference threshold;

[0181] The correlation between the PODs in each POD group is not less than a preset correlation threshold.

[0182] Optionally, the allocating unit 903 is specifically configured to:

[0183] Allocate the special resource node group contained in the container cluster to the POD group with specific resource requirements among multiple POD groups;

[0184] Allocate the common resource node groups included in the container cluster to other POD groups among multiple POD groups except the POD groups with specific resource requirements;

[0185] If there are other POD groups that are not assigned to the corresponding common resource node groups, the special resource node group is assigned to the other unassigned POD groups.

[0186] Optionally, the device further includes a migration unit 905, which is specifically used for:

[0187] When the load of the container cluster is lower than the preset load threshold, determine whether the activity of each POD is lower than the preset activity threshold; the activity represents the probability of the resource demand of the POD, or the amount of resources when the resource demand is expanded;

[0188] When the activity of a target POD is lower than a preset activity threshold, the target POD is migrated to a node whose available resources are lower than the preset resource threshold.

[0189] Through the above-mentioned device, multiple PODs are created according to the resource requirements of the POD indicated by the received container scheduling request, and the created PODs are grouped according to the resource requirements of the POD and the upper limit of the resources that the node group can provide, and then the corresponding node group is allocated to each POD group from the node group included in the container cluster, and for each POD included in each POD group, the corresponding target node is determined from the node group allocated to the POD group, and each POD is bound to the corresponding target node. Among them, the container cluster scheduling method adopted in the embodiment of the present application, by grouping PODs and nodes respectively, and then using the hierarchical scheduling mechanism of large-granularity batch group scheduling from POD group to node group and POD scheduling within node group, on the basis of satisfying node load balancing, improves the scheduling efficiency of the container cluster through hierarchical scheduling, and at the same time reduces the load pressure of the native scheduler of the container cluster, so that the resource utilization rate of the node is higher.

[0190] For the convenience of description, the above parts are divided into each unit module (or module) according to the function and described separately. Of course, when implementing this application, the functions of each unit (or module) can be implemented in the same or multiple software or hardware. The device can be used to execute the method shown in each embodiment of the present application. Therefore, for the functions that can be implemented by each functional module of the device, reference can be made to the description of the aforementioned embodiment, and no further details are given.

[0191] See also Fig.10 Based on the same technical concept, the present application also provides a computer device. In one embodiment, the computer device can be Figure 1 The container cluster scheduling device shown in FIG. Fig.10 As shown, it includes a memory 1001 , a communication module 1003 and one or more processors 1002 .

[0192] The memory 1001 is used to store computer programs executed by the processor 1002. The memory 1001 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system and programs required for running the instant messaging function, etc.; the data storage area may store various instant messaging information and operation instruction sets, etc.

[0193] The memory 1001 may be a volatile memory, such as a random-access memory (RAM); the memory 1001 may also be a non-volatile memory, such as a read-only memory, a flash memory, a hard disk drive (HDD) or a solid-state drive (SSD); or the memory 1001 may be any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 1001 may be a combination of the above memories.

[0194] The processor 1002 may include one or more central processing units (CPU) or a digital processing unit, etc. The processor 1002 is configured to implement the above container cluster scheduling method when calling the computer program stored in the memory 1001 .

[0195] The communication module 1003 is used to communicate with the rearward sensor and other servers.

[0196] The specific connection medium between the memory 1001, the communication module 1003 and the processor 1002 is not limited in the embodiment of the present application. Fig.10 In the embodiment, the memory 1001 and the processor 1002 are connected via a bus 1004. The bus 1004 is connected to the processor 1002 via a bus 1004. Fig.10 The connections between the other components are only for illustration and are not intended to be limiting. The bus 1004 can be divided into an address bus, a data bus, a control bus, etc. For ease of description, Fig.10 The diagram shows that only one thick line is used, but this does not mean that there is only one bus or one type of bus.

[0197] The memory 1001 stores a computer storage medium, and the computer storage medium stores computer executable instructions, and the computer executable instructions are used to implement the container cluster scheduling method of the embodiment of the present application. The processor 1002 is used to execute the container cluster scheduling method of each of the above embodiments.

[0198] In some possible implementations, various aspects of the container cluster scheduling method provided by the present application may also be implemented in the form of a program product, which includes a program code. When the program product is run on a computer device, the program code is used to enable the computer device to execute the steps of the container cluster scheduling method according to various exemplary embodiments of the present application described above in this specification. For example, the computer device may execute the steps of each embodiment.

[0199] The program product may use any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0200] The program product of the embodiment of the present application may adopt a portable compact disk read-only memory (CD-ROM) and include program code, and can be run on a computing device. However, the program product of the present application is not limited thereto. In the present application, the readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with a command execution system, device or device.

[0201] The readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, wherein the readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The readable signal medium may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with a command execution system, apparatus, or device.

[0202] The program code embodied on the readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the foregoing.

[0203] Program code for performing the operations of the present application may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" language or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user equipment, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device via any type of network including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0204] It should be noted that, although several units or subunits of the device are mentioned in the above detailed description, this division is merely exemplary and not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more units described above can be embodied in one unit. Conversely, the features and functions of one unit described above can be further divided into multiple units to be embodied.

[0205] In addition, although the operations of the method of the present application are described in a specific order in the drawings, this does not require or imply that the operations must be performed in this specific order, or that all the operations shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps.

[0206] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0207] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

[0208] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A container cluster scheduling method, characterized in that: The method comprises: Based on the nodes with specific resources in the container cluster, at least one special resource node group is obtained; Based on the resource utilization rate of each node, clustering is performed on common resource nodes in the container cluster except nodes with specific resources, and the number of groups N is determined according to the clustering result; The common resource nodes in each topological structure in the container cluster are sorted in ascending order of resource utilization to obtain a common resource node queue corresponding to each topological structure; Each common resource node queue is divided into N subqueues respectively, the order of common resource nodes in each subqueue is consistent with the corresponding common resource node queue, and the minimum resource utilization rate in one of any two subqueues is greater than the maximum resource utilization rate in the other subqueue; In the N sub-queues corresponding to each common resource node queue, the nodes included in the sub-queues corresponding to the positions are combined to obtain N common resource node groups; Creating a plurality of PODs based on the received container scheduling request, wherein the scheduling request is used to indicate resource requirements of the plurality of PODs; Based on the resource requirements of each POD and the upper limit of the amount of resources that can be provided by each node group, the multiple PODs are grouped to obtain multiple POD groups, each POD group including at least one POD; Allocate a corresponding node group to each POD group from the node groups included in the container cluster; For each POD included in each POD group, a corresponding target node is determined from the node group allocated to the POD group, and each POD is bound to the corresponding target node.

2. The method according to claim 1, characterized in that The node attributes between multiple nodes in each node group have one or more of the following relationships: The resource attributes are the same; The difference between the resource utilization rates is not greater than a first preset difference threshold; The container cluster includes nodes in at least two topological structures.

3. The method according to claim 1, characterized in that After obtaining N common resource node groups, the method further includes: For each node group in the multiple node groups, respectively, a resource utilization set of each node group is obtained, wherein the resource utilization set includes the resource utilization of each node in the corresponding node group; For each resource utilization set, if the difference between the maximum value and the minimum value included therein is greater than a second preset difference threshold, the multiple node groups are grouped and adjusted to obtain multiple adjusted node groups.

4. The method according to claim 1, characterized in that After binding each POD to the corresponding target node, the method further includes: When the load of the container cluster is lower than a preset load threshold, determining whether the activity of each POD is lower than a preset activity threshold; the activity represents the probability of resource demand of the POD, or the amount of resources when the resource demand is expanded; When the activity of a target POD is lower than the preset activity threshold, the target POD is migrated to a node whose available resource amount is lower than the preset resource amount threshold.

5. The method according to claim 1, characterized in that Based on the resource requirements of each POD and the upper limit of the amount of resources that each node group can provide, the multiple PODs are grouped to obtain multiple POD groups, including: The multiple PODs are grouped to obtain multiple POD groups, where the multiple POD groups satisfy one or more of the following conditions: Each POD group has the same resource requirements; The total resource demand of each POD group is less than or equal to the upper limit of the resource amount, and the difference between the total resource demand and the upper limit of the resource amount does not exceed a third preset difference threshold; The correlation between the PODs in each POD group is not less than a preset correlation threshold.

6. The method according to claim 5, characterized in that The step of allocating a corresponding node group to each POD group from the node groups included in the container cluster includes: Allocating the special resource node group included in the container cluster to a POD group having specific resource requirements among the multiple POD groups; Allocating the common resource node group included in the container cluster to other POD groups among the multiple POD groups except the POD group with specific resource requirements; If there are other POD groups that are not allocated to corresponding common resource node groups, the special resource node group is allocated to the other unallocated POD groups.

7. A container cluster scheduling device, characterized in that: include: A grouping unit, used to obtain at least one special resource node group based on nodes having specific resources in the container cluster; Based on the resource utilization rate of each node, cluster the common resource nodes in the container cluster except the nodes with specific resources, and determine the number of groups N according to the clustering result; sort the common resource nodes in each topological structure in the container cluster in order from small to large in resource utilization, and obtain the common resource node queue corresponding to each topological structure; divide each common resource node queue into N subqueues, the order between the common resource nodes in each subqueue is consistent with the corresponding common resource node queue, and the minimum resource utilization rate in one of any two subqueues is greater than the maximum resource utilization rate in the other subqueue; combine the nodes included in the subqueues corresponding to the positions in the N subqueues corresponding to each common resource node queue, and obtain N common resource node groups; A creating unit, configured to create a plurality of PODs based on a received container scheduling request, wherein the scheduling request is used to indicate resource requirements of the plurality of PODs; The grouping unit is used to group the multiple PODs based on the resource requirements of each POD and the upper limit of the resource amount that each node group can provide, so as to obtain multiple POD groups, each POD group including at least one POD; The allocation unit is used to allocate a corresponding node group to each POD group from the node groups included in the container cluster; The binding unit is used to determine, for each POD included in each POD group, a corresponding target node from the node group allocated to the POD group, and bind each POD to the corresponding target node.

8. A computer device, characterized in that: include: Memory for storing computer programs; A processor, configured to implement the steps of the method according to any one of claims 1 to 6 when executing the computer program.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images