Resource scheduling method, system and device of database and storage medium

Abstract

The invention relates to the technical field of computers, in particular to a resource scheduling method, system and device of a database and a storage medium, and the method comprises the following steps: when a newly created initial container group exists in a database cluster, determining node residual resources of each node; obtaining predicted demand resources of the initial container group based on resource occupation information of at least one historical container group associated with the initial container group; selecting candidate nodes from the nodes of which the node residual resources meet preset demand resources, and for each candidate node, predicting a resource inclination condition of the latest node residual resources on the candidate node after the initial container group is scheduled to the candidate node; and taking one candidate node of which the resource inclination condition meets a preset resource distribution condition as a target node, and scheduling the initial container group to the target node. The inclination condition of the residual resources on the nodes after the container group is placed can be predicted, so that serious inclination of various resources on the nodes is avoided, and the resource utilization rate of the nodes is improved.

Description

Background Art

[0002] With the development of Internet technology, database clusters are gradually replacing traditional single databases with their high reliability, scalability, and high fault tolerance. Compared with a single database, there are multiple nodes in a database cluster. In order to ensure the stability of the database cluster, it is necessary to reasonably schedule the resources of each node (such as CPU, memory, and storage space, etc.), that is, the container group that records the data to be processed needs to be allocated to the appropriate node.

[0003] The resource scheduling strategy in the related technology is often: according to the number of resources applied for by each container group and the number of remaining resources of each node, each container group is allocated to any node capable of accommodating the resources. However, the container group's demand for various resources is often uneven, and it is easy for a container group to have a large demand for one or more resources but a small demand for other resources. If a node happens to be allocated multiple container groups with similar resource requirements, it may eventually result in a large amount of a certain resource on the node being left, but because the usage of another resource has reached the limit, it can no longer be allocated to other container groups, resulting in low resource utilization of the node.

[0004] For example, if a certain type of container group has a large demand for CPU resources but a small demand for memory resources, after multiple such container groups are allocated to a node, the CPU resources are almost fully occupied, but there are still a lot of remaining memory resources. At this time, when other container groups need to be allocated to the node, although the node meets the allocation conditions for memory resources, it cannot be allocated due to insufficient CPU resources, which ultimately causes a large amount of memory resources on the node to be wasted.

[0005] In summary, how to improve resource utilization is an urgent issue to be solved. Summary of the invention

[0006] Embodiments of the present application provide a database resource scheduling method, system, device and storage medium to improve resource utilization of a database cluster.

[0007] A database resource scheduling method provided in an embodiment of the present application includes:

[0008] When there is a newly created initial container group in the database cluster, based on the node resource information of each node included in the database cluster, respectively determining the node remaining resources of the corresponding nodes;

[0009] Based on resource occupancy information of at least one historical container group associated with the initial container group, the predicted required resources of the initial container group are obtained; the initial container group and the historical container group are created by applying for the same resource application object;

[0010] Among the nodes whose node remaining resources meet the preset required resources, at least one candidate node is selected, and the following operations are performed for each candidate node: predicting the resource inclination of the latest node remaining resources on the candidate node after scheduling the initial container group to the candidate node, wherein the resource inclination represents: the distribution difference of the node remaining resources under each resource dimension;

[0011] A candidate node whose resource inclination condition meets a preset resource distribution condition is selected as a target node, and the initial container group is scheduled to the target node.

[0012] An embodiment of the present application provides a database resource scheduling device, including:

[0013] A determination unit, configured to, when a newly created initial container group exists in the database cluster, determine the node remaining resources of the corresponding nodes based on the node resource information of each node included in the database cluster;

[0014] A first selection unit is configured to obtain predicted required resources of the initial container group based on resource occupancy information of at least one historical container group associated with the initial container group; the initial container group and the historical container group are created by applying for the same resource application object;

[0015] A prediction unit is used to select at least one candidate node from each node whose node remaining resources meet the preset required resources, and perform the following operations for each candidate node: predict the resource inclination of the latest node remaining resources on the candidate node after the initial container group is scheduled to the candidate node, wherein the resource inclination represents: the distribution difference of the node remaining resources under each resource dimension;

[0016] The second selection unit is used to select a candidate node whose resource inclination condition meets the preset resource distribution condition as the target node, and schedule the initial container group to the target node.

[0017] In a possible implementation manner, the prediction unit is specifically configured to:

[0018] For each of the nodes, the following operations are respectively performed: based on the difference between the node remaining resources of the node and the predicted required resources of the initial container group, combined with the partition size of the node under each resource dimension, determining the evaluation value of the node remaining resources of the node;

[0019] At least one node whose evaluation value meets the preset selection condition is selected as a candidate node.

[0020] In a possible implementation manner, the prediction unit is specifically configured to:

[0021] Based on the difference between the node remaining resources of the candidate node and the predicted required resources of the initial container group, predict the latest node remaining resources on the candidate node after the initial container group is scheduled to the candidate node;

[0022] For each preset resource dimension, the following operations are performed: based on the latest node remaining resources, determining the remaining resource amount of the candidate node in each resource dimension after the initial container group is scheduled to the candidate node;

[0023] The resource tilt situation is determined based on the difference between the remaining resource amounts.

[0024] In a possible implementation manner, the second selection unit is specifically configured to:

[0025] Among the candidate nodes, the candidate nodes whose difference between the remaining resource amounts meets the preset resource difference threshold are selected as the target nodes;

[0026] or,

[0027] Based on the difference between the remaining resource amounts corresponding to the candidate nodes, the bottleneck resources corresponding to the candidate nodes are determined respectively, each of the bottleneck resources being: the resource with the lowest remaining resource amount of the corresponding candidate node under the resource dimensions; and the remaining resource amounts under the bottleneck resources corresponding to the candidate nodes are compared across resource dimensions, and based on the comparison results, a target node is determined among the candidate nodes.

[0028] In a possible implementation manner, the prediction unit is specifically configured to:

[0029] Under each resource dimension, respectively determining the resource similarity between the predicted demand resource and the node remaining resource of the one candidate node;

[0030] Based on the resource similarity, predicting the resource inclination of the latest node remaining resources on a candidate node after scheduling the initial container group to the candidate node, the resource similarity being negatively correlated with the resource inclination;

[0031] The second selection unit is specifically used for:

[0032] A candidate node whose resource similarity satisfies a preset similarity condition is selected as the target node.

[0033] In a possible implementation, the device further includes:

[0034] A migration unit, configured to select nodes among the nodes whose corresponding node resource information exceeds a preset threshold as nodes to be migrated out;

[0035] For each node to be migrated, container group resource information corresponding to each execution container group loaded on the node to be migrated is obtained; and the execution container group whose corresponding container group resource information meets the migration condition is used as the execution container group to be migrated; the container group resource information represents the resource consumption of the corresponding execution container group;

[0036] A node to be migrated into is determined from each node other than the node to be migrated out, and the execution container group to be migrated is migrated to the node to be migrated in.

[0037] In a possible implementation manner, the migration unit is specifically used for:

[0038] Based on the historical changes of the node resource information of each other node in a preset historical time period, respectively predict the estimated changes of the node resource information of each other node in a preset future time period;

[0039] Based on historical changes in the container group resource information of the execution container group to be migrated within a preset historical time period, predicting estimated changes in the container group resource information of the execution container group to be migrated within a preset future time period;

[0040] The difference between the estimated change of the node resource information and the estimated change of the container group resource information is used as another node that meets the preset migration condition, and is used as the node to be migrated.

[0041] In a possible implementation manner, the migration unit is specifically used for:

[0042] Based on the estimated changes in the resource information of each node, respectively evaluate the comprehensive node resource conditions corresponding to each of the other nodes in the future time period;

[0043] Determine, based on the estimated change of the container group resource information, the comprehensive container group resource status of the execution container group to be migrated in the future time period;

[0044] In each resource dimension, the overall similarity between the resource situation of the comprehensive container group and the resource situation of each comprehensive node is compared respectively;

[0045] Another node whose overall similarity in each resource dimension is lower than a preset similarity threshold is used as the node to be migrated.

[0046] In a possible implementation manner, the comprehensive node resource situation includes: the comprehensive node resource allocation situation of the corresponding other nodes under the resource dimensions, and the comprehensive node resource usage situation; the comprehensive container group resource situation includes: the comprehensive container group resource allocation situation of the execution container group to be migrated under the resource dimensions, and the comprehensive container group resource usage situation;

[0047] The migration unit is specifically used for:

[0048] In each resource dimension, a first similarity between the resource allocation of the integrated container group and the resource allocation of each integrated node is compared, and a second similarity between the resource usage of the integrated container group and the resource usage of each integrated node is compared; and based on the first similarity and the second similarity, an overall similarity is determined.

[0049] An electronic device provided in an embodiment of the present application includes a processor and a memory, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of any one of the above-mentioned database resource scheduling methods.

[0050] An embodiment of the present application provides a computer-readable storage medium, which includes a computer program. When the computer program is executed on an electronic device, the computer program is used to enable the electronic device to execute the steps of any one of the above-mentioned database resource scheduling methods.

[0051] An embodiment of the present application provides a computer program product, which includes a computer program, and the computer program is stored in a computer-readable storage medium; when a processor of an electronic device reads the computer program from the computer-readable storage medium, the processor executes the computer program, so that the electronic device performs the steps of any one of the above-mentioned database resource scheduling methods.

[0052] The beneficial effects of this application are as follows:

[0053] The embodiments of the present application provide a method, system, device and storage medium for resource scheduling of a database. After a database cluster creates a new initial container group, a suitable placement node is selected for the initial container group.

[0054] First, determine the remaining node resources of each node in the database cluster, that is, the size of various resources that can be allocated on each node. Furthermore, the present application takes into account that the resource occupancy of the initial container group after placement often changes dynamically. The method of allocating a node that can meet the application amount of the initial container group only based on the resource application amount of the initial container group cannot adapt to such dynamic changes. Therefore, the present application obtains the historical container groups that were applied for creation by the object that applied to create the initial container group in the past, and uses the resource occupancy of the historical container groups as a reference to predict the future demand for various resources by the initial container group, so as to screen out nodes whose remaining node resources can meet the predicted demand, so that after the initial container group is subsequently placed on the node, the resource waste caused by the reduction in the resource occupancy of the initial container group, or the resource shortage caused by the increase in the resource occupancy of the initial container group, is reduced.

[0055] Furthermore, after the initial container group is scheduled to a node, the distribution difference of the latest node remaining resources on the node in each resource dimension is predicted; the node whose distribution difference meets the preset conditions is selected as the placement node of the initial container group. This method can predict whether the remaining resources of the node are evenly distributed after the initial container group is placed. The node whose latest remaining resources are relatively evenly distributed in each resource dimension is used as the placement node, which can avoid the situation where the placement node can no longer provide services for the new container group due to severe resource tilt (that is, there are still a lot of remaining resources in one or more dimensions, but the resources in other dimensions are insufficient), thereby further causing resource waste.

[0056] Other features and advantages of the present application will be described in the following description, and partly become apparent from the description, or be understood by practicing the present application. The purpose and other advantages of the present application can be realized and obtained by the structures specifically pointed out in the written description, claims, and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0058] Figure 1 A schematic diagram of an application scenario of a database resource scheduling method provided in an embodiment of the present application;

[0059] Figure 2 An overall flow chart of a database resource scheduling method provided in an embodiment of the present application;

[0060] Figure 3 A schematic diagram for comparing differences between various remaining resource amounts provided in an embodiment of the present application;

[0061] Figure 4 A schematic diagram of representing P when the number of resource dimensions provided in an embodiment of the present application is 3;

[0062] Figure 5 A schematic diagram of a method for determining a target node provided in an embodiment of the present application;

[0063] Figure 6 A flow chart of container group migration provided in an embodiment of the present application;

[0064] Figure 7 Several types of time series diagrams provided for the embodiments of the present application;

[0065] Figure 8 A type of diagram of several time series provided in the embodiment of the present application;

[0066] Fig. 9 A schematic diagram of a fluctuation type corresponding to a fluctuation type provided in an embodiment of the present application;

[0067] Fig.10 A schematic diagram of obtaining overall similarity provided in an embodiment of the present application;

[0068] Fig.11 A quadrant division schematic diagram provided for an embodiment of the present application;

[0069] Fig.12 A schematic diagram of a database resource scheduling system provided in an embodiment of the present application;

[0070] Fig.13 A schematic diagram of the composition structure of a database resource scheduling device provided in an embodiment of the present application;

[0071] Fig.14 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0072] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the technical solution of the present application, rather than all of the embodiments. Based on the embodiments recorded in the application documents, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the technical solution of the present application.

[0073] To facilitate understanding of the technical solutions provided in the embodiments of the present application, some key terms used in the embodiments of the present application are explained here:

[0074] Database cluster: A database system formed by combining multiple database servers together. Each database server is a node. There are connections between the nodes, which can be coordinated and share data through the network.

[0075] Node: Each database server in the database cluster is a node. Each node has the ability to process read and write requests from the database. The node is usually loaded with one or more container groups.

[0076] Container group: The smallest scheduling unit in a database cluster (also called the smallest deployment unit), which represents a collection of one or more containers. A container can run a database instance, that is, a container group can carry one or more database instances. This application mainly includes the initial container group, the historical container group, and the execution container group. The initial container group is the newly created container group to be placed in the cluster, the historical container group is the resource application object for applying to create the initial container group, and the container group that has been applied in the past period of time. The execution container group is the container group loaded on the node.

[0077] Resource skew: This is caused by the imbalance of remaining resources in various resource dimensions of the node, such as more remaining CPU resources and less remaining memory and disk space resources. When the resource skew is severe, it may cause a waste of resources.

[0078] The following is a brief introduction to the design concept of the embodiment of the present application:

[0079] With the development of Internet technology, database clusters are gradually replacing traditional single databases with their high reliability, scalability, and high fault tolerance. Compared with a single database, there are multiple nodes in a database cluster. In order to ensure the stability of the database cluster, it is necessary to reasonably schedule the resources of each node, that is, the container group that records the data to be processed needs to be allocated to the appropriate node.

[0080] The resource scheduling strategy in the related technology is often: according to the number of resources applied for by each container group and the remaining number of resources of each node, each container group is allocated to any node capable of accommodating. However, the occupation of resources by the container group is not static, but will change over time. Node allocation based only on the number of resources applied for by the container group cannot adapt well to such dynamic changes. Furthermore, the demand for various resources by the container group is often uneven. It is easy for a container group to have a large demand for one or more resources but a small demand for other resources. If the allocation is based only on the number of resources applied for by the container group without considering the problem of resource skew, it may result in a large amount of a certain resource on some nodes being left, but because the usage of another resource has reached the limit, it can no longer be allocated to other container groups, resulting in low resource utilization of the node.

[0081] Based on this, the embodiment of the present application provides a method, system, device and storage medium for resource scheduling of a database. When selecting a placement node (i.e., a target node) for the initial container group, the resource occupation of the historical container group that was applied for by the object that applied to create the initial container group in the past is used to predict the demand for various resources of the initial container group in the future, i.e., predict the demand resources. After that, the target node is selected from the nodes that can meet the predicted demand resources. The predicted demand resources can reflect the resource demand of the initial container group in the future. Compared with the method of assigning a node that can meet the application amount of the initial container group to the initial container group only based on the application amount of the initial container group, the method proposed in the present application can better adapt to the dynamic changes of the resource occupation of the initial container group in the future, reduce the waste of resources caused by the decrease in the resource occupation of the initial container group, or the shortage of resources caused by the increase in the resource occupation of the initial container group, and further reduce the situation that the container group needs to be frequently migrated due to insufficient node resources in the future.

[0082] Furthermore, the present application also takes into account the problem of resource tilt. In order to avoid the resource waste problem caused by serious resource tilt, the present application will predict the distribution difference of the latest node remaining resources on a node in each resource dimension after the initial container group is scheduled to the node; select the node whose distribution difference meets the preset conditions as the placement node of the initial container group, that is, select the node with the latest remaining resources distributed more evenly in each resource dimension as the placement node, so as to avoid the situation where the placement node can no longer provide services for the new container group due to severe resource tilt in the future, thereby improving resource utilization.

[0083] The preferred embodiments of the present application are described below in conjunction with the drawings in the specification. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present application, and are not used to limit the present application. In addition, the embodiments and features in the embodiments of the present application may be combined with each other if there is no conflict.

[0084] like Figure 1 As shown, it is a schematic diagram of an application scenario of an embodiment of the present application. The application scenario diagram includes a database cluster 110, a resource detection device 120 and a scheduling device 130.

[0085] In an embodiment of the present application, the database cluster 110 includes multiple nodes, each of which can be a database server; the resource detection device 120 is responsible for detecting the resource status on each node and the resource occupancy of the container group on each node, and can also perform data processing on the detected data related to the resources. It can be a background server of a platform specifically used for performing resource detection on nodes, for example, it can be an independent physical server, or it can be a server cluster or distributed system composed of multiple physical servers, or it can be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (Content Delivery Network, CDN), and big data and artificial intelligence platforms; the scheduling device 130 is responsible for predicting the resource requirements of the initial container group based on the data detected by the resource detection device 120, and predicting the resource status after the initial container group is placed on the node. It can also be a server specifically used for resource scheduling.

[0086] It should be noted that the resource scheduling method of the database in each embodiment of the present application is mainly executed jointly by the resource detection device 120 and the scheduling device 130. When there is a newly created initial container group in the database cluster, the resource detection device 120 can determine the node remaining resources of the corresponding nodes based on the node resource information of each node contained in the detected database cluster; the resource detection device 120 can send the information of the relevant resources, such as node resource information and node remaining resources, to the scheduling device 130.

[0087] The scheduling device 130 can predict the predicted demand resources of the initial container group in the future based on the resource occupancy information of the historical container group belonging to the same resource application object as the initial container group, and select at least one candidate node from each node whose remaining resources meet the preset demand resources.

[0088] Afterwards, for each candidate node, the scheduling device 130 predicts the resource inclination of the latest node remaining resources on a candidate node after the initial container group is scheduled to the candidate node, and then selects a candidate node with a more balanced resource distribution, that is, a candidate node with a smaller resource inclination, as the target node, and schedules the initial container group to the target node.

[0089] In an optional implementation, the database cluster 110 , the resource detection device 120 , and the scheduling device 130 may communicate with each other via a communication network.

[0090] In an optional implementation, the communication network is a wired network or a wireless network.

[0091] It should be noted that Figure 1 The figure is only an example. In fact, the number of terminal devices and servers is not limited and is not specifically limited in the embodiments of the present application.

[0092] The following describes the resource scheduling method of the database provided by the exemplary embodiment of the present application in combination with the application scenario described above and with reference to the accompanying drawings. It should be noted that the above application scenario is only shown to facilitate understanding of the spirit and principles of the present application, and the implementation of the present application is not limited in this respect.

[0093] See also Figure 2 As shown, it is an implementation flow chart of a database resource scheduling method provided in an embodiment of the present application. The specific implementation process of the method is as follows:

[0094] S201: When there is a newly created initial container group in the database cluster, the resource detection device determines the node remaining resources of the corresponding nodes based on the node resource information of each node included in the database cluster.

[0095] The initial container group is created by the database cluster. One or more database instances can be stored in the initial container group. For each newly created initial container group, you need to select a suitable node for placement.

[0096] The resource detection device is responsible for collecting node resource information of each node in the database cluster, which may involve multiple resource dimensions such as memory, CPU, disk, network, read and write traffic, etc. In addition, the resource detection device can also collect the cluster resource information of the entire database cluster as a whole, as well as the container group resource information of the container group placed on each node.

[0097] When an initial container group needs to be placed, the resource detection device can obtain the node remaining resources of each node based on the resources occupied by the container group on each node.

[0098] Taking a specific scenario as an example, a resource application object can apply to a database cluster for the use of a database and select the amount of resources it applies for in each resource dimension, such as 2 CPU cores and 100G storage space. The database cluster can provide an initial container group for it based on its application amount and its business needs. For example, if its business needs require a container group with a smaller storage capacity, it can create 10 initial container groups of 10G or 5 initial container groups of 20G; if it requires a container group with a larger storage capacity, it can create 2 initial container groups of 50G or 1 initial container group of 100G. The resource application object divides the data to be stored into years and stores it in the initial container group it applies for. The data of each year is a database instance, and an initial container group can store one or more database instances.

[0099] If the database cluster is managed and orchestrated using Kubernetes (K8s), each container group can be a Pod, which is the smallest virtual scheduling unit.

[0100] S202: The scheduling device obtains predicted required resources of the initial container group based on resource occupancy information of at least one historical container group associated with the initial container group.

[0101] The initial container group and the historical container group are created by the same resource application object.

[0102] In the related art, when selecting a node for placing a newly created initial container group, it is usually directly based on the application amount of various resources by the resource application object, and a node that can meet its application amount is selected for placement. However, the application amount is a static data, and the occupation of resources on the node by the container group is often not static. Take the disk as an example. For example, the resource application object applies for 100G of disk space, and the database cluster creates a 100G initial container group for it. After the initial container group is just created and placed on a node, the resource application object only stores 10G of data in it. After a period of time, the resource application object gradually stores 50G of data in it, that is, the initial container group has a 50G change in the disk resource dimension. Further, the database cluster allocates 100G of disk space for it, but the actual usage is only 60G at most, that is, the utilization rate is 60%.

[0103] It can be seen that in the above case, the strategy of selecting the node to be placed based solely on the application amount may cause a waste of resources.

[0104] Therefore, the present application proposes that at least one historical container group applied for by the resource application object in the past can be obtained, and the occupation of resources on the node by the historical container group in the past period of time can be used as a reference to predict the future demand for resources on the node by the initial container group. The occupation situation can be the change in the utilization rate of the container group in various resource dimensions over a period of time. That is, the resource occupation of various resources (such as CPU, memory, disk, etc.) on the node by the initial container group in the future can be predicted based on the change in utilization rate.

[0105] For example, if there is only one historical container group, for each resource dimension, if the usage change is a set of discrete data that changes over time, the mean of multiple data can be used as the predicted demand resources of the initial container group under this resource dimension; if it is continuous data, the mean can be obtained by dividing the definite integral by the length of the interval.

[0106] If there are multiple historical container groups, after obtaining the data means of each historical container group under the resource dimension, the mean of each data mean can be further calculated, and the result can be used as the predicted demand resource of the initial container group under the resource dimension.

[0107] In addition, the predicted required resources of the initial container group in each resource dimension may also be determined based on the highest resource usage rate of the historical container group in each resource dimension over a period of time, which is not specifically limited in this application.

[0108] An optional implementation is to train a model to predict the changes in resource demand of the initial container group in the future, and further obtain the predicted demand resources based on the demand changes. However, it should be noted that the continuous addition of new container groups will lead to a decrease in the prediction performance of the model, so it is necessary to regularly re-collect data to train a new model. Predicting the future resource demand of the newly created initial container group can be defined as a regression prediction problem, which can be predicted using regression prediction algorithms such as K-nearest neighbor regression, regression decision tree, adaptive boosting learning (English: Adaptive Boosting, AdaBoost), and regression random forest.

[0109] S203: The scheduling device selects at least one candidate node from each node whose node remaining resources meet the preset required resources, and performs the following operations for each candidate node: predicting the resource tilt of the latest node remaining resources on a candidate node after scheduling the initial container group to the candidate node.

[0110] S204: The scheduling device selects a candidate node whose resource inclination condition meets the preset resource distribution condition as the target node, and schedules the initial container group to the target node.

[0111] Among them, resource tilt is characterized by the difference in the distribution of node surplus resources in various resource dimensions. Resource tilt refers to the imbalance of various resources on the node, some resources have more surplus, and some resources have less surplus. When the degree of resource tilt is more serious, it may cause waste of resources.

[0112] In the prior art, nodes are allocated only based on the resource application amount of the newly created initial container group, and the application amount has no direct correlation with the resource demand of the initial container group in the later stage. Therefore, this method is likely to increase the imbalance of resources on the nodes, that is, the degree of resource tilt, thereby limiting the improvement of cluster resource allocation rate and utilization rate, resulting in low resource utilization. Therefore, this application adds resource tilt into the consideration criteria for selecting target nodes.

[0113] For each node whose remaining resources meet the preset required resources, one or more candidate nodes need to be selected first. The present application proposes an evaluation strategy to evaluate the remaining resources of each node. Finally, one or more nodes with higher evaluation values ​​can be selected as candidate nodes based on the evaluation values ​​of each node.

[0114] An optional implementation is to perform the following operations for each node: determine the evaluation value of the node remaining resources of a node based on the difference between the node remaining resources of a node and the predicted required resources of the initial container group, combined with the partition size of a node in each resource dimension; and select at least one node whose evaluation value meets the preset selection conditions as a candidate node.

[0115] Exemplarily, the node remaining resources of a node may be quantified as follows:

[0116]

[0117] Among them, r is the vector representation of the node remaining resources of the corresponding node, which includes the remaining amount of resources under each resource dimension; r(i) represents the node remaining resources under the i-th resource dimension (for example, if it is the CPU resource dimension, it is the remaining amount of CPU resources), d is the number of resource dimensions, if the resource dimension includes CPU, memory and disk, then d = 3; α and β are preset weights, mod is the remainder operation, fi is the scaling number of the partition size of the corresponding node under the i-th resource dimension, which can be set according to the resource size applied by the resource application object and can be adjusted at any time. For example, assuming that in the past month, most of the requests for the i-th resource dimension (such as memory) are 2GB, then the partition size under the i-th resource dimension is 2, and fi is a scaling number of 2.

[0118] That is, in the above formula, the first term is used to evaluate the size of the node's remaining resources, and the second term is used to evaluate the fragmentation.

[0119] Furthermore, the predicted demand resources of the initial container group in each resource dimension are brought into it to obtain the evaluation value of the remaining resources of the node, that is, l(rI), where I is the vector representation of the predicted demand resources of the initial container group, which includes each resource dimension.

[0120] Afterwards, the nodes can be sorted from large to small according to the evaluation value, and the nodes in the first few orders are selected as candidate nodes; or, a node z with the highest evaluation value of the remaining resources of a node is selected, and the difference between the evaluation values ​​of other nodes and the evaluation value of the node z is compared in each resource dimension, and the nodes whose differences with the evaluation values ​​of the node z in each resource dimension meet the preset difference, as well as the node z, are selected as candidate nodes.

[0121] l(r z (i)-I(i))-l(r(i)-I(i))<δ; 1≤i≤d

[0122] Among them, δ is the preset difference, r z is the vector representation of the node remaining resources of node z, r is the vector of the node remaining resources of other nodes except node z, r∈M, M includes: each node whose node remaining resources can meet the preset demand resources; i represents the i-th resource dimension, and d is the number of resource dimensions.

[0123] In addition, the resources that can be allocated on the candidate node must also satisfy the resources requested by the resource application object.

[0124] After obtaining the candidate nodes, it is necessary to further determine the target node among the candidate nodes. Considering that the node allocation methods in the related art do not take into account the situation of node resource tilt, this application proposes a method for obtaining the target node, aiming to avoid the situation of serious waste of resources caused by serious node resource tilt.

[0125] An optional implementation is to predict the latest node remaining resources on a candidate node after the initial container group is scheduled to the candidate node based on the difference between the node remaining resources of a candidate node and the predicted required resources of the initial container group; then, for each preset resource dimension, perform the following operations: based on the latest node remaining resources, determine the remaining resource amount of a candidate node in each resource dimension after the initial container group is scheduled to the candidate node; finally, determine the resource tilt situation based on the difference between the remaining resource amounts.

[0126] That is, it is predicted whether the remaining resources on a candidate node are balanced in each resource dimension (i.e., the resource tilt situation) after the initial container group is scheduled to the candidate node. Then, the target node is determined according to the resource tilt situation. Next, the present application provides three methods for obtaining the target node.

[0127] Method 1: Among the candidate nodes, the candidate nodes whose difference between the remaining resource quotas meets the preset resource difference threshold are selected as the target nodes.

[0128] like Figure 3 As shown, a schematic diagram for comparing the differences between various remaining resource amounts provided in an embodiment of the present application is provided. Taking the resource dimensions including CPU, memory and disk space as an example, it is assumed that the remaining amounts (i.e., remaining resource amounts) of a candidate node in these three resource dimensions are 50% remaining for CPU, 60% remaining for memory, and 65% remaining for disk space, respectively. It is further assumed that the resource difference threshold is 10%. Since the difference between the remaining resources under the CPU and the remaining resources under the disk space is greater than 10%, the candidate node does not meet the requirements.

[0129] In addition, the above judgment method is only an example. The remaining amounts under each resource dimension can also be set to be sorted by size. As long as the difference between two adjacent remaining amounts does not exceed the resource difference threshold, the candidate node is judged to meet the requirements, etc. This application does not make specific limitations.

[0130] In specific applications, the above method can also be implemented in a vector manner, that is, the remaining resources of the candidate node in each resource dimension are represented by a vector r, and the predicted required resources of the initial container group in each resource dimension are represented by a vector I, and the dimensions of r and I depend on the number of resource dimensions. Then, the angle of the difference vector between the two vectors is calculated. The closer the angle is to the preset angle P, the more balanced the latest remaining resources of the candidate node in each resource dimension are, and the smaller the resource tilt is, after the initial container group is placed on the candidate node.

[0131] If the number of resource dimensions is 2, the preset angle P is 45 degrees. If the number of resource dimensions is greater than 2, P can be represented by a vector, specifically a vector starting from the origin with equal distances from each coordinate axis. For example, in four dimensions, P can be (1,1,1,1), and in five dimensions, P can be (1,1,1,1,1,1). However, no matter how high the dimension is, the distance from P to each coordinate axis is equal.

[0132] Since high-dimensional images are difficult to draw, the following example takes the case where the number of resource dimensions is 3 as an example. Figure 4As shown, a schematic diagram of P when the number of resource dimensions provided in an embodiment of the application is 3, the resource dimensions in the figure, that is, the three coordinate axes are CPU, memory and disk, and the P vector is projected to the CPU-memory plane and the disk-memory plane. According to the coordinates of the two projections, it can be seen that the distances from the P vector to the three coordinate axes are equal, all a, which means absolute balance of resources under the three resource dimensions. Therefore, the closer the angle of the difference vector between the two vectors is to P, the smaller the resource inclination of the node is after the initial container group is placed on the node.

[0133] The target node can also be determined by the following formula, the principle is the same as above:

[0134] Resource tilt degree = θ(rI)

[0135]

[0136] Wherein, argmin represents the value of the independent variable (r) when the function reaches the minimum value, which is the candidate node corresponding to the vector r when θ(rI) is the minimum in this application; θ(rI) is the angle between the vector r and the vector I, θ(rI)∈[0,π / 2).

[0137] Furthermore, although the angle between two vectors in high dimensions cannot be displayed intuitively, it can be quantified by the cosine similarity between the two vectors.

[0138] Method 2: Based on the difference between the remaining resource quotas corresponding to each candidate node, the bottleneck resources corresponding to each candidate node are determined respectively, and the remaining resource quotas under the bottleneck resources corresponding to each candidate node are compared across resource dimensions. Based on the comparison results, a target node is determined among the candidate nodes.

[0139] This method measures resource tilt by bottleneck resources, where bottleneck resources are the resources with the lowest remaining resource quota in each resource dimension of the corresponding candidate node. Since bottleneck resources are more likely to be exhausted, in order to prevent insufficient resource provision, this method selects the candidate node with the largest bottleneck resource as the target node.

[0140] The specific formula can be as follows:

[0141]

[0142] Among them, argmax represents the value of the independent variable (r) when the function reaches the maximum value, i represents the i-th resource dimension, d is the number of resource dimensions, min is 1≤i≤d (rI)(i) is the remaining amount of resources of the node under the bottleneck resource.

[0143] Take a specific scenario as an example, Figure 5 As shown, it is a schematic diagram of a method for determining a target node provided by an embodiment of the present application, assuming that there are currently three candidate nodes, and the resource dimensions include cpu and memory. Candidate node 1 has 70% remaining cpu and 30% remaining memory, then for candidate node 1, the bottleneck resource is memory, and the remaining amount is 30%, candidate node 2 has 40% remaining cpu and 50% remaining memory, then for candidate node 2, the bottleneck resource is cpu, and the remaining amount is 40%, candidate node 3 has 60% remaining cpu and 50% remaining memory, then for candidate node 3, the bottleneck resource is memory, and the remaining amount is 50%, that is, so candidate node 1 has 30% remaining bottleneck resource, candidate node 2 has 40% remaining bottleneck resource, candidate node 3 has 50% remaining bottleneck resource, the largest of which is 50% of candidate node 3, that is, candidate node 3 is selected as the target node.

[0144] Method 3: Under each resource dimension, determine the resource similarity between the predicted demand resources and the node remaining resources of a candidate node respectively; based on the resource similarity, predict the resource inclination of the latest node remaining resources on a candidate node after scheduling the initial container group to a candidate node, and then select a candidate node whose resource similarity meets the preset similarity condition as the target node.

[0145] Compared with the first two methods, method 3 is slightly different. It does not simulate placing the initial container group on the node by calculating the difference between the remaining resources of the candidate node in each resource dimension and the predicted demand resources of the initial container group in each resource dimension, and further predict the remaining resources on the node after placement. Instead, it infers the resource tilt of the latest remaining node resources on the candidate node after placement based on the similarity between the actual remaining resources of the candidate node and the predicted demand resources before placement. The higher the resource similarity, the more balanced the resources, that is, the lower the tilt.

[0146] In specific applications, this method can also be implemented in a vector manner. The remaining resources of the candidate node in each resource dimension are still represented by a vector r, and the predicted demand resources of the initial container group in each resource dimension are represented by a vector I. The larger the dot product I·r between the two vectors, the higher the similarity between the two vectors. The candidate node corresponding to r when I·r is the largest can be used as the target node. The specific formula can be as follows:

[0147]

[0148] In summary, the method for determining the target node proposed in this application takes into account the balanced allocation under multiple resource dimensions to ensure that the allocation result does not cause serious resource tilt on the node. Especially when the evaluation values ​​of the node remaining resources of multiple candidate nodes are close, the resource tilt situation becomes a key criterion. Therefore, in the above method, the resource tilt problem can be solved by measuring the resource tilt situation according to the above three different methods.

[0149] After the target node is selected, the initial container group is placed on the target node.

[0150] In addition to the need to select a target node when placing the newly created initial container group, the database cluster also has the situation of container group migration. As business access changes, cluster resource tilt may occur due to sudden business access, disaster recovery triggering redundant container groups, etc. Or, as time goes by, the number of execution container groups on a node increases, resulting in a shortage of resources on the node. At this time, one or more execution container groups on the node need to be migrated.

[0151] In the prior art, the migration of container groups lacks the analysis of the resource occupancy information of container groups in the historical time period. It only considers the migration based on the current value of each dimension resource at the time of migration. This may cause a large number of container groups to be allocated to a node, and the resources of the node cannot meet the demand, so the container group needs to be migrated, resulting in frequent migration. In addition, the prior art also does not consider the resource tilt problem caused by container group migration. The utilization rate and allocation rate of each node in the database cluster are tilted in terms of resource dimensions such as CPU and memory. For example, a node may have a low memory utilization rate but a high CPU utilization rate. This will result in the inability to host the migrated container group due to insufficient CPU resources during migration, but a large amount of remaining memory is wasted, resulting in low resource utilization of the database cluster.

[0152] Based on this, Figure 6 As shown, it is a flow chart of container group migration provided by an embodiment of the present application, and the specific implementation process is as follows:

[0153] S601: The scheduling device selects nodes whose corresponding node resource information exceeds a preset threshold among the nodes as nodes to be migrated out.

[0154] The node resource information mentioned above includes various resource dimensions. Furthermore, the resource detection device can further divide it into allocation status and usage status, that is, the amount of resources that the node has allocated to each execution container group (under each resource dimension), and the amount of resources actually used by all execution container groups on the node (under each resource dimension).

[0155] Assume that there are two execution container groups on a node in the database cluster. The node has a total of 1000G of disk space. 100G of disk space is allocated to execution container group 1 and 200G of disk space is allocated to execution container group 2. However, at a certain moment, execution container group 1 actually only uses 50G and execution container group 2 uses 150G. In the resource dimension of disk space, the node's allocation rate is 30% and its utilization rate is 20%. The allocation rate of execution container group 1 is 10% and its utilization rate is 50%. The allocation rate of execution container group 2 is 20% and its utilization rate is 75%.

[0156] The preset thresholds may be specifically set according to specific circumstances. For example, a threshold may be set for the allocation and usage corresponding to each resource dimension. When one or more of the thresholds are exceeded, the node is considered a node to be migrated out.

[0157] Furthermore, for each node that exceeds the threshold, a multidimensional vector (the vector dimension is related to the number of resource dimensions) can be constructed according to the resource allocation rate of the node in each resource dimension, and a multidimensional vector (the vector dimension is related to the number of resource dimensions) can be constructed according to the resource utilization rate of the node in each resource dimension, and the vector sum of the two vectors can be calculated, and the above nodes can be sorted according to the size of the vector sum, and several nodes with larger vector sums can be selected as nodes to be migrated.

[0158] For example, assuming that the resource dimensions include CPU, memory and disk space, the allocation rate of node A in the CPU resource dimension is 70%, and the utilization rate is 40%, the allocation rate in the memory resource dimension is 50%, and the utilization rate is 40%, and the allocation rate in the disk space resource dimension is 60%, and the utilization rate is 60%. Then the vector of resource allocation rate corresponding to the node can be [0.7, 0.5, 0.6], and the vector of resource utilization rate corresponding to the node can be [0.4, 0.4, 0.6].

[0159] S602: For each node to be migrated, the scheduling device obtains container group resource information corresponding to each execution container group of the load on the node to be migrated; and uses the execution container group whose corresponding container group resource information meets the migration condition as the execution container group to be migrated.

[0160] In the above, the container group resource information represents the resource consumption of the corresponding execution container group (under each resource dimension). Similar to the node resource information, for the container group resource information, the resource detection device can further divide it into allocation status and usage status, that is, the amount of resources that have been allocated to the execution container group (under each resource dimension) and the amount of resources actually used by the execution container group (under each resource dimension).

[0161] The container group resource information can be detected by the resource detection device and then sent to the scheduling device.

[0162] The node to be migrated usually has multiple execution container groups loaded on it, and the execution container groups with greater resource requirements are preferentially migrated.

[0163] Specifically, for each execution container group under a node to be migrated, a multidimensional vector (the vector dimension is related to the number of resource dimensions) can be constructed according to the resource allocation rate of the execution container group under each resource dimension, and a multidimensional vector (the vector dimension is related to the number of resource dimensions) can be constructed according to the resource utilization rate of the execution container group under each resource dimension, and the vector sum of the two vectors is calculated, and the above execution container groups are sorted according to the size of the vector sum, and several execution container groups with larger vector sums are selected as the execution container groups to be migrated.

[0164] For example, assuming that the resource dimensions include CPU, memory, and disk space, the allocation rate of execution container group B in the CPU resource dimension is 20%, and the utilization rate is 80%, the allocation rate in the memory resource dimension is 20%, and the utilization rate is 70%, and the allocation rate in the disk space resource dimension is 10%, and the utilization rate is 90%, then the vector of resource allocation rate corresponding to the execution container group B can be [0.2, 0.2, 0.1], and the vector of resource utilization rate corresponding to the execution container group B can be [0.8, 0.7, 0.9], and the vector sum of the two vectors is [1, 0.9, 1]. Perform the above operation on each execution container group, and finally determine the container group to be migrated by comparing the modulus of the resource utilization rate vector corresponding to each execution container group and the vector sum of the resource allocation rate vector.

[0165] S603: The scheduling device determines a node to be migrated in from other nodes except the node to be migrated out, and migrates the execution container group to be migrated to the node to be migrated in.

[0166] The migration of container groups also needs to take resource skew into consideration. Therefore, it is necessary to ensure that the remaining resources in each resource dimension of the node to be migrated can remain relatively balanced after receiving the execution container to be migrated.

[0167] An optional implementation is to predict the estimated changes of node resource information of each other node in a preset future time period based on the historical changes of node resource information of each other node in a preset historical time period; predict the estimated changes of container group resource information of the execution container group to be migrated in a preset future time period based on the historical changes of container group resource information of the execution container group to be migrated in the preset historical time period; and use the difference between the estimated changes of node resource information and the estimated changes of container group resource information as one of the other nodes that meets the preset migration conditions as a node to be migrated.

[0168] In the above, the historical changes of node resource information of each other node over time can be in the form of a time series, and the prediction of the estimated changes can be completed based on the prediction model. The model can be a long short-term memory network, a convolutional neural network, a recurrent neural network, a Transformer, etc. The most suitable prediction model can be selected according to the actual situation of the database cluster to guide the subsequent container group migration.

[0169] In order to reduce the cost of training a model for each time series separately and reduce overfitting, clustering can be used, that is, clustering similar time series, training a node prediction model for each cluster, and using the trained node prediction model to predict the estimated changes in node resource information. When there is another node that needs to be predicted, first analyze the type of the time series based on the historical changes in its corresponding node resource information, that is, a time series, and input it into the corresponding node prediction model according to its type. The node prediction model outputs the estimated changes in node resource information in the future time period.

[0170] The similarity between time series can be calculated by dynamic time warping algorithm, K-Means clustering algorithm, etc., which is not specifically limited in this application.

[0171] The same is true for the historical changes in the container group resource information of the execution container group to be migrated and the prediction of the estimated changes in the container group resource information. The historical changes in the container group resource information can also be a time series, and the container group prediction model can be used to predict the estimated changes in the container group resource information in the future time period; the training of the container group prediction model can also be completed by clustering, that is, for the time series corresponding to different container groups, first cluster similar time series, and train a container group prediction model for each cluster. When there is an execution container group to be migrated that needs to be predicted, first analyze the type of the time series based on the historical changes in the container group resource information corresponding to the execution container group to be migrated, that is, a time series, and input it into the corresponding container group prediction model according to its type. The container group prediction model outputs the estimated changes in the container group resource information in the future time period.

[0172] Compared with the predicted resource requirements of the initial container group, the migration of the execution container group requires more detailed and real-time data. The migration method proposed in this application can observe the estimated changes in the future container group resource information of the execution container group from an advanced perspective before migrating the container group. This can avoid the situation where multiple sudden execution container groups in the same time range are migrated to the same node, resulting in reduced node availability and excessive subsequent migration times.

[0173] Regardless of nodes or container groups, time series can be divided into different types according to their trend, volatility, and periodicity. Figure 7 As shown, there are several types of time series provided in the embodiments of the present application, including continuous rising type, continuous falling type, periodic rising type, periodic falling type, stable type, fluctuating type, etc.

[0174] In the above description, the historical changes of a node resource information and the historical changes of a container group resource information both correspond to a time series, which integrates resources under various resource dimensions, such as CPU, memory, disk, etc.; in addition, the historical changes of a node resource information can also correspond to multiple time series, each time series corresponds to a resource dimension, and the historical changes of the container group resource information are similar, and this application does not make specific limitations.

[0175] Then, the node to be migrated is selected based on the difference between the estimated change of the node resource information and the estimated change of the container group resource information.

[0176] Exemplarily, the node to be migrated may be determined in the following two ways.

[0177] Method 1: The estimated change of node resource information and the estimated change of container group resource information can also be divided into various types such as continuous increase type, continuous decrease type, periodic increase type, periodic decrease type, stable type, and fluctuating type. The node to be migrated can be directly selected for the execution container group to be migrated according to the preset correspondence between the types. For example, the continuous increase type corresponds to the continuous decrease type, the stable type corresponds to the stability type, the periodic increase type corresponds to the periodic decrease type, and the fluctuating type corresponds to the fluctuating type. That is, if the estimated change of the container group resource information is the continuous increase type, then another node whose estimated change of the node resource information is the continuous decrease type is selected as the node to be migrated.

[0178] Among them, the correspondence between cyclical rise and cyclical decline needs to take into account cyclical issues, such as Figure 8 As shown, it is a schematic diagram of a periodic rise corresponding to a periodic decline provided by an embodiment of the present application, that is, the two time series need to be kept consistent in period as much as possible; the fluctuation type-fluctuation type correspondence requires staggering the peaks of both sides, such as Fig. 9 As shown, it is a schematic diagram of a fluctuation type corresponding to a fluctuation type provided in an embodiment of the present application. The two time series need to ensure as much as possible that the peak of one of them corresponds to the trough of the other.

[0179] Method 2: Based on the estimated changes in the resource information of each node, evaluate the comprehensive node resource situation of each other node in the future time period; based on the estimated changes in the container group resource information, determine the comprehensive container group resource situation of the execution container group to be migrated in the future time period; in each resource dimension, compare the overall similarity between the comprehensive container group resource situation and the comprehensive node resource situation. Finally, select another node whose overall similarity in each resource dimension is lower than the preset similarity threshold as a node to be migrated.

[0180] In the above, the comprehensive node resource situation includes: the comprehensive node resource allocation situation of the corresponding other nodes in each resource dimension, and the comprehensive node resource usage situation; the comprehensive container group resource situation includes: the comprehensive container group resource allocation situation of the execution container group to be migrated in each resource dimension, and the comprehensive container group resource usage situation.

[0181] Taking another node and a resource dimension as an example, the first similarity between the comprehensive container group resource allocation of the execution container group to be migrated and the comprehensive node resource allocation of the other node can be compared, and the second similarity between the comprehensive container group resource usage and the comprehensive node resource usage can be compared; then, the overall similarity under the resource dimension is determined based on the first similarity and the second similarity.

[0182] The estimated change of a node resource information or the estimated change of a container group resource information can be represented in the form of multiple time series. Assuming that the resource dimensions include memory and CPU, the estimated change of the node resource information may include the estimated change of memory usage, the estimated change of memory allocation rate, the estimated change of CPU usage and the estimated change of CPU allocation rate; the estimated change of the container group resource information also includes the above four types.

[0183] like Fig.10As shown, it is a schematic diagram of obtaining overall similarity provided by an embodiment of the present application. Assuming that the resource dimension includes memory and CPU, for another node, according to the estimated change of its node resource information, the comprehensive node resource allocation of the other node under the CPU resource dimension (referred to as CPU allocation in the figure), the comprehensive node resource usage under the CPU resource dimension (referred to as CPU usage in the figure), the comprehensive node resource allocation under the memory resource dimension (referred to as memory allocation in the figure), and the comprehensive node resource usage under the memory resource dimension (referred to as memory usage in the figure); for the execution container group to be migrated, according to the estimated change of its container group resource information, the comprehensive container group resource allocation of the container group under the CPU resource dimension (referred to as CPU allocation in the figure), the comprehensive container group resource usage under the CPU resource dimension (referred to as CPU usage in the figure), the comprehensive container group resource allocation under the memory resource dimension (referred to as memory allocation in the figure), and the comprehensive container group resource usage under the memory resource dimension (referred to as memory usage in the figure).

[0184] Then, the overall similarity in the CPU resource dimension is obtained according to the first similarity between the two CPU allocation situations and the second similarity between the two CPU usage situations; and the overall similarity in the memory resource dimension is obtained according to the first similarity between the two memory allocation situations and the second similarity between the two memory usage situations.

[0185] Assuming that the estimated changes in memory usage of other nodes 1 in the next 5 minutes are 15%, 30%, 35%, 45%, and 50%, the estimated changes in memory allocation rate are 30%, 30%, 60%, 60%, and 60%, the estimated changes in CPU usage are 40%, 42%, 45%, 50%, and 57%, and the estimated changes in CPU allocation rate are 30%, 30%, 55%, 55%, and 55%. An optional implementation method is to take the average of the above four groups of data respectively to obtain the comprehensive node resource allocation and comprehensive node resource usage of other nodes 1 in the memory resource dimension, as well as the comprehensive node resource allocation and comprehensive node resource usage in the CPU resource dimension.

[0186] Similarly, assuming that the estimated change in memory usage of the execution container group to be migrated in the next 5 minutes is 70%, 80%, 80%, 80%, 75%, the estimated change in memory allocation rate is 30%, 30%, 30%, 30%, 30%, 30%, the estimated change in CPU usage is 60%, 66%, 67%, 66%, 58%, and the estimated change in CPU allocation rate is 20%, 20%, 20%, 20%, 20%, an optional implementation method is to obtain the average of the above four groups of data respectively to obtain the comprehensive container group resource allocation and comprehensive container group resource usage of the execution container group to be migrated in the memory resource dimension, as well as the comprehensive container group resource allocation and comprehensive container group resource usage in the CPU resource dimension.

[0187] In addition, the data group corresponding to the estimated change can be directly represented in the form of a vector. For example, in the above, the estimated changes in the memory usage of other nodes 1 in the next 5 minutes are 15%, 30%, 35%, 45%, and 50%. The vector form can be [15, 30, 35, 45, 50]. This vector is used as the comprehensive node resource usage of other nodes 1 in the memory resource dimension. The same applies to the comprehensive node resource allocation, comprehensive container group resource allocation, and comprehensive container group resource usage.

[0188] It should be noted that the above-mentioned methods for obtaining the comprehensive node resource allocation and comprehensive node resource usage of the nodes in each resource dimension, as well as the comprehensive container group resource allocation and comprehensive container group resource usage of the container group in each resource dimension are only examples, and this application does not make specific limitations.

[0189] Afterwards, a first similarity between the comprehensive node resource allocation and the comprehensive container group resource allocation under the memory resource dimension, and a second similarity between the comprehensive node resource usage and the comprehensive container group resource usage are calculated, and the overall similarity between other nodes 1 and the execution container group to be migrated under the memory resource dimension is obtained according to the first similarity and the second similarity under the memory resource dimension.

[0190] Based on the same method, the overall similarity between other nodes 1 and the execution container group to be migrated in the CPU resource dimension can also be obtained. Finally, another node whose overall similarity in each resource dimension is lower than the preset similarity threshold is used as a node to be migrated, that is, the similarity between the occupied resources of the node to be migrated and the resources required by the execution container group to be migrated should be as low as possible, or in other words, the similarity between the remaining resources of the node to be migrated and the resources required by the execution container group to be migrated should be as high as possible.

[0191] In the specific implementation process, method 2 can be implemented by quadrant division. Fig.11 As shown, a quadrant division schematic diagram provided by an embodiment of the present application is provided. In the figure, based on the utilization rate of resources and the allocation rate of resources, the two-dimensional coordinate system is divided into four quadrants, wherein the first quadrant has both high utilization rate and high allocation rate, the second quadrant has low utilization rate but high allocation rate, the third quadrant has low utilization rate and low allocation rate, and the fourth quadrant has high utilization rate but low allocation rate. The dividing line between high utilization rate and low utilization rate, and high allocation rate and low allocation rate can be determined based on actual conditions. Each resource dimension corresponds to a quadrant diagram. Assuming that the resource dimension includes CPU and memory, there are CPU quadrant diagrams and memory quadrant diagrams. The quadrant diagrams of nodes and container groups under the same resource dimension can be the same, or they can be set separately.

[0192] For each other node, its utilization rate and allocation rate in each dimension can be determined according to its comprehensive node resource utilization and comprehensive node resource allocation in each dimension, and it can be allocated to the corresponding quadrant in the quadrant diagram corresponding to each resource dimension; similarly, according to the comprehensive container group resource utilization and comprehensive container group resource allocation in each dimension of the execution container group to be migrated, the utilization rate and allocation rate of the container group in each dimension can be determined, and it can be allocated to the corresponding quadrant in the quadrant diagram corresponding to each resource dimension. Furthermore, the utilization rate and allocation rate of the node in each resource dimension can be marked on the node in the form of a label, and the utilization rate and allocation rate of the container group in each resource dimension can be marked on the container group in the form of a label.

[0193] For each resource dimension, select other nodes in mutually exclusive quadrants with the quadrant where the execution container group to be migrated is located. Assuming that the execution container group to be migrated has low memory usage, low memory allocation rate, high CPU usage, and low CPU allocation rate, that is, it is located in the third quadrant of the memory quadrant diagram and the fourth quadrant of the CPU quadrant diagram, then among all other nodes, select other nodes that are located in the first quadrant of the memory quadrant diagram corresponding to the node and the second quadrant of the CPU quadrant diagram corresponding to the node as the nodes to be migrated.

[0194] If there are no other nodes that can meet the above requirements at the same time, then you can select another node that can meet one of the conditions, or directly select another node with the most remaining resources as the node to be migrated based on the evaluation value of the remaining resources of the node. If there are multiple other nodes that can meet the above requirements at the same time, then you can select one of them as the node to be migrated based on the evaluation value of the remaining resources of the node, and so on.

[0195] After the node to be migrated is determined, the execution container group to be migrated is migrated from the node to be migrated out to the node to be migrated in. After the migration is completed, the node to be migrated out is determined as the migrated out node, and the node to be migrated in is determined as the migrated in node.

[0196] In summary, this application can avoid serious resource skew by analyzing the similarity of resource usage and allocation between the container group and the node, and migrating the container group to a node with a reverse similarity to itself.

[0197] Thereafter, it is also possible to re-determine whether the node resource information on the migrated node exceeds a preset threshold. If yes, it is re-treated as a node to be migrated and returns to step S602; if no, the migration task for the migrated node is terminated.

[0198] In addition, since the migration operation of the container group will occupy the read and write traffic and bandwidth resources of the database cluster, and frequent migration operations during peak hours will seriously affect the performance and service quality of the cluster system, you can choose to initiate the migration plan during the off-peak period of business to avoid online traffic.

[0199] The migration method proposed in this application can also be combined with heuristic algorithms such as the best adaptation algorithm and the simulated annealing algorithm to determine the best node to be migrated by integrating the results of multiple solutions.

[0200] On the other hand, the present application also proposes a system, such as Fig.12 As shown, it is a schematic diagram of a resource scheduling system of a database provided in an embodiment of the present application. The system mainly includes a database cluster, a resource detection device and a scheduling device.

[0201] The resource detection device is mainly used to determine the node remaining resources of the corresponding nodes based on the node resource information of each node included in the database cluster when there is a newly created initial container group in the database cluster.

[0202] The resource detection device is responsible for collecting resource information of the database cluster. It can be a background server of a platform specifically used to detect database cluster resources. The resource detection device can summarize the collected resource information and send it to the front end of the platform for display. The collected information includes resource information of the cluster as a whole, each node and each container group in each resource dimension. The resource dimensions can include CPU, memory, disk, read traffic, write traffic, etc. For CPU, memory, and disk, it can be further divided into memory allocation rate, memory utilization rate, CPU allocation rate, CPU utilization rate, disk space allocation rate, disk space utilization rate, etc. It can also include the set threshold upper limit of each dimension of resources in the cluster within a reasonable range, so that when the set threshold is exceeded, the scheduling device can migrate the container group.

[0203] The resource detection device can also perform simple processing on the data, for example, according to the current occupancy of each resource (specifically, it can be the usage or allocation, or the usage and allocation, etc.), obtain the remaining amount of each resource.

[0204] The scheduling device may also be a background server of a platform dedicated to resource scheduling, which may include multiple functional modules, such as a static resource prediction module, a dynamic resource prediction module, a resource allocation module, a resource migration module and a task execution module.

[0205] Among them, the static resource prediction module can analyze the information detected by the resource detection device, and predict the predicted demand resources of the initial container group based on the resource occupancy information of the historical container group; the dynamic resource prediction module is responsible for analyzing the resource information of the nodes and container groups in the past period of time, and based on the historical changes in resource information in the historical time period, predict the estimated changes in resource information of the nodes and container groups in the future time period. Different nodes and container groups can also be divided into continuous rising type, continuous falling type, periodic rising type, periodic falling type, stable type, fluctuating type, etc. according to the changes in resource information. The static resource prediction module is mainly used to guide the placement of the newly created initial container group, and the dynamic load prediction module is mainly used to guide the migration of the execution container group.

[0206] The resource allocation module can obtain the predicted demand resources output by the static resource prediction module, select at least one candidate node from each node whose remaining resources meet the preset demand resources, and determine a target node from the candidate nodes based on the remaining resources of the node and the predicted resource requirements of the initial container group while minimizing the degree of resource inclination; that is, it mainly adopts a two-stage vectorized online placement strategy, in the first stage, a group of nodes with the largest remaining resources are selected as candidate nodes, and in the second stage, the candidate nodes with the smallest degree of resource inclination after the initial container group is placed are selected from the candidate nodes as the target nodes; the resource migration module can obtain the estimated changes in resource information output by the dynamic resource prediction module, and determine a node to be migrated based on the estimated changes in the node resource information and the estimated changes in the container group resource information.

[0207] The task execution module is used to place the initial container group to the target node according to the target node determined by the resource allocation module, and migrate the execution container group to be migrated from the node to be migrated out to the node to be migrated in according to the node to be migrated in determined by the resource migration module.

[0208] In addition, the initial container group placement and container group migration method proposed in the present application can also be applied to other application scenarios. For example, in a microservice scenario, a large application is split according to functions (such as user management function, product management function, live broadcast function, etc.), and each function is a microservice, which can also be regarded as a container group. The container group can be placed according to the resources required by the container group and the remaining resources of the node. According to the estimated changes in the node resource information and the estimated changes in the container group resource information, nodes with opposite similarities to the estimated changes in the container group are selected for migration.

[0209] It should be noted that the resource dimensions concerned may be different in different application scenarios.

[0210] In summary, the method proposed in the present application can more accurately predict the future resource requirements of the newly created initial container group and the changes in the future container group resource information of the execution container group to be migrated. Compared with the method of determining the target node or the node to be migrated based only on the node resource information (the amount of resources occupied) of each node in the database cluster during resource allocation and migration, the present application can significantly reduce the cluster resource skewness (that is, the tilt), ensure that the cluster remains balanced in allocation and usage in each resource dimension, effectively reduce the need for frequent migration in the future due to severe resource tilt, and improve the utilization of resources in the cluster.

[0211] Based on the same inventive concept, an embodiment of the present application also provides a resource scheduling device for a database.

[0212] like Fig.13 As shown, it is a schematic diagram of the structure of a resource scheduling device of a database, which may include:

[0213] The determining unit 1301 is used to determine the node remaining resources of the corresponding nodes based on the node resource information of each node included in the database cluster when there is a newly created initial container group in the database cluster;

[0214] The first selection unit 1302 is used to obtain the predicted required resources of the initial container group based on the resource occupancy information of at least one historical container group associated with the initial container group; the initial container group and the historical container group are created by the same resource application object;

[0215] The prediction unit 1303 is used to select at least one candidate node from each node whose node remaining resources meet the preset required resources, and perform the following operations for each candidate node: predict the resource tilt of the latest node remaining resources on a candidate node after scheduling the initial container group to the candidate node, where the resource tilt represents: the distribution difference of the node remaining resources in each resource dimension;

[0216] The second selection unit 1304 is configured to select a candidate node whose resource inclination condition satisfies a preset resource distribution condition as a target node, and schedule the initial container group to the target node.

[0217] In a possible implementation manner, the prediction unit 1303 is specifically configured to:

[0218] For each node, the following operations are performed respectively: based on the difference between the node remaining resources of a node and the predicted required resources of the initial container group, combined with the partition size of a node under each resource dimension, determine the evaluation value of the node remaining resources of a node;

[0219] At least one node whose evaluation value meets the preset selection condition is selected as a candidate node.

[0220] In a possible implementation manner, the prediction unit 1303 is specifically configured to:

[0221] Based on the difference between the node remaining resources of a candidate node and the predicted required resources of the initial container group, predict the latest node remaining resources on a candidate node after the initial container group is scheduled to the candidate node;

[0222] For each preset resource dimension, the following operations are performed: based on the latest node remaining resources, determine the remaining resource quota of a candidate node in each resource dimension after the initial container group is scheduled to the candidate node;

[0223] Determine the resource tilt based on the difference between the remaining resource amounts.

[0224] In a possible implementation manner, the second selection unit 1304 is specifically configured to:

[0225] The candidate nodes whose difference between the remaining resource quotas among the candidate nodes meets the preset resource difference threshold are selected as the target nodes;

[0226] or,

[0227] Based on the difference between the remaining resource amounts corresponding to each candidate node, the bottleneck resources corresponding to each candidate node are determined respectively, and each bottleneck resource is: the resource with the lowest remaining resource amount of the corresponding candidate node under each resource dimension; and the remaining resource amount under the bottleneck resource corresponding to each candidate node is compared across resource dimensions, and based on the comparison results, a target node is determined in each candidate node.

[0228] In a possible implementation manner, the prediction unit 1303 is specifically configured to:

[0229] Under each resource dimension, the resource similarity between the predicted demand resources and the node remaining resources of a candidate node is determined respectively;

[0230] Based on resource similarity, the resource tilt of the latest node remaining resources on a candidate node is predicted after the initial container group is scheduled to a candidate node. The resource similarity is negatively correlated with the resource tilt.

[0231] The second selection unit 1304 is specifically used for:

[0232] A candidate node whose resource similarity meets the preset similarity condition is selected as the target node.

[0233] In a possible implementation, the device further includes:

[0234] A migration unit 1305 is used to select nodes whose corresponding node resource information exceeds a preset threshold value as nodes to be migrated out;

[0235] For each node to be migrated, the container group resource information corresponding to each execution container group loaded on the node to be migrated is obtained; and the execution container group whose corresponding container group resource information meets the migration condition is used as the execution container group to be migrated; the container group resource information represents the resource consumption of the corresponding execution container group;

[0236] A node to be migrated into is determined from among all other nodes except the node to be migrated out, and the execution container group to be migrated is migrated to the node to be migrated in.

[0237] In a possible implementation manner, the migration unit 1305 is specifically configured to:

[0238] Based on the historical changes of the node resource information of each other node in the preset historical time period, the estimated changes of the node resource information of each other node in the preset future time period are predicted respectively;

[0239] Based on historical changes in container group resource information of the execution container group to be migrated within a preset historical time period, predict estimated changes in container group resource information of the execution container group to be migrated within a preset future time period;

[0240] The difference between the estimated change of the node resource information and the estimated change of the container group resource information is used as another node that meets the preset migration conditions and is used as a node to be migrated.

[0241] In a possible implementation manner, the migration unit 1305 is specifically configured to:

[0242] Based on the estimated changes in the resource information of each node, evaluate the comprehensive node resource conditions of each other node in the future time period;

[0243] Based on the estimated changes in the container group resource information, determine the comprehensive container group resource status of the execution container group to be migrated in the future time period;

[0244] In each resource dimension, the overall similarity between the resource situation of the comprehensive container group and the resource situation of each comprehensive node is compared;

[0245] Another node whose overall similarity in each resource dimension is lower than a preset similarity threshold is used as a node to be migrated.

[0246] In a possible implementation, the comprehensive node resource situation includes: the comprehensive node resource allocation situation of the corresponding other nodes in each resource dimension, and the comprehensive node resource usage situation; the comprehensive container group resource situation includes: the comprehensive container group resource allocation situation of the execution container group to be migrated in each resource dimension, and the comprehensive container group resource usage situation;

[0247] The migration unit 1305 is specifically used for:

[0248] In each resource dimension, a first similarity between the resource allocation of the integrated container group and the resource allocation of each integrated node is compared, and a second similarity between the resource usage of the integrated container group and the resource usage of each integrated node is compared; and based on the first similarity and the second similarity, an overall similarity is determined.

[0249] For the convenience of description, the above parts are divided into modules (or units) according to their functions and described separately. Of course, when implementing this application, the functions of each module (or unit) can be implemented in the same or multiple software or hardware.

[0250] In the embodiments of the present application, the term "module" or "unit" refers to a computer program or a part of a computer program with a predetermined function, and works together with other related parts to achieve a predetermined goal, and can be implemented in whole or in part by using software, hardware (such as processing circuits or memories) or a combination thereof. Similarly, a processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be part of an overall module or unit that includes the function of the module or unit.

[0251] After introducing the resource scheduling method and device of a database according to an exemplary embodiment of the present application, next, an electronic device according to another exemplary embodiment of the present application is introduced.

[0252] Those skilled in the art will appreciate that various aspects of the present application may be implemented as a system, method or program product. Therefore, various aspects of the present application may be specifically implemented in the following forms, namely: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software, which may be collectively referred to as "circuit", "module" or "system" herein.

[0253] Based on the same inventive concept as the above method embodiment, an electronic device is also provided in the embodiment of the present application. In one embodiment, the electronic device may be a server, such as Figure 1 Each node in the database cluster 110 shown in the figure, a backend server of a platform for detecting database cluster resources (resource detection device 120), and a backend server of a resource scheduling platform (scheduling device 130). In this embodiment, the structure of the electronic device can be as follows: Fig.14 As shown, it includes a memory 1401 , a communication module 1403 and one or more processors 1402 .

[0254] The memory 1401 is used to store computer programs executed by the processor 1402. The memory 1401 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system and programs required for running the instant messaging function, etc.; the data storage area may store various instant messaging information and operation instruction sets, etc.

[0255] The memory 1401 may be a volatile memory, such as a random-access memory (RAM); the memory 1401 may also be a non-volatile memory, such as a read-only memory, a flash memory, a hard disk drive (HDD) or a solid-state drive (SSD); or the memory 1401 may be any other medium that can be used to carry or store a desired computer program in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 1401 may be a combination of the above memories.

[0256] The processor 1402 may include one or more central processing units (CPU) or a digital processing unit, etc. The processor 1402 is used to implement the resource scheduling method of the above database when calling the computer program stored in the memory 1401 .

[0257] The communication module 1403 is used to communicate with terminal devices and other servers.

[0258] The specific connection medium between the memory 1401, the communication module 1403 and the processor 1402 is not limited in the embodiment of the present application. Fig.14 In the embodiment, the memory 1401 and the processor 1402 are connected via a bus 1404. Fig.14 The connections between the other components are only for illustration and are not intended to be limiting. The bus 1404 can be divided into an address bus, a data bus, a control bus, etc. For ease of description, Fig.14 The diagram shows that only one thick line is used, but this does not mean that there is only one bus or one type of bus.

[0259] The memory 1401 stores a computer storage medium, and the computer storage medium stores computer executable instructions, and the computer executable instructions are used to implement the resource scheduling method of the database in the embodiment of the present application. The processor 1402 is used to execute the above-mentioned resource scheduling method of the database, such as Figure 2 shown.

[0260] In some possible implementations, various aspects of the database resource scheduling method provided in the present application may also be implemented in the form of a program product, which includes a computer program. When the program product is run on an electronic device, the computer program is used to enable the electronic device to execute the steps of the database resource scheduling method according to various exemplary implementations of the present application described above in this specification. For example, the electronic device may execute the following steps: Figure 2 Follow the steps shown in .

[0261] The program product may use any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0262] The program product of the embodiment of the present application may adopt a portable compact disk read-only memory (CD-ROM) and include a computer program, and can be run on an electronic device. However, the program product of the present application is not limited thereto, and in this document, a readable storage medium may be any tangible medium containing or storing a program, which can be used by or in combination with a command execution system, apparatus, or device.

[0263] A readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, wherein a readable computer program is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing. A readable signal medium may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with a command execution system, apparatus, or device.

[0264] The computer program embodied on the readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0265] Computer programs for performing the operations of the present application may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" language or similar programming languages. The computer program may be executed entirely on the subject electronic device, partially on the subject electronic device, as a separate software package, partially on the subject electronic device and partially on a remote electronic device, or entirely on a remote electronic device or server. In cases involving remote electronic devices, the remote electronic device may be connected to the subject electronic device via any type of network including a local area network (LAN) or a wide area network (WAN), or may be connected to an external electronic device (e.g., via the Internet using an Internet service provider).

[0266] It should be noted that, although several units or subunits of the device are mentioned in the above detailed description, this division is merely exemplary and not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more units described above can be embodied in one unit. Conversely, the features and functions of one unit described above can be further divided into multiple units to be embodied.

[0267] In addition, although the operations of the method of the present application are described in a specific order in the drawings, this does not require or imply that the operations must be performed in this specific order, or that all the operations shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps.

[0268] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain a computer-usable computer program.

[0269] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program commands. These computer program commands can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the commands executed by the processor of the computer or other programmable data processing device generate commands for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0270] These computer program commands may also be stored in a computer readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the commands stored in the computer readable memory produce an article of manufacture comprising a command device, the command device implementing the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0271] These computer program commands can also be loaded onto a computer or other programmable data processing device so that a series of operation steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the commands executed on the computer or other programmable device provide the instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0272] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

[0273] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A method for resource scheduling of a database, characterized in that: The method comprises: When there is a newly created initial container group in the database cluster, based on the node resource information of each node included in the database cluster, respectively determining the node remaining resources of the corresponding nodes; Based on resource occupancy information of at least one historical container group associated with the initial container group, the predicted required resources of the initial container group are obtained; the initial container group and the historical container group are created by the same resource application object; Among the nodes whose node remaining resources meet the preset required resources, at least one candidate node is selected, and the following operations are performed for each candidate node: predicting the resource inclination of the latest node remaining resources on the candidate node after scheduling the initial container group to the candidate node, wherein the resource inclination represents: the distribution difference of the node remaining resources under each resource dimension; A candidate node whose resource inclination condition meets a preset resource distribution condition is selected as a target node, and the initial container group is scheduled to the target node.

2. The method according to claim 1, characterized in that The selecting at least one candidate node from each node whose remaining resources meet the preset required resources comprises: For each of the nodes, the following operations are respectively performed: based on the difference between the node remaining resources of the node and the predicted required resources of the initial container group, combined with the partition size of the node under each resource dimension, determining the evaluation value of the node remaining resources of the node; At least one node whose evaluation value meets the preset selection condition is selected as a candidate node.

3. The method according to claim 1, characterized in that The prediction of the resource tilt of the latest node remaining resources on a candidate node after scheduling the initial container group to the candidate node includes: Based on the difference between the node remaining resources of the candidate node and the predicted required resources of the initial container group, predict the latest node remaining resources on the candidate node after scheduling the initial container group to the candidate node; For each preset resource dimension, the following operations are performed: based on the latest node remaining resources, determining the remaining resource amount of the candidate node in each resource dimension after the initial container group is scheduled to the candidate node; The resource tilt situation is determined based on the difference between the remaining resource amounts.

4. The method according to claim 3, characterized in that The step of selecting a candidate node whose resource inclination condition satisfies a preset resource distribution condition as a target node includes: Among the candidate nodes, the candidate nodes whose difference between the remaining resource amounts meets the preset resource difference threshold are selected as the target nodes; or, Based on the difference between the remaining resource amounts corresponding to the candidate nodes, the bottleneck resources corresponding to the candidate nodes are determined respectively, each of the bottleneck resources being: the resource with the lowest remaining resource amount of the corresponding candidate node under the resource dimensions; and the remaining resource amounts under the bottleneck resources corresponding to the candidate nodes are compared across resource dimensions, and based on the comparison results, a target node is determined among the candidate nodes.

5. The method according to claim 1, characterized in that The prediction of the resource tilt of the latest node remaining resources on a candidate node after scheduling the initial container group to the candidate node includes: Under each resource dimension, respectively determining the resource similarity between the predicted demand resource and the node remaining resource of the one candidate node; Based on the resource similarity, predicting the resource inclination of the latest node remaining resources on a candidate node after scheduling the initial container group to the candidate node, the resource similarity being negatively correlated with the resource inclination; The step of selecting a candidate node whose resource inclination condition satisfies a preset resource distribution condition as a target node includes: A candidate node whose resource similarity satisfies a preset similarity condition is selected as the target node.

6. The method according to any one of claims 1 to 5, characterized in that: The method further comprises: Among the nodes, the nodes whose corresponding node resource information exceeds a preset threshold are regarded as nodes to be migrated out; For each node to be migrated, container group resource information corresponding to each execution container group loaded on the node to be migrated is obtained; and the execution container group whose corresponding container group resource information meets the migration condition is used as the execution container group to be migrated; the container group resource information represents the resource consumption of the corresponding execution container group; A node to be migrated into is determined from each node other than the node to be migrated out, and the execution container group to be migrated is migrated to the node to be migrated in.

7. The method according to claim 6, characterized in that The step of determining a node to be migrated into from other nodes except the node to be migrated out includes: Based on the historical changes of the node resource information of each other node in a preset historical time period, respectively predict the estimated changes of the node resource information of each other node in a preset future time period; Based on historical changes in the container group resource information of the execution container group to be migrated within a preset historical time period, predicting estimated changes in the container group resource information of the execution container group to be migrated within a preset future time period; The difference between the estimated change of the node resource information and the estimated change of the container group resource information is used as another node that meets the preset migration condition, and is used as the node to be migrated.

8. The method according to claim 7, characterized in that The step of using a difference between the estimated change of the node resource information and the estimated change of the container group resource information and another node that meets a preset migration condition as the node to be migrated includes: Based on the estimated changes in the resource information of each node, respectively evaluate the comprehensive node resource conditions corresponding to each of the other nodes in the future time period; Determine, based on the estimated change of the container group resource information, the comprehensive container group resource status of the execution container group to be migrated in the future time period; In each resource dimension, the overall similarity between the resource situation of the comprehensive container group and the resource situation of each comprehensive node is compared respectively; Another node whose overall similarity in each resource dimension is lower than a preset similarity threshold is used as the node to be migrated.

9. The method according to claim 8, characterized in that The comprehensive node resource situation includes: the comprehensive node resource allocation situation of the corresponding other nodes under the resource dimensions, and the comprehensive node resource usage situation; the comprehensive container group resource situation includes: the comprehensive container group resource allocation situation of the execution container group to be migrated under the resource dimensions, and the comprehensive container group resource usage situation; In each resource dimension, the overall similarity between the resource situation of the integrated container group and the resource situation of each integrated node is compared, including: In each resource dimension, a first similarity between the resource allocation of the integrated container group and the resource allocation of each integrated node is compared, and a second similarity between the resource usage of the integrated container group and the resource usage of each integrated node is compared; and based on the first similarity and the second similarity, an overall similarity is determined.

10. A database resource scheduling system, characterized in that: The system includes: resource detection equipment, scheduling equipment and database cluster; The resource detection device is used to determine the node remaining resources of the corresponding nodes based on the node resource information of each node included in the database cluster when there is a newly created initial container group in the database cluster; The scheduling device is used to obtain the predicted required resources of the initial container group based on the resource occupancy information of at least one historical container group associated with the initial container group; the initial container group and the historical container group are created by the same resource application object; Among the nodes whose node remaining resources meet the preset required resources, at least one candidate node is selected, and the following operations are performed for each candidate node: predicting the resource inclination of the latest node remaining resources on the candidate node after scheduling the initial container group to the candidate node, wherein the resource inclination represents: the distribution difference of the node remaining resources under each resource dimension; A candidate node whose resource inclination condition meets a preset resource distribution condition is selected as a target node, and the initial container group is scheduled to the target node.

11. A resource scheduling device for a database, characterized in that: include: A determination unit, configured to, when a newly created initial container group exists in the database cluster, determine the node remaining resources of the corresponding nodes based on the node resource information of each node included in the database cluster; A first selection unit is configured to obtain predicted required resources of the initial container group based on resource occupancy information of at least one historical container group associated with the initial container group; the initial container group and the historical container group are created by applying for the same resource application object; A prediction unit is used to select at least one candidate node from each node whose node remaining resources meet the preset required resources, and perform the following operations for each candidate node: predict the resource inclination of the latest node remaining resources on the candidate node after the initial container group is scheduled to the candidate node, wherein the resource inclination represents: the distribution difference of the node remaining resources under each resource dimension; The second selection unit is used to select a candidate node whose resource inclination condition meets the preset resource distribution condition as the target node, and schedule the initial container group to the target node.

12. A computer-readable storage medium, characterized in that: It comprises a computer program. When the computer program is run on an electronic device, the computer program is used to enable the electronic device to execute the steps of any method described in claims 1 to 9.

13. A computer program product, characterized in that The method comprises a computer program stored in a computer-readable storage medium; when a processor of an electronic device reads the computer program from the computer-readable storage medium, the processor executes the computer program, so that the electronic device executes the steps of any one of the methods described in claims 1 to 9.

Images