Cluster resource dynamic adjustment method, system and device and storage medium

Abstract

The embodiment of the invention discloses a cluster resource dynamic adjustment method, system and device and a storage medium. According to the technical scheme provided by the embodiment of the invention, first real-time load data of each working node in a current cluster and second real-time load data of each resource unit in the working nodes are acquired; calculating a load threshold value of the current cluster based on the first real-time load data, and determining a target node from each working node according to the load threshold value; and screening a target unit from each resource unit of the target node based on the second real-time load data, and performing resource scheduling on the target unit. By adopting the technical means, accurate and dynamic scheduling of the Kubernetes cluster resources can be realized, so that the situation of unbalanced resource allocation is avoided, the allocation effect of the Kubernetes cluster resources is improved, and the stability and reliability of the Kubernetes cluster are guaranteed.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of computer technologies, and in particular, to a method, system, device, and storage medium for dynamically adjusting cluster resources. Background Art

[0002] Currently, in the applications of Kubernetes (an open-source container orchestration platform) clusters, it is often necessary to use a scheduling component to schedule cluster resources. The scheduling component allocates newly created Pods (the smallest scheduling resource unit in the Kubernetes cluster, representing a process or a group of processes running on the cluster) to appropriate Node nodes (worker nodes in the Kubernetes cluster) according to predefined scheduling policies and constraint conditions, so as to efficiently manage and allocate resources in the cluster and ensure the high availability and performance of the application program. Moreover, in order to enable Pods to be more healthy and evenly distributed dynamically on the nodes of the cluster, it is necessary to use a secondary scheduling tool to regularly check the cluster status and evaluate which Pods need to be adjusted according to the configured policies. For example, by comparing the Request (business request) value of the Node node with the fixed load threshold of the node, when the Request value is greater than the load threshold, the Pods on the Node node are scheduled to other nodes. Thus, the healthy state and efficient operation of the cluster are maintained, the reasonable distribution of resources is ensured, and resource bottlenecks or imbalance problems are avoided.

[0003] However, since the statistics of the Request value cannot reflect the real load of the Node node, triggering the dynamic adjustment of Pods based on comparing the Request value with the fixed load threshold of the Node node cannot accurately allocate resources, resulting in the situation that high-load nodes cannot adjust resources or low-load nodes frequently adjust resources, thereby affecting the resource allocation effect of the Kubernetes cluster and the business processing effect of the Kubernetes cluster. Summary of the Invention

[0004] The embodiments of the present application provide a method, system, device, and storage medium for dynamically adjusting cluster resources, which can accurately and dynamically allocate Kubernetes cluster resources, improve the resource allocation effect of the Kubernetes cluster, and solve the technical problem of inaccurate resource allocation in the Kubernetes cluster.

[0005] In a first aspect, the embodiments of the present application provide a method for dynamically adjusting cluster resources, including:

[0006] Obtaining first real-time load data of each worker node in the current cluster and second real-time load data of each resource unit in the worker node;

[0007] Calculate the load threshold of the current cluster based on the first real-time load data, and determine the target nodes from each working node according to the load threshold;

[0008] Filter the target units from each resource unit of the target node based on the second real-time load data, and perform resource scheduling on the target units.

[0009] In a second aspect, an embodiment of the present application provides a cluster resource dynamic adjustment system, including:

[0010] An acquisition module, configured to acquire the first real-time load data of each working node in the current cluster and the second real-time load data of each resource unit in the working node;

[0011] A calculation module, configured to calculate the load threshold of the current cluster based on the first real-time load data, and determine the target nodes from each working node according to the load threshold;

[0012] A scheduling module, configured to filter the target units from each resource unit of the target node based on the second real-time load data, and perform resource scheduling on the target units.

[0013] In a third aspect, an embodiment of the present application provides a cluster resource dynamic adjustment device, including:

[0014] A memory and one or more processors;

[0015] The memory is configured to store one or more programs;

[0016] When the one or more programs are executed by the one or more processors, the one or more processors implement the cluster resource dynamic adjustment method as described in the first aspect.

[0017] In a fourth aspect, an embodiment of the present application provides a non-volatile computer-readable storage medium, the non-volatile computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are configured to execute the cluster resource dynamic adjustment method as described in the first aspect when executed by a computer processor.

[0018] In a fifth aspect, an embodiment of the present application provides a computer program product, the computer program product contains instructions, and when the instructions run on a computer or a processor, the computer or the processor executes the cluster resource dynamic adjustment method as described in the first aspect.

[0019] In the embodiments of the present application, the first real-time load data of each worker node in the current cluster and the second real-time load data of each resource unit in the worker node are obtained; the load threshold of the current cluster is calculated based on the first real-time load data, and the target node is determined from each worker node according to the load threshold; the target unit is screened from each resource unit of the target node based on the second real-time load data, and resource scheduling is performed on the target unit. By adopting the above technical means, by determining the real-time load data of each worker node and each resource unit in the Kubernetes cluster, and screening the target unit of the target node based on the real-time load data for resource scheduling, the accurate and dynamic scheduling of the Kubernetes cluster resources can be realized, thereby avoiding the situation of uneven resource allocation, improving the allocation effect of the Kubernetes cluster resources, and ensuring the stability and reliability of the Kubernetes cluster. Description of the Drawings

[0020] Figure 1 is a flowchart of a method for dynamically adjusting cluster resources provided by an embodiment of the present application;

[0021] Figure 2 is a flowchart for calculating the load threshold in an embodiment of the present application;

[0022] Figure 3 is a flowchart for simulating the scheduling of the target unit in an embodiment of the present application;

[0023] Figure 4 is a schematic diagram of the dynamic scheduling of the target unit in an embodiment of the present application;

[0024] Figure 5 is a schematic structural diagram of a system for dynamically adjusting cluster resources provided by an embodiment of the present application;

[0025] Figure 6 is a schematic structural diagram of a device for dynamically adjusting cluster resources provided by an embodiment of the present application. Detailed Embodiments

[0026] To make the objectives, technical solutions, and advantages of this application clearer, the following further describes specific embodiments of this application in detail with reference to the accompanying drawings. It can be understood that the specific embodiments described herein are only for explaining this application and not for limiting this application. Additionally, it should be noted that for ease of description, only parts related to this application are shown in the drawings rather than all content. Before discussing exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although the flowcharts describe operations (or steps) as sequential processes, many of the operations can be implemented in parallel, concurrently, or simultaneously. In addition, the order of the operations can be rearranged. The process can be terminated when its operations are completed, but it can also have additional steps not included in the drawings. The process can correspond to a method, function, procedure, subroutine, subprogram, and so on.

[0027] The cluster resource dynamic adjustment method provided by this application aims to determine the real-time load data of each worker node and each resource unit in the worker nodes of the Kubernetes cluster, and screen the target units of the target nodes for resource scheduling based on the real-time load data, so as to achieve precise and dynamic scheduling of the resources in the Kubernetes cluster.

[0028] The Kubernetes cluster is an open-source container orchestration platform, aiming to automate the deployment, scaling, and operation of containerized applications. Kubernetes has high availability, scalability, and flexibility, and is widely used in enterprises of various scales. Currently, it has become the implementation standard for container orchestration.

[0029] Among them, kube-scheduler is a core component in Kubernetes, responsible for allocating newly created resource units (Pods) to appropriate worker nodes (Nodes) according to predefined scheduling policies and constraint conditions. Through the scheduling of kube-scheduler, Kubernetes can efficiently manage and allocate resources in the cluster, ensuring the high availability and performance of application programs. However, the state of the Kubernetes cluster is dynamic, and factors such as uneven resource utilization, resource contention, and priority changes may cause the Kubernetes cluster scheduling policy to fail.

[0030] Therefore, a mechanism is needed to enable Pods to be more dynamically and evenly distributed across the Node nodes of the cluster, rather than being fixed on a single host after a one-time scheduling. Thus, the Kubernetes community provides the Descheduler tool. Descheduler is a tool that can reschedule Pods. It can periodically check the cluster status and evaluate which Pods need to be scheduled according to the configured policies. For example, the policies may include removing duplicate Pods, balancing node resource utilization, removing Pods that violate node taints or Pod anti-affinity rules, or removing Pods that have been running for more than a specified threshold. Each time it runs, the Descheduler tool marks the Pods that need to be scheduled according to these policies and sends a scheduling request to the API server. After the scheduled Pods are deleted, the Kubernetes scheduler will reschedule them to more suitable nodes, thereby achieving optimized resource utilization. In this way, Descheduler helps maintain the health and efficient operation of the cluster, ensures reasonable resource distribution, and avoids resource bottlenecks or imbalance problems.

[0031] However, despite the important role played by the Descheduler tool in optimizing load scheduling, the following problems still exist:

[0032] 1. The resource quantity statistics method based on the Request (business request) value cannot reflect the true load of the node. The request value of a Pod is used to declare the minimum amount of resources required for the Pod to run, but it cannot reflect the true load during the Pod's operation. Therefore, it is not very accurate for Descheduler to calculate the node load by counting the Request values of Pods.

[0033] 2. The load threshold cannot be adjusted adaptively. Currently, the Descheduler policy triggers scheduling based on a pre-configured fixed threshold. If the node load is greater than the threshold, it checks for scheduling; otherwise, it does not schedule. Therefore, the load threshold can greatly affect the effect of reverse scheduling. If the threshold is configured too large, it may cause high-load machines to not be schedulable; if the threshold is configured too small, scheduling may occur frequently, resulting in service fluctuations.

[0034] 3. The process of scheduling rebalancing lacks coordination with the scheduler. Descheduler does not implement a scheduler but relies on the Kubernetes scheduler. This also means that all Descheduler can do is schedule Pods and let them go through the scheduling process again. If there is a conflict between the scheduling decisions of Descheduler and the scheduling results of the scheduler, then repeated scheduling of Pods may occur.

[0035] To solve the above problems, a method for dynamically adjusting cluster resources according to an embodiment of the present application is provided. By obtaining the real-time load data of nodes, the load threshold of the reverse scheduler is dynamically calculated, and then the execution of Pod scheduling is determined in combination with the prediction result of simulated scheduling, thereby improving the accuracy and stability of the scheduling optimization of the reverse scheduler.

[0036] Embodiment:

[0037] Figure 1 The flowchart of a method for dynamically adjusting cluster resources according to an embodiment of the present application is given. The method for dynamically adjusting cluster resources provided in this embodiment can be executed by a cluster resource dynamic adjustment device. The cluster resource dynamic adjustment device can be implemented in a software and / or hardware manner. The cluster resource dynamic adjustment device can be composed of two or more physical entities or one physical entity. Generally, the cluster resource dynamic adjustment device can be a processing device such as a Kubernetes cluster server.

[0038] The following takes the cluster resource dynamic adjustment device as the main body for executing the cluster resource dynamic adjustment method as an example for description. Refer to Figure 1 The cluster resource dynamic adjustment method specifically includes:

[0039] S110. Obtain the first real-time load data of each working node in the current cluster and the second real-time load data of each resource unit in the working node.

[0040] In the process of scheduling Kubernetes cluster resources in the present application, by analyzing the real load of each working node Node in the cluster in real time in the container orchestration system deployed based on Kubernetes, the load threshold of the reverse scheduler is adaptively adjusted; finally, simulated scheduling is performed before scheduling, and only when the preset verification conditions are met, the Pod scheduling is actually executed, so as to avoid invalid scheduling or repeated scheduling as much as possible.

[0041] Among them, for each working node Node of the Kubernetes cluster and each resource unit Pod included in each working node Node, real-time load data is collected, which are respectively defined as the first real-time load data and the second real-time load data.

[0042] The first real-time load data can be key performance indicators such as CPU usage rate, memory usage rate, disk I / O, network bandwidth, etc. These indicators can comprehensively reflect the overall load status of the worker nodes. Kubernetes provides rich API interfaces through which the status information of each node in the cluster can be queried, including the usage of resources such as CPU and memory. Or by integrating an open-source monitoring system into the Kubernetes cluster, the load data of the nodes can be collected and displayed in real time. In addition, custom scripts can be written or third-party tools can be used to collect the load data of the nodes.

[0043] The second real-time load data is the load data of each Pod within the worker node, which can also include the consumption of resources such as CPU and memory. These data help to identify which Pods are the main contributors to the current load, so as to perform targeted resource scheduling and adjustment.

[0044] By obtaining the first real-time load data of each worker node in the current cluster and the second real-time load data of each resource unit in the worker node, a basis for precise resource scheduling and adjustment can be provided, improving the accuracy of resource scheduling.

[0045] Specifically, the data sources of the real-time load data of Pods and nodes can be selected from common metric collectors such as the metrics-server or the open-source service monitoring system Prometheus. When performing dynamic scheduling of cluster resources, first initialize the information, fully obtain the list and status of all worker nodes and Pods in the cluster, so as to initialize and build the data cache of Pods and Nodes. After initialization, the anti-scheduler continuously listens for changes in events of Pods and Nodes. When a new or deletion event of a Pod or Node occurs, the corresponding Pod or Node object is inserted into or removed from the data cache synchronously. At the same time, the anti-scheduler communicates with the metric collector regularly, and according to the list of Pods and Nodes in the current data cache, obtains the real-time load data (such as CPU, memory, etc.) of the objects, and updates the real-time load data to the data cache of the corresponding Pod and Node objects. Finally, the data cache layer of the anti-scheduler provides a query interface, which can specify or batch query the real-time CPU and memory and other real-time load data of Pods and Nodes, so as to serve as the data metrics for the subsequent scheduling decisions of the anti-scheduler.

[0046] S120. Calculate the load threshold of the current cluster based on the first real-time load data, and determine the target nodes from each worker node according to the load threshold.

[0047] Furthermore, based on the determined first real-time load data, the present application dynamically calculates the load threshold by combining the first real-time load data, so as to screen the target nodes from the worker nodes of the cluster according to the load threshold.

[0048] Among them, the calculation of the load threshold can be determined by finding the median or mean value from the first real-time load data of each node, or by calculating the weighted average value in combination with historical load data. The present application does not impose a fixed limit on the method for determining the load threshold. In addition, after determining the load threshold, it is also possible to dynamically adjust and update the load threshold according to the real-time load situation and business requirements of the cluster.

[0049] Furthermore, compare the first real-time load data of each worker node with the set load threshold. Identify the nodes whose load exceeds or is close to the threshold, thereby determining these worker nodes as target nodes.

[0050] Optionally, referring to Figure 2 , calculating the load threshold of the current cluster based on the first real-time load data includes:

[0051] S1201. Sort the worker nodes according to the first real-time load data to construct a node load list of the current cluster;

[0052] S1202. Determine the sampling quantity of the node load list, and collect the corresponding quantity of the first real-time load data from the node load list according to the sampling quantity;

[0053] S1203. Calculate the data mean value based on the collected first real-time load data, and determine the load threshold of the current cluster based on the data mean value.

[0054] The reverse scheduler needs to perform calculations based on the above first real-time load data to obtain the load thresholds of each load data type, and then screen out the target nodes according to the load thresholds for scheduling. Among them, if there is one type of load data, the worker node is determined as the target node only when the first real-time load data reaches the load threshold. If there are multiple types of load data, the worker node is determined as the target node only when the first real-time load data reaches the corresponding load thresholds of each type.

[0055] The following is described by taking the CPU utilization rate as an example. Assume that there are N Nodes in the cluster and the sampling percentage is α, then the sampling number Obtain the latest CPU utilization rate list of all nodes from the data cache as {U 1 , U 2 ,..., U N}, among which, select the m Nodes with the highest CPU utilization rate and denote them as {h 1 , h 2 ,..., h m}. Then the load threshold of the reverse scheduler is expressed as

[0056]

[0057] Optionally, in the case where there are multiple data types in the first real-time load data, calculate the data mean based on the collected first real-time load data, and determine the load threshold of the current cluster based on the data mean, including:

[0058] Calculate the data mean of each corresponding data type based on the collected first real-time load data;

[0059] Perform a weighted sum of the data means of each data type, and use the weighted sum result of the data means as the load threshold of the current cluster.

[0060] By collecting data of various load data types in real time from each worker node of the cluster, for each data type, calculate its mean value on all worker nodes. The mean value can be calculated by adding up the data of all nodes and then dividing by the number of nodes. According to business requirements and the characteristics of the cluster, determine a weight for each data type. The weight reflects the degree of influence of this data type on the overall load of the cluster. For example, for critical business applications, CPU usage rate and memory usage rate may have higher weights; while for data-intensive applications, disk I / O and network bandwidth may be more important. Multiply the data mean of each data type by its corresponding weight, and then add up the results. Use the result of this weighted sum as the load threshold of the current cluster.

[0061] Through the above steps, the data mean can be calculated based on the first real-time load data of multiple data types, and the load threshold of the current cluster can be determined based on the weighted sum result of the data mean, so as to achieve effective monitoring and management of the cluster and ensure the stability and performance of the cluster.

[0062] Optionally, determine the target nodes from each worker node according to the load threshold, including:

[0063] In the case where the first real-time load data of the current worker node is greater than the load threshold or the weighted sum result of the first real-time load data of multiple data types is greater than the load threshold, and the number of consecutive resource scheduling times of the current worker node is lower than the scheduling times threshold, determine the current worker node as the target node.

[0064] Finally, for the first real-time load data of one type, taking CPU utilization rate as an example, by traversing and checking each Node, determine whether the CPU utilization rate of the node is greater than the load threshold U evict . If the CPU utilization rate is greater than the load threshold, determine the node as the target node and add it to the pending scheduling list E.

[0065] For the first real-time load data of multiple types, calculate the weighted sum result of the current worker node through the first real-time load data, and determine whether the weighted sum result is greater than the load threshold. If so, determine the current worker node as the target node.

[0066] In addition, considering that multiple scheduling affects the stability of the cluster, if the number of consecutive resource scheduling times is higher than the set scheduling times threshold, it is considered that the node cannot be scheduled anymore to ensure the reliability of the cluster operation.

[0067] S130. Screen target units from each resource unit of the target node based on the second real-time load data, and perform resource scheduling on the target units.

[0068] Furthermore, by comparing the second real-time load data of each resource unit of the target node with the corresponding screening criteria, the resource units with higher loads are screened out as target units. Then, according to the current load situation and business requirements of the cluster, a suitable resource scheduling strategy is formulated. The scheduling strategy may include migrating the load to other resource units, increasing the capacity of resource units, reducing the load of resource units, etc. According to the formulated scheduling strategy, corresponding scheduling operations are performed on the screened target units. For example, if the target unit is a CPU resource unit, it can be considered to migrate part of the load to other CPU resource units or increase the capacity of the CPU. After the scheduling operation is executed, the load situation and resource usage of the target unit are monitored in real time. Feedback is collected and the scheduling effect is evaluated to adjust and optimize the scheduling strategy.

[0069] It should be noted that the above screening criteria for target units can be set in ways such as a set load threshold. The present application does not make a fixed limit on the specific screening criteria and will not elaborate here.

[0070] By introducing real-time load data (such as CPU and memory utilization rates), the statistical error based on the Request value is avoided, and the actual load status of the node is truly reflected. By dynamically adjusting the load threshold, the resource scheduling is made more accurate and adaptable, reducing resource overload and imbalance phenomena. During the load peak period, through real-time load analysis and dynamic water level adjustment, the resource bottleneck is effectively alleviated, and the service capacity of the cluster is improved. In addition, through deep integration with the Kubernetes scheduler, the consistency between the scheduling decision and the scheduling result is ensured, and the problem of repeated migration of Pods is avoided. The seamless connection between scheduling and the scheduling process is realized, and the efficiency and stability of the entire reverse scheduling process are improved.

[0071] Optionally, screening target units from each resource unit of the target node based on the second real-time load data includes:

[0072] Selecting the resource unit with the largest load from each resource unit of the target node as the target unit based on the second real-time load data.

[0073] Exemplarily, taking CPU utilization as an example, for each Node in the above scheduling list E, obtain the list of Pods running and schedulable in the current Node from the data cache, and query the set of their real-time CPU utilization, which is recorded as {PU 1 , PU 2 ,..., PU k} in descending order. Then, it is necessary to select the resource unit with the largest load PU 1 as the target unit.

[0074] Furthermore, referring to Figure 3 , perform resource scheduling on the target unit, including:

[0075] S1301. Calculate the predicted load data of the simulation node when the target unit is scheduled to the simulation node;

[0076] S1302. When the predicted load data is lower than the load threshold, schedule the target unit to the simulation node.

[0077] Assume that before scheduling, the CPU utilization of the current Node is recorded as U node , and the successful scheduling count is recorded as C succee = 0. Start traversing the set {PU 1 , PU 2 ,..., PU k}, i ∈ {1, 2,..., k}, and judge whether the following conditions are met: U node > U evict and C succee < the maximum schedulable number; if the conditions are met, schedule the Pod of the resource unit with the largest load PU 1 .

[0078] As Figure 4 shown, after filtering out the target unit to be scheduled, the reverse scheduler will communicate with the k8s scheduler (Kubernetes scheduler) and send a request for simulated scheduling. This simulated scheduling interface will only return the scheduling result of the target unit scheduled to the simulation node, but will not actually create and schedule the target unit. Assume that the CPU utilization of the current target unit is recorded as U pod , and the CPU utilization of the simulation node for simulated scheduling is recorded as U mock . Check whether the following condition is met after the target unit is simulated-scheduled to the simulation node: U mock + U pod < U evict . If the condition is met, it means that the simulation node will not become a high-watermark node after adding the target unit, and the actual scheduling process can be executed; if not, the scheduling cannot be executed.

[0079] After performing resource scheduling on the target unit, it also includes:

[0080] Update the first real-time load data and the continuous resource scheduling times of the current working node.

[0081] If the target unit is successfully scheduled, then let U node - = PU i , C succee + = 1; if the scheduling fails, then continue to traverse the next Pod in the Pod list and re-execute the above scheduling process.

[0082] Through simulation scheduling technology, verify whether the simulated node can meet the load requirements before actual scheduling to avoid invalid Pod scheduling.

[0083] As described above, by obtaining the first real-time load data of each working node in the current cluster and the second real-time load data of each resource unit in the working node; calculating the load threshold of the current cluster based on the first real-time load data, and determining the target node from each working node according to the load threshold; screening the target unit from each resource unit of the target node based on the second real-time load data, and performing resource scheduling on the target unit. By adopting the above technical means, by determining the real-time load data of each working node and each resource unit in the working node of the Kubernetes cluster, screening the target unit of the target node based on the real-time load data for resource scheduling, so as to achieve accurate and dynamic scheduling of the Kubernetes cluster resources, thereby avoiding the situation of unbalanced resource allocation, improving the allocation effect of the Kubernetes cluster resources, and ensuring the stability and reliability of the Kubernetes cluster.

[0084] Based on the above embodiments, Figure 5 This is a schematic structural diagram of a cluster resource dynamic adjustment system provided by the present application. Refer to Figure 5 This cluster resource dynamic adjustment system provided in this embodiment specifically includes: an acquisition module 21, a calculation module 22, and a scheduling module 23;

[0085] Among them, the acquisition module 21 is configured to acquire the first real-time load data of each working node in the current cluster and the second real-time load data of each resource unit in the working node;

[0086] The calculation module 22 is configured to calculate the load threshold of the current cluster based on the first real-time load data, and determine the target node from each working node according to the load threshold;

[0087] The scheduling module 23 is configured to screen the target unit from each resource unit of the target node based on the second real-time load data, and perform resource scheduling on the target unit.

[0088] Specifically, calculating the load threshold of the current cluster based on the first real-time load data includes:

[0089] Sorting each working node according to the first real-time load data to construct a node load list of the current cluster;

[0090] Determining the sampling quantity of the node load list, and collecting the corresponding quantity of the first real-time load data from the node load list according to the sampling quantity;

[0091] Calculating the data mean value based on the collected first real-time load data, and determining the load threshold of the current cluster based on the data mean value.

[0092] Wherein, in the case that there are multiple data types in the first real-time load data, calculating the data mean value based on the collected first real-time load data, and determining the load threshold of the current cluster based on the data mean value includes:

[0093] Calculating the data mean value of each corresponding data type based on the collected first real-time load data;

[0094] Performing weighted summation on the data mean values of each data type, and using the weighted summation result of the data mean values as the load threshold of the current cluster.

[0095] Determining the target node from each working node according to the load threshold, including:

[0096] In the case that the first real-time load data of the current working node is greater than the load threshold or the weighted summation result of the first real-time load data of multiple data types is greater than the load threshold, and the continuous resource scheduling times of the current working node are lower than the scheduling times threshold, determining the current working node as the target node.

[0097] Specifically, screening the target unit from each resource unit of the target node based on the second real-time load data includes:

[0098] Selecting the resource unit with the largest load from each resource unit of the target node as the target unit based on the second real-time load data.

[0099] Performing resource scheduling on the target unit, including:

[0100] Calculating the predicted load data of the simulation node in the case that the target unit is scheduled to the simulation node;

[0101] In the case that the predicted load data is lower than the load threshold, scheduling the target unit to the simulation node.

[0102] After performing resource scheduling on the target unit, it further includes:

[0103] Updating the first real-time load data and the continuous resource scheduling times of the current working node.

[0104] As described above, by obtaining the first real-time load data of each worker node in the current cluster and the second real-time load data of each resource unit in the worker node; calculating the load threshold of the current cluster based on the first real-time load data, and determining the target node from each worker node according to the load threshold; screening the target unit from each resource unit of the target node based on the second real-time load data, and performing resource scheduling on the target unit. By adopting the above technical means, by determining the real-time load data of each worker node in the Kubernetes cluster and each resource unit in the worker node, screening the target unit of the target node based on the real-time load data for resource scheduling, so as to realize the accurate and dynamic scheduling of the Kubernetes cluster resources, thereby avoiding the situation of unbalanced resource allocation, improving the allocation effect of the Kubernetes cluster resources, and ensuring the stability and reliability of the Kubernetes cluster.

[0105] The cluster resource dynamic adjustment system provided by the embodiments of the present application can be configured to execute the cluster resource dynamic adjustment method provided by the above embodiments, and has corresponding functions and beneficial effects.

[0106] On the basis of the above actual example, the embodiments of the present application also provide a cluster resource dynamic adjustment device. Referring to Figure 6 , the cluster resource dynamic adjustment device includes: a processor 31, a memory 32, a communication module 33, an input device 34, and an output device 35. The memory, as a computer-readable storage medium, can be configured to store software programs, computer-executable programs, and modules, such as the program instructions / modules corresponding to the cluster resource dynamic adjustment method described in any embodiment of the present application (for example, the acquisition module, calculation module, and scheduling module in the cluster resource dynamic adjustment system). The communication module is configured to perform data transmission. The processor executes various functional applications and data processing of the device by running the software programs, instructions, and modules stored in the memory, that is, to implement the above cluster resource dynamic adjustment method. The input device can be configured to receive input digital or character information, and generate key signal inputs related to the user settings and function control of the device. The output device may include a display device such as a display screen. The above provided cluster resource dynamic adjustment device can be configured to execute the cluster resource dynamic adjustment method provided by the above embodiments, and has corresponding functions and beneficial effects.

[0107] Based on the above embodiments, an embodiment of the present application further provides a non-volatile computer-readable storage medium. The non-volatile computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are configured to execute a method for dynamically adjusting cluster resources when executed by a computer processor. The storage medium can be any various types of memory devices or storage devices. Of course, for the non-volatile computer-readable storage medium provided by an embodiment of the present application, its computer-executable instructions are not limited to the method for dynamically adjusting cluster resources as described above, and can also execute related operations in the method for dynamically adjusting cluster resources provided by any embodiment of the present application.

[0108] Based on the above embodiments, an embodiment of the present application further provides a computer program product. Essentially, or the part that contributes to the prior art, or all or part of the technical solution of the present application can be embodied in the form of a software product. The computer program product is stored in a storage medium and includes several instructions for causing a computer device, a mobile terminal, or a processor therein to execute all or part of the steps of the method for dynamically adjusting cluster resources described in various embodiments of the present application.

Claims

1. A method for dynamically adjusting cluster resources, characterized in that: include: Acquire first real-time load data of each working node in the current cluster and second real-time load data of each resource unit in the working node; Calculating a load threshold of the current cluster based on the first real-time load data, and determining a target node from each of the working nodes according to the load threshold; A target unit is selected from each of the resource units of the target node based on the second real-time load data, and resource scheduling is performed on the target unit.

2. The cluster resource dynamic adjustment method according to claim 1, characterized in that: The calculating the load threshold of the current cluster based on the first real-time load data includes: Sorting each of the working nodes according to the first real-time load data to build a node load list of the current cluster; Determine a sampling quantity of the node load list, and collect a corresponding quantity of the first real-time load data from the node load list according to the sampling quantity; A data mean is calculated based on the collected first real-time load data, and a load threshold of the current cluster is determined based on the data mean.

3. The cluster resource dynamic adjustment method according to claim 2, characterized in that: In a case where the first real-time load data has multiple data types, calculating a data mean based on the collected first real-time load data, and determining a load threshold of the current cluster based on the data mean includes: Calculate the data mean of each corresponding data type based on the first real-time load data collected; A weighted sum is performed on the data means of each data type, and the weighted sum result of the data means is used as the load threshold of the current cluster.

4. The method for dynamically adjusting cluster resources according to any one of claims 1 to 3, characterized in that: The determining the target node from each of the working nodes according to the load threshold comprises: When the first real-time load data of the current working node is greater than the load threshold or the weighted sum of the first real-time load data of multiple data types is greater than the load threshold, and the number of continuous resource scheduling of the current working node is lower than the scheduling number threshold, the current working node is determined to be the target node.

5. The cluster resource dynamic adjustment method according to claim 4, characterized in that: After the resource scheduling is performed on the target unit, the method further includes: The first real-time load data and the number of continuous resource scheduling of the current working node are updated.

6. The method for dynamically adjusting cluster resources according to any one of claims 1 to 3, characterized in that: The selecting a target unit from each of the resource units of the target node based on the second real-time load data includes: Based on the second real-time load data, a resource unit with the largest load is selected from each of the resource units of the target node as the target unit.

7. The method for dynamically adjusting cluster resources according to any one of claims 1 to 3, characterized in that: The performing resource scheduling on the target unit includes: Calculating predicted load data of the simulation node when the target unit is dispatched to the simulation node; In a case where the predicted load data is lower than the load threshold, the target unit is scheduled to the simulation node.

8. A cluster resource dynamic adjustment system, characterized in that: include: An acquisition module configured to acquire first real-time load data of each working node in the current cluster and second real-time load data of each resource unit in the working node; A calculation module, configured to calculate a load threshold of the current cluster based on the first real-time load data, and determine a target node from each of the working nodes according to the load threshold; The scheduling module is configured to select a target unit from each of the resource units of the target node based on the second real-time load data, and perform resource scheduling on the target unit.

9. A cluster resource dynamic adjustment device, characterized in that: include: memory and one or more processors; The memory is configured to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the cluster resource dynamic adjustment method as described in any one of claims 1-7.

10. A non-volatile computer-readable storage medium, characterized in that: The non-volatile computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a computer processor, the computer-executable instructions are configured to execute the cluster resource dynamic adjustment method according to any one of claims 1 to 7.

11. A computer program product, characterized in that The computer program product includes instructions, and when the instructions are executed on a computer or a processor, the computer or the processor executes the cluster resource dynamic adjustment method according to any one of claims 1 to 7.

Images