Cluster parameter adjustment method and device, storage medium and electronic equipment
By counting and predicting the memory and processor data of the target pod, determining the recommended value and adjusting the cluster parameters, the problems of low scaling efficiency and poor stability in the existing technology are solved, and more efficient and stable scaling is achieved.
Patent Information
- Application Number
- CN202411009198.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-25
- Publication Date
- 2024-11-22
AI Technical Summary
In the prior art, the expansion and shrinkage efficiency is low and the stability is poor, making it difficult to respond to emergencies quickly, resulting in system instability.
By counting the memory and processor data of the target pod in the historical time period, predicting the memory and processor requirements at the target moment, determining the recommended value of the memory and processor limits, and adjusting the cluster parameters to optimize the scaling capacity.
It improves the scaling efficiency and system stability, reduces the need to adjust the target cluster parameters multiple times, and adapts to the current load conditions.
Smart Images

Figure CN119003153A_ABST
Abstract
Description
Technical Field
[0001] The embodiments of the present application relate to the computer field, and in particular, to a cluster parameter adjustment method, device, storage medium and electronic device. Background Art
[0002] In related technologies, there are two ways to implement pod-level scaling: the manual mode cannot quickly handle situations where the volume of business requests is huge; the automated scaling is based on threshold operations and is triggered by real-time data. Frequent scaling in emergencies makes it difficult to ensure system stability.
[0003] It can be seen from this that the relevant technology has problems of low expansion and contraction efficiency and poor stability.
[0004] With respect to the above-mentioned problems existing in the related technologies, no effective solution has been proposed yet. Summary of the invention
[0005] The embodiments of the present application provide a cluster parameter adjustment method, device, storage medium and electronic device to at least solve the problems of low expansion and contraction efficiency and poor stability existing in the related art.
[0006] According to one embodiment of the present application, a cluster parameter adjustment method is provided, comprising: determining historical memory statistics data and historical processor statistics data of a target pod included in a target cluster within a historical predetermined time period under a workload state; predicting predicted memory data and predicted processor data of the target pod included in the target cluster at a target moment under the workload state; determining a recommended memory limit value of the target pod based on the historical memory statistics data and the predicted memory data, and determining a recommended processor limit value of the target pod based on the historical processor statistics data and the predicted processor data; and adjusting target parameters of the target cluster based on the recommended memory limit value and the recommended processor limit value.
[0007] According to another embodiment of the present application, a cluster parameter adjustment device is provided, including: a first determination module, used to determine historical memory statistics data and historical processor statistics data of a target pod included in the target cluster within a historical predetermined time period under a workload state; a prediction module, used to predict the predicted memory data and predicted processor data of the target pod included in the target cluster at a target moment under the workload state; a second determination module, used to determine a recommended memory limit value of the target pod based on the historical memory statistics data and the predicted memory data, and to determine a recommended processor limit value of the target pod based on the historical processor statistics data and the predicted processor data; a third determination module, used to adjust the target parameters of the target cluster based on the recommended memory limit value and the recommended processor limit value.
[0008] According to another embodiment of the present application, a computer-readable storage medium is provided, in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above method embodiments when run.
[0009] According to another embodiment of the present application, an electronic device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.
[0010] According to another embodiment of the present application, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the steps in any of the above method embodiments are implemented.
[0011] Through this application, the historical memory data and historical processor data of the target pod in the target cluster under the workload in the historical time period are first statistically confirmed, and the memory data and processor data of the target pod at the target time are predicted. The recommended value of the memory limit of the target pod is determined based on the historical memory statistical data and the predicted memory data, and the recommended value of the processor limit of the target pod is determined based on the historical processor statistical data and the predicted processor data. Finally, the recommended value of the memory limit and the recommended value of the processor limit are used to adjust the target parameters in their respective target clusters. Since the recommended value of the memory limit can be determined based on the historical memory statistical data and the predicted memory data, and the recommended value of the processing limit can be determined based on the historical processor statistical data and the predicted processor data, the determined recommended value of the memory limit and the recommended value of the processor limit can be made more suitable for the current target cluster. Therefore, there is no need to adjust the target parameters of the target cluster multiple times. Therefore, the problems of low efficiency and poor stability of expansion and contraction in the related technology can be solved, and the effect of improving the efficiency and stability of expansion and contraction can be achieved. BRIEF DESCRIPTION OF THE DRAWINGS
[0012] Figure 1 The hardware structure frame of the server device according to the cluster parameter adjustment method of the embodiment of the present application Figure 2
[0013] Figure 2 The process of the cluster parameter adjustment method according to the embodiment of the present application is as follows Figure 2
[0014] Figure 3 The following is a flow chart of a cluster parameter adjustment method according to a specific embodiment of the present invention. Figure 2
[0015] Figure 4 It is a structural block diagram of a cluster parameter adjustment device according to an embodiment of the present application. DETAILED DESCRIPTION
[0016] The embodiments of the present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.
[0017] It should be noted that the terms "first", "second", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.
[0018] The method embodiments provided in the embodiments of the present application can be executed in a server device or a similar computing device. Taking running on a server device as an example, Figure 1 FIG. 1 is a hardware structure diagram of a server device of a cluster parameter adjustment method according to an embodiment of the present application. Figure 1 As shown, the server device may include one or more ( Figure 1 Only one is shown in the figure) processor 102 (processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data, wherein the server device may also include a transmission device 106 and an input / output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above server device. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations are shown.
[0019] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the cluster parameter adjustment method in the embodiment of the present application. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, to implement the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the server device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.
[0020] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the server device. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.
[0021] In this embodiment, a cluster parameter adjustment method is provided. Figure 2 is a flow chart of a cluster parameter adjustment method according to an embodiment of the present application, such as Figure 2 As shown, the process includes the following steps:
[0022] Step S202, determining historical memory statistics and historical processor statistics of a target pod included in the target cluster in a historical predetermined time period under a workload state;
[0023] Step S204, predicting the predicted memory data and predicted processor data of the target pod included in the target cluster at the target time under the workload state;
[0024] Step S206, determining a recommended memory limit value of the target pod based on the historical memory statistical data and the predicted memory data, and determining a recommended processor limit value of the target pod based on the historical processor statistical data and the predicted processor data;
[0025] Step S208: adjusting the target parameters of the target cluster based on the recommended memory limit value and the recommended processor limit value.
[0026] In the above embodiment, the target cluster can be a K8s cluster, an MQ cluster, or a distributed storage cluster SAN, NAS, or a distributed computing cluster HPC, but is not limited thereto. The target pod is the smallest resource management component in the K8s cluster, or it can be a resource object that minimizes the running of containerized applications. A pod represents an execution task running in the cluster, and most other components in the K8s cluster support and extend functions around the pod.
[0027] In the above embodiment, a pod may be a container or may include multiple containers at the same time. The workload state may be the state of the target cluster executing tasks in a pod or the state of the target cluster executing tasks in multiple pods at the same time. In this embodiment, the historical scheduled time may be the working time of the historical memory or the historical processor in the previous two weeks, or the time in the historical three weeks, or the data in the historical three days, but not limited thereto. The historical memory statistics may include the maximum value, minimum value, average value, median, etc. of the historical memory usage, and the historical processor statistics may include the maximum value, minimum value, average value, median, etc. of the historical processor usage, such as the cup. The predicted memory data may be the data obtained by predicting the memory usage of the memory at the target time. The predicted processor data may be the data obtained by predicting the processor usage of the processor at the target time. Among them, the target time may be the time after the current time. Among them, the processor usage is the number of processor cores used when the container is running, and the memory usage is the memory information used when the container is running.
[0028] In the above embodiment, the recommended value of memory or processor limit may be the maximum value of the recommended container's memory or container's processor usage during actual use, may be the maximum value or minimum value of the container resource usage statistics obtained by predicting and counting the monitoring historical data, or may be an average value or predicted value, etc., and is not limited thereto. For example, determining the recommended value of memory limit of the target pod based on the historical memory statistical data and the predicted memory data may include: determining the maximum value of the maximum memory limit included in the historical memory statistical data and the maximum memory usage included in the predicted memory data as the recommended value of memory limit, and determining the average value of the maximum memory limit included in the historical memory statistical data and the maximum memory usage included in the predicted memory data as the recommended value of memory limit. Similarly, determining the recommended value of memory limit of the target pod based on the historical memory statistical data and the predicted memory data may include: determining the maximum value of the maximum processor limit included in the historical processor statistical data and the maximum processor usage included in the predicted processor data as the recommended value of processor limit, and determining the average value of the maximum processor limit included in the historical processor statistical data and the maximum processor usage included in the predicted processor data as the recommended value of processor limit. When the target pod in the target cluster executes the task, the target parameters may include the number of target pods, the memory limit in the container of each pod, the processor limit, etc.
[0029] Through this application, the historical memory data and historical processor data of the target pod in the target cluster under the workload in the historical time period are first statistically confirmed, and the memory data and processor data of the target pod at the target time are predicted. The recommended value of the memory limit of the target pod is determined based on the historical memory statistical data and the predicted memory data, and the recommended value of the processor limit of the target pod is determined based on the historical processor statistical data and the predicted processor data. Finally, the recommended value of the memory limit and the recommended value of the processor limit are used to adjust the target parameters in their respective target clusters. Since the recommended value of the memory limit can be determined based on the historical memory statistical data and the predicted memory data, and the recommended value of the processing limit can be determined based on the historical processor statistical data and the predicted processor data, the determined recommended value of the memory limit and the recommended value of the processor limit can be made more suitable for the current target cluster. Therefore, there is no need to adjust the target parameters of the target cluster multiple times. Therefore, the problems of low efficiency and poor stability of expansion and contraction in the related technology can be solved, and the effect of improving the efficiency and stability of expansion and contraction can be achieved.
[0030] Among them, the execution entity of the above steps can be a cloud platform K8s cluster, etc., but is not limited to this.
[0031] In an exemplary embodiment, adjusting the target parameters of the target cluster based on the recommended memory limit value and the recommended processor limit value includes: determining an expansion method allowed by the target cluster; and determining to adjust the target parameters of the target cluster based on the expansion method, the recommended memory limit value, and the recommended processor limit value.
[0032] In the above embodiment, the expansion method can be a vertical expansion method or a horizontal expansion method. The vertical expansion method can be that the system of the target cluster only allows changes to the processor and memory limits of the pod, and the horizontal expansion method can be that the number of replicas used to execute tasks is increased or decreased, where the replica is the number of replicas that execute tasks.
[0033] In the above embodiment, the specific scaling method suitable for the target cluster is determined by the recommended values of memory limit and processor limit, and the target parameters of the target cluster are adjusted according to different scaling methods. By scaling the container using different scaling methods according to the actual situation of container operation, specific problems can be addressed specifically, maximizing resource utilization, and maintaining system stability while ensuring full utilization of resources.
[0034] In an exemplary embodiment, the determining of adjusting the capacity of the target cluster based on the expansion method, the recommended memory limit value, and the recommended processor limit value includes: determining a first size relationship between the recommended memory limit value and the initial memory limit value when the expansion method indicates that the target cluster allows changes to the memory limit value and the processor limit value of the target pod; adjusting the memory limit value of the target pod to the recommended memory limit value when the first size relationship indicates that the recommended memory limit value is greater than the initial memory limit value; and adjusting the memory limit value of the target pod to the recommended memory limit value when the first size relationship indicates that the recommended memory limit value is less than a first product of the initial memory limit value and a first ratio. The amount is adjusted to the first product, wherein the first ratio is a value less than 1; determining a second size relationship between the recommended processor limit value and the initial processor limit value; when the second size relationship indicates that the recommended processor limit value is greater than the initial processor limit value, adjusting the processor limit value of the target pod to the recommended processor limit value; when the second size relationship indicates that the recommended processor limit value is less than a second product of the initial processor limit value and a second ratio, adjusting the processor limit value of the target pod to the second product, wherein the second ratio is a value less than 1; wherein the target parameter includes at least one of the following: the memory limit value of the target pod, the processor limit value of the target pod.
[0035] In the above embodiment, the first size relationship is a comparison relationship between the recommended memory limit value and the initial memory limit value. When the first size relationship indicates that the recommended memory limit value is greater than the initial memory limit value, the memory limit value can be adjusted to the recommended memory limit value; when the recommended memory limit value is less than a certain ratio of the original memory limit value, the first product of the recommended memory value and the first ratio is used as the adjustment value; when the recommended memory limit value is between the memory limit value and the original memory limit value at a certain ratio, no adjustment is made. The first ratio can be 80%, 70%, 85%, etc., but is not limited thereto.
[0036] In the above embodiment, the second size relationship may be a comparison relationship between the processor limit value recommendation and the processor limit value. When the second size relationship indicates that the processor recommended value is greater than the initial processor limit value, it can be determined that the initial processor limit value does not meet the business requirement value, and the processor limit value is adjusted to the processor limit value recommendation value; when the processor limit value recommendation value is less than a certain ratio of the initial processor limit value, the second product of the processor limit value recommendation value and the second ratio is used as the adjustment value; when the processor limit value recommendation value is between the processor limit value and the initial processor limit value at a certain ratio, the value is not adjusted. Among them, the second ratio can be 80%, 70%, 85%, etc., but is not limited thereto.
[0037] In the above embodiment, the recommended value of the memory limit and the recommended value of the processor limit can be expressed as: l_r , the initial memory limit and the initial processor limit can be expressed as y l_y , the method of adjusting the target parameters can be found in Table 1. As shown in Table 1, when y l_r >y l_y , it can be determined that the original limit does not meet the business requirements and needs to be changed. The limit is changed to the recommended value y l_r When y l_r <0.8*y l_y When , the recommended value y can be determined l_r Less than 0.8*y l_y , needs to be changed, the limit is changed to 0.8*y l_y , if 0.8*y l_y <y l_r <y l_y , it is determined that the required limit is within the controllable range, in which case no change will occur. This strategy can ensure that sufficient resources can be provided at one time when the recommended value is greater than the original limit. This method ensures that sufficient resources can be provided at one time when the recommended value is greater than the initial limit, avoiding the impact of one-time supply on the system's processing efficiency, while also ensuring the stability of the system.
[0038] It should be noted that the initial memory limit may be the current memory limit, and the initial processor limit may be the current processor limit.
[0039] Table 1
[0040] Comparison of recommended values and original limits Change Changing Values <![CDATA[y i_r >and i_y ]]> yes <![CDATA[y i_r ]]> <![CDATA[y i_r <0.8*y i_y ]]> yes <![CDATA[0.8*y i_y ]]> <![CDATA[0.8*y i_y and i_y< and i_y ]]> no ---
[0041] It should be noted that the target parameters may include the memory limit of the target pod, such as the memory limit of the container included in the target pod. The target parameters may also include the processor limit of the target pod, such as the processor limit of the container included in the target pod.
[0042] In the above embodiment, historical processor statistics data, historical memory statistics data, predicted processor data, and predicted memory data are obtained by predicting and counting the container processor / memory resource usage, and the recommended memory limit value and the recommended processor limit value are determined based on the historical processor statistics data, historical memory statistics data, predicted processor data, and predicted memory data. Finally, the recommended value is compared with the original value, and the target parameter is changed according to the situation to ensure the stability of the business to the greatest extent. The utilization rate of resources by workload is improved, and the work efficiency of workload is improved.
[0043] In an exemplary embodiment, the method of determining to adjust the capacity of the target cluster based on the expansion method, the recommended memory limit value, and the recommended processor limit value includes: when the expansion method indicates that the target cluster allows an increase in the number of replicas, determining the expected number of replicas based on the recommended memory limit value and the recommended processor limit value, wherein the number of replicas is the number of pods used to execute the workload; determining the initial number of replicas of the pods that execute the workload included in the target cluster; when the expected number of replicas is greater than the initial number of replicas, adjusting the number of pods used to execute the workload to the expected number of replicas; when the expected number of replicas is less than the initial number of replicas, adjusting the number of pods used to execute the workload to the difference between the initial number of replicas and a target constant; wherein the target parameters include the number of pods used to execute the workload.
[0044] In the above embodiment, when the target cluster is in a workload state, it can be in a single-copy running state, that is, one pod executes the task, or it can be in a multi-copy running state, that is, multiple pods execute the same task. The initial number of copies can be the number of pods working in the target cluster in the initial working state, and the expected number of copies can be the maximum number of executing pods that the target cluster can have under the condition of ensuring system stability. The relationship between the expected number of copies and the original number of copies can be shown in Table 2, where d is the expected number of copies and k is the original number of copies. When the expected number of copies is greater than the original number of copies, it indicates that the original number of copies does not meet the business requirement value, and the pod executing the work is adjusted to the expected number of copies; when the expected number of copies is less than the original number of copies, it indicates that the system is overloaded, and the number of copies required should be reduced. The method of shrinking is based on the setting of the step size, which can be set to 1, 2, 3, etc., but is not limited to this. Among them, the step size is equivalent to the above-mentioned target constant. By setting the step size, each time the number of copies changes, only the length unit of the step size is reduced until the requirements are met; when the expected number of copies is equal to the original number of copies, it indicates that the number of container pods in the system is positively saturated and does not need to be changed. When the business needs to scale down, a gradual scaling method is adopted. The target constant is set according to the actual working conditions in the system and then reduced until it reaches the recommended resource range to ensure stable operation of the system. When the business needs to expand, the number of replicas is directly adjusted to the expected number of replicas, which improves the working efficiency of the target container.
[0045] Table 2
[0046] Comparison of expected number of copies with original number of copies Change Number of copies after change d>k yes d d<k yes k-1 d=k No ---
[0047] In an exemplary embodiment, determining the expected number of replicas based on the recommended memory limit value and the recommended processor limit value includes: determining a first ratio of the recommended memory limit value to the initial memory limit value; determining a third product of the first ratio and the initial number of replicas; rounding up the third product to obtain a first value; determining a second ratio of the recommended processor limit value to the processor limit value; determining a fourth product of the second ratio and the initial number of replicas; rounding up the fourth product to obtain a second value; and determining the maximum value included in the first value and the second value as the expected number of replicas.
[0048] In the above embodiment, the expected number of memory copies can be calculated by using the recommended memory value, the original memory limit and the original number of memory copies. The formula is as follows: d = ceil [y i_r / y i_y *k], where d represents the expected number of copies, k represents the original number of copies, and y i_r Indicates the recommended value, y i_yThe ratio of the recommended memory value to the original memory limit value is the first ratio, the product of the first ratio and the original number of memory copies represents the third product, and the first numerical value represents the third product rounded up.
[0049] In this embodiment, the above formula can still be referred to for calculating the expected number of copies of the processor by using the processor recommended value, the original limit of the processor and the original number of copies of the processor. The ratio of the processor recommended value to the processor original limit value is the above second ratio. The product of the second ratio and the original number of copies of the processor represents the fourth product, and the second numerical value represents the fourth product rounded up. The maximum value of the first numerical value of the container memory and the second numerical value of the container processor is determined as the expected number of copies. By calculating the expected number of copies by using the recommended value, the original limit and the original number of copies, the resource occupancy of the container during runtime in the workload can be fully considered, and the change method can be adjusted according to the situation, which can maximize the stability of the business, while improving the use of resources by the workload and improving the work efficiency of the workload.
[0050] In an exemplary embodiment, determining the recommended memory limit value of the target pod based on the historical memory statistics data and the predicted memory data includes: determining the historical maximum memory usage, the historical minimum memory usage, and the historical average memory usage included in the historical memory statistics data, and determining the weighted sum of the historical maximum memory usage, the historical minimum memory usage, the historical average memory usage, and the predicted memory usage included in the predicted memory data as the recommended memory limit value; determining the recommended processor limit value of the target pod based on the historical processor statistics data and the predicted processor data includes: determining the historical maximum processor usage, the historical minimum processor usage, and the historical average processor usage included in the historical processor statistics data, and determining the weighted sum of the historical maximum processor usage, the historical minimum processor usage, the historical average processor usage, and the predicted processor usage included in the predicted processor data as the recommended processor limit value.
[0051] In the above embodiment, the Holt algorithm or the machine learning algorithm can be used to predict the historical memory statistical data, but it is not limited thereto. The Holt algorithm predicts based on the overall situation and consists of three parts: first-order difference, exponential smoothing, and prediction. The first-order difference is used to calculate the trend of the data; exponential smoothing is used to fit the data to reduce noise, and the prediction is to perform prediction analysis based on the fitted data and trend.
[0052] In the above embodiment, due to the complexity of the workload, there are states where a single copy and multiple copies are executed together, that is, there are multiple pods under one workload, which can be divided into two cases for separate prediction. For the single copy operation, there is only one monitoring curve for the processor or memory of a container under a single pod under the workload. This usage monitoring curve is defined as X = {x1, x2, ...x n}, which is represented by the memory or processor usage curve within the historical scheduled time, with a length of n. The algorithm is used to predict the usage change within the historical scheduled time as Y = {y1, y2, ...y n}, the maximum value of r can be used as historical data to predict the final value Y pre , that is, Y pre = max(Y). In this embodiment, for the multi-copy operation, since there are multiple pods in the workload, each pod is independent but has the same business, it can be considered that the containers under the pod have the same dependence on processor and memory resources. The processor or memory usage monitoring historical data curve of the container under a single pod can be defined as X = {x1, x2, ...x n}, multiple monitoring historical data curves {X1, X2, ...X k}, where k is the number of pods in a workload. We predict {Y1, Y2, ...Y k}, the maximum value of historical data prediction constitutes the sequence y max ={max(y1), max(y2), max(y k )}, the final historical data predicted maximum value is Y pre =max(y max ).
[0053] In the above embodiment, the recommended value of the container memory or container processor limit is obtained by statistically analyzing and predicting the resource conditions during the actual container usage, and the calculation formula is as follows: i_r =k1y min +k2y max +k3y cng +k4y pre , where y i_r is the recommended value of resource limit, k1, k2, k3, k4 are smoothing parameters. The change of smoothing parameters can adjust the proportional relationship between the predicted value and the statistical value, and then change the recommended value of resource limit.
[0054] By differentiating the container pod data in the workload, the development trend of historical data can be described more accurately, thereby increasing the accuracy of data prediction.
[0055] In an exemplary embodiment, determining the historical memory statistics and historical processor statistics of the target pod included in the target cluster under the workload state within a historical predetermined time period includes: determining the historical maximum memory usage, historical minimum memory usage, historical average memory usage, historical maximum processor usage, historical minimum processor usage, and historical average processor usage of the target pod within the historical predetermined time period; determining the historical maximum memory usage, the historical minimum memory usage, and the historical average memory usage as the historical memory statistics; determining the historical maximum processor usage, the historical minimum processor usage, and the historical average processor usage as the historical processor statistics.
[0056] In this embodiment, historical memory statistics and historical processor data are counted, and the historical data may include: historical maximum memory usage, historical minimum memory usage, historical average memory usage, historical maximum processor usage, historical minimum processor usage, and historical average processor usage. As with the historical data prediction, the statistical results also distinguish between single-copy and multi-copy situations. For the single-copy operation, according to the monitoring historical data curve X = {x1, x2, ...x n}Determine the maximum value y pre =max(X), minimum value y min =min(X), mean value y avg =avg(X). For the operation of multiple copies, according to the monitoring historical data curve {X1, X2, ...X k}Determine the maximum value y among all data points pre =max({X1, X2, ...X k}), minimum value y min =min({X1, X2, ...X k}), average value y avg =avg({X1, X2, ...X k}). Consistent with the predicted data, by distinguishing the workload status of the pods in the system and performing specific processing according to different situations, the statistical data can be made more accurate, laying the foundation for subsequent operations.
[0057] In an exemplary embodiment, determining the expansion method allowed by the target cluster may include: determining the target task type of the task executed by the pod included in the target cluster, obtaining the corresponding relationship between the task type and the expansion method, and determining the expansion method corresponding to the target task type from the corresponding relationship. Determining the expansion method allowed by the target cluster may also include: determining the first number of all pods included in the target cluster and the second number of pods used to execute the target task, determining the memory usage and CPU usage of the containers included in the pods used to execute the target task, determining a first ratio of the second number to the first number, determining a second ratio of the memory usage to the memory limit, determining a third ratio of the CPU usage to the CPU limit, and determining the expansion method as increasing the number of replicas when the first ratio is less than the first threshold, the second ratio is greater than the second threshold, and / or the third ratio is greater than the third threshold, and determining the expansion method as changing the memory limit and processor limit of the target pod when the first ratio is greater than the first threshold, the second ratio is greater than the second threshold, and / or the third ratio is greater than the third threshold. In this embodiment, the first threshold, the second threshold, and the third threshold may be 50%, 60%, 70%, etc., and the present invention is not limited to this.
[0058] In the above embodiments, in addition to adopting the expansion method specified by the user, the expansion method can also be determined according to the task type, and the expansion method can also be determined according to the status of the pod and the status of the container included in the target cluster. Determining the expansion method according to the task type and determining the expansion method according to the status of the pod and the status of the container included in the target cluster can be a determined expansion method that is more suitable for the target cluster, ensuring that the target cluster can be expanded efficiently.
[0059] It should be noted that capacity expansion includes capacity expansion and contraction.
[0060] The following describes the cluster parameter adjustment method in conjunction with a specific implementation plan:
[0061] Figure 3 is a flow chart of a cluster parameter adjustment method according to a specific embodiment of the present invention. Figure 3 As shown, the process includes:
[0062] Step S302, obtaining container CPU and memory monitoring data.
[0063] Step S304, monitoring historical data prediction.
[0064] Step S306, monitoring historical data statistics.
[0065] Step S308: Calculate recommended values of container CPU and memory limits.
[0066] Step S310, determining the CPU and memory limits of the container.
[0067] Step S312, determine whether an accident occurs, if yes, execute step S314, if no, execute step S316.
[0068] Step S314, sending a change request.
[0069] Step S316, end.
[0070] In the above-mentioned embodiment, by analyzing the CPU / memory resource occupancy of the container in the workload, the CPU / memory resource usage of the container is predicted and counted, and the predicted value and the statistical value are used to recommend the CPU / memory limit. Finally, according to the recommended value and the original value, the change strategy is described according to the situation, and the horizontal expansion and vertical expansion strategies are adapted to ensure the stability of the business to the greatest extent. This method can improve the utilization rate of resources by the workload and improve the work efficiency of the workload.
[0071] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in each embodiment of the present application.
[0072] In this embodiment, a cluster parameter adjustment device is also provided, which is used to implement the above embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware of a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.
[0073] Figure 4 is a structural block diagram of a cluster parameter adjustment device according to an embodiment of the present application, such as Figure 4 As shown, the device comprises:
[0074] A first determination module 42 is used to determine historical memory statistics and historical processor statistics of a target pod included in the target cluster in a historical predetermined time period under a workload state;
[0075] A prediction module 44, configured to predict predicted memory data and predicted processor data of the target pod included in the target cluster at a target time under a workload state;
[0076] A second determination module 46, configured to determine a recommended value of a memory limit of the target pod based on the historical memory statistical data and the predicted memory data, and to determine a recommended value of a processor limit of the target pod based on the historical processor statistical data and the predicted processor data;
[0077] The adjustment module 48 is configured to adjust the target parameters of the target cluster based on the recommended memory limit value and the recommended processor limit value.
[0078] In an exemplary embodiment, the adjustment module 48 can adjust the target parameters of the target cluster based on the recommended memory limit value and the recommended processor limit value in the following manner: determine the expansion method allowed by the target cluster; and determine to adjust the target parameters of the target cluster based on the expansion method, the recommended memory limit value, and the recommended processor limit value.
[0079] In an exemplary embodiment, the adjustment module 48 can also implement the determination of adjusting the capacity of the target cluster based on the expansion method, the recommended memory limit value, and the recommended processor limit value in the following manner: when the expansion method indicates that the target cluster allows the memory limit and the processor limit of the target pod to be changed, determine a first size relationship between the recommended memory limit value and the initial memory limit; when the first size relationship indicates that the recommended memory limit value is greater than the initial memory limit, adjust the memory limit of the target pod to the recommended memory limit value; when the first size relationship indicates that the recommended memory limit value is less than a first product of the initial memory limit and a first ratio, adjust the target pod to the recommended memory limit value. The memory limit of d is adjusted to the first product, wherein the first ratio is a value less than 1; a second size relationship between the recommended processor limit value and the initial processor limit value is determined; when the second size relationship indicates that the recommended processor limit value is greater than the initial processor limit value, the processor limit of the target pod is adjusted to the recommended processor limit value; when the second size relationship indicates that the recommended processor limit value is less than a second product of the initial processor limit value and a second ratio, the processor limit of the target pod is adjusted to the second product, wherein the second ratio is a value less than 1; wherein the target parameter includes at least one of the following: the memory limit of the target pod, the processor limit of the target pod.
[0080] In an exemplary embodiment, the adjustment module 48 can also implement the determination of adjusting the capacity of the target cluster based on the expansion method, the recommended memory limit value, and the recommended processor limit value in the following manner: when the expansion method indicates that the target cluster allows an increase in the number of replicas, determine the expected number of replicas based on the recommended memory limit value and the recommended processor limit value, wherein the number of replicas is the number of pods used to execute the workload; determine the initial number of replicas of the pods that execute the workload included in the target cluster; when the expected number of replicas is greater than the initial number of replicas, adjust the number of pods used to execute the workload to the expected number of replicas; when the expected number of replicas is less than the initial number of replicas, adjust the number of pods used to execute the workload to the difference between the initial number of replicas and the target constant; wherein the target parameters include: the number of pods used to execute the workload.
[0081] In an exemplary embodiment, the adjustment module 48 can also determine the expected number of replicas based on the recommended memory limit value and the recommended processor limit value in the following manner: determine a first ratio of the recommended memory limit value to the initial memory limit value; determine a third product of the first ratio and the initial number of replicas; round up the third product to obtain a first value; determine a second ratio of the recommended processor limit value to the processor limit value; determine a fourth product of the second ratio and the initial number of replicas; round up the fourth product to obtain a second value; and determine the maximum value included in the first value and the second value as the expected number of replicas.
[0082] In an exemplary embodiment, the second determination module 46 may determine the recommended memory limit value of the target pod based on the historical memory statistical data and the predicted memory data in the following manner: determine the historical maximum memory usage, the historical minimum memory usage, and the historical average memory usage included in the historical memory statistical data, and determine the weighted sum of the historical maximum memory usage, the historical minimum memory usage, the historical average memory usage, and the predicted memory usage included in the predicted memory data as the recommended memory limit value; the second determination module 46 may determine the recommended processor limit value of the target pod based on the historical processor statistical data and the predicted processor data in the following manner: determine the historical maximum processor usage, the historical minimum processor usage, and the historical average processor usage included in the historical processor statistical data, and determine the weighted sum of the historical maximum processor usage, the historical minimum processor usage, the historical average processor usage, and the predicted processor usage included in the predicted processor data as the recommended processor limit value.
[0083] In an exemplary embodiment, the first determination module 42 can determine the historical memory statistics and historical processor statistics of the target pod included in the target cluster under the workload state within a historical predetermined time period in the following manner: determine the historical maximum memory usage, historical minimum memory usage, historical average memory usage, historical maximum processor usage, historical minimum processor usage, and historical average processor usage of the target pod within the historical predetermined time period; determine the historical maximum memory usage, the historical minimum memory usage, and the historical average memory usage as the historical memory statistics; determine the historical maximum processor usage, the historical minimum processor usage, and the historical average processor usage as the historical processor statistics.
[0084] It should be noted that the above modules can be implemented by software or hardware. For the latter, it can be implemented in the following ways, but not limited to: the above modules are all located in the same processor; or the above modules are located in different processors in any combination.
[0085] An embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above method embodiments when running.
[0086] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.
[0087] An embodiment of the present application further provides an electronic device, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.
[0088] In an exemplary embodiment, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.
[0089] An embodiment of the present application further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the steps in any one of the above method embodiments are implemented.
[0090] For specific examples in this embodiment, reference may be made to the examples described in the above embodiments and exemplary implementation modes, and this embodiment will not be described in detail herein.
[0091] Obviously, those skilled in the art should understand that the above modules or steps of the present application can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order from that herein, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present application is not limited to any specific combination of hardware and software.
[0092] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the principles of the present application shall be included in the protection scope of the present application.
Claims
1. A cluster parameter adjustment method, characterized in that: include: Determine historical memory statistics and historical processor statistics of target pods included in the target cluster under the workload state within a historical predetermined time period; Predicting predicted memory data and predicted processor data of the target pod included in the target cluster at a target time under a workload state; Determine a recommended memory limit value of the target pod based on the historical memory statistical data and the predicted memory data, and determine a recommended processor limit value of the target pod based on the historical processor statistical data and the predicted processor data; The target parameter of the target cluster is adjusted based on the recommended memory limit value and the recommended processor limit value.
2. The method according to claim 1, characterized in that The adjusting the target parameter of the target cluster based on the recommended memory limit value and the recommended processor limit value includes: Determine the expansion methods allowed by the target cluster; Determine and adjust target parameters of the target cluster based on the capacity expansion method, the recommended memory limit value, and the recommended processor limit value.
3. The method according to claim 2, characterized in that The determining to adjust the capacity of the target cluster based on the capacity expansion method, the recommended memory limit value, and the recommended processor limit value includes: When the expansion mode indicates that the target cluster allows the memory limit and the processor limit of the target pod to be changed, determining a first size relationship between the recommended memory limit value and the initial memory limit; If the first size relationship indicates that the recommended memory limit value is greater than the initial memory limit value, adjusting the memory limit value of the target pod to the recommended memory limit value; If the first size relationship indicates that the recommended memory limit value is less than a first product of the initial memory limit and a first ratio, adjusting the memory limit of the target pod to the first product, wherein the first ratio is a value less than 1; Determine a second magnitude relationship between the processor limit recommendation value and the initial processor limit; If the second size relationship indicates that the recommended processor limit value is greater than the initial processor limit value, adjusting the processor limit value of the target pod to the recommended processor limit value; If the second size relationship indicates that the recommended processor limit value is less than a second product of the initial processor limit and a second ratio, adjusting the processor limit of the target pod to the second product, wherein the second ratio is a value less than 1; The target parameter includes at least one of the following: a memory limit of the target pod and a processor limit of the target pod.
4. The method according to claim 2, characterized in that: The determining to adjust the capacity of the target cluster based on the capacity expansion method, the recommended memory limit value, and the recommended processor limit value includes: When the expansion mode indicates that the target cluster allows an increase in the number of replicas, determining an expected number of replicas based on the recommended value of the memory limit and the recommended value of the processor limit, wherein the number of replicas is the number of pods used to execute the workload; Determining an initial number of replicas of pods included in the target cluster that execute the workload; When the expected number of replicas is greater than the initial number of replicas, adjusting the number of pods used to execute the workload to the expected number of replicas; When the expected number of replicas is less than the initial number of replicas, the number of pods used to execute the workload is adjusted to the difference between the initial number of replicas and the target constant; The target parameter includes the number of pods used to execute the workload.
5. The method according to claim 4, characterized in that The determining the expected number of replicas based on the recommended value of the memory limit and the recommended value of the processor limit includes: Determine a first ratio of the recommended memory limit value to the initial memory limit value; determining a third product of the first ratio and the initial number of copies; Rounding up the third product to obtain a first value; Determining a second ratio of the processor limit recommendation value to the processor limit; determining a fourth product of the second ratio and the initial number of copies; Rounding up the fourth product to obtain a second value; A maximum value included in the first value and the second value is determined as the expected number of copies.
6. The method according to claim 1, characterized in that The determining the recommended value of the memory limit of the target pod based on the historical memory statistical data and the predicted memory data comprises: determining the historical maximum memory usage, the historical minimum memory usage, and the historical average memory usage included in the historical memory statistical data, and determining the weighted sum of the historical maximum memory usage, the historical minimum memory usage, the historical average memory usage, and the predicted memory usage included in the predicted memory data as the recommended value of the memory limit; The determining the recommended processor limit value of the target pod based on the historical processor statistical data and the predicted processor data includes: determining a historical maximum processor usage, a historical minimum processor usage, and a historical average processor usage included in the historical processor statistical data, and determining a weighted sum of the historical maximum processor usage, the historical minimum processor usage, the historical average processor usage, and the predicted processor usage included in the predicted processor data as the recommended processor limit value.
7. The method according to claim 1, characterized in that The determining of historical memory statistics and historical processor statistics of the target pod included in the target cluster in the workload state within a historical predetermined time period includes: Determine the historical maximum memory usage, historical minimum memory usage, historical average memory usage, historical maximum processor usage, historical minimum processor usage, and historical average processor usage of the target pod within the historical predetermined time period; Determine the historical maximum memory usage, the historical minimum memory usage, and the historical average memory usage as the historical memory statistical data; The historical maximum processor usage, the historical minimum processor usage, and the historical average processor usage are determined as the historical processor statistical data.
8. A cluster parameter adjustment device, characterized in that: include: A first determination module is used to determine historical memory statistics and historical processor statistics of a target pod included in the target cluster in a historical predetermined time period under a workload state; A prediction module, used to predict the predicted memory data and the predicted processor data of the target pod included in the target cluster at a target time under a workload state; A second determination module, configured to determine a recommended value of a memory limit of the target pod based on the historical memory statistical data and the predicted memory data, and to determine a recommended value of a processor limit of the target pod based on the historical processor statistical data and the predicted processor data; An adjustment module is used to adjust the target parameters of the target cluster based on the recommended memory limit value and the recommended processor limit value.
9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program implements the steps of the method described in any one of claims 1 to 7 when executed by a processor.
10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method described in any one of claims 1 to 7 are implemented.
