Container processing method and device and electronic equipment

Abstract

The invention relates to a container processing method and device and electronic equipment, and relates to the technical field of cloud computing, and the method comprises the steps: obtaining the total container resource usage amount of an application, and generating a total resource usage amount time sequence of the total container resource usage amount; obtaining a candidate period set of the total resource usage amount time sequence by using a power spectral density estimation algorithm, and determining a main period from the candidate period set by using an autocorrelation function of the total resource usage amount time sequence; the peak value and the mean value of the historical total container resource usage amount of the preset number of main periods are determined based on the main periods, and the resource recommendation value and / or the memory recommendation value of the application and the Pod copy are / is calculated based on the peak value and the mean value of the historical total container resource usage amount. And the resource utilization rate is improved.

Description

Technical Field

[0001] The present application relates to the field of cloud computing technology, and in particular to a container processing method, device and electronic equipment. Background Art

[0002] With the rapid development of cloud computing technology, containerization technology has become the core means of modern application deployment and operation and maintenance due to its advantages such as lightweight, portability and efficient resource utilization. Kubernetes (k8s for short) is the current mainstream open source container cluster management platform, providing users with a series of powerful functions such as automatic application deployment, elastic scaling, and service discovery.

[0003] In the k8s environment, users need to clearly specify the number of Pod replicas and the required resource quotas such as CPU and memory when deploying applications. However, this process often relies on experience or preliminary performance testing, lacks scientific and accurate resource planning guidance, and often results in improper resource allocation in actual applications, resulting in over-allocation or under-allocation of resources, reducing resource utilization. Summary of the invention

[0004] In view of this, the present application provides a container processing method, device and electronic device, the main purpose of which is to solve the technical problem of improper resource allocation that often occurs in practical applications, resulting in over-allocation or under-allocation of resources and reduced resource utilization.

[0005] According to a first aspect of the present disclosure, there is provided a container processing method, the method comprising:

[0006] Obtaining total container resource usage of the application, and generating a total resource usage time series of the total container resource usage;

[0007] Acquire a candidate period set of the total resource usage time series by using a power spectrum density estimation algorithm, and determine a main period from the candidate period set by using an autocorrelation function of the total resource usage time series;

[0008] Based on the main cycle, the peak value and the average value of the historical total container resource usage of a preset number of main cycles are determined, and the resource recommendation value and / or the memory recommendation value of the application and the Pod copy are calculated based on the peak value and the average value of the historical total container resource usage.

[0009] According to a second aspect of the present disclosure, there is provided a container processing device, the device comprising:

[0010] An acquisition module, used to acquire total container resource usage of an application and generate a total resource usage time series of the total container resource usage;

[0011] A determination module, configured to obtain a candidate period set of the total resource usage time series using a power spectrum density estimation algorithm, and determine a main period from the candidate period set using an autocorrelation function of the total resource usage time series;

[0012] A calculation module is used to determine the peak value and average value of the historical total container resource usage of a preset number of main cycles based on the main cycle, and calculate the resource recommendation value and / or memory recommendation value of the application and Pod replica based on the peak value and average value of the historical total container resource usage.

[0013] According to the third aspect of the present disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method of the aforementioned first aspect.

[0014] According to a fourth aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable a computer to execute the method of the aforementioned first aspect.

[0015] According to a fifth aspect of the present disclosure, a computer program product is provided, comprising a computer program, wherein the computer program implements the method of the aforementioned first aspect when executed by a processor.

[0016] The container processing method, device and electronic device provided by the present disclosure, compared with the prior art, obtain the total container resource usage of the application and generate the total resource usage time series of the total container resource usage; use the power spectrum density estimation algorithm to obtain the candidate period set of the total resource usage time series, and use the autocorrelation function of the total resource usage time series to determine the main period from the candidate period set; determine the peak and mean of the historical total container resource usage of a preset number of main periods based on the main period, and calculate the resource recommendation value and / or memory recommendation value of the application and Pod replica based on the peak and mean of the historical total container resource usage. For the scheme of the present disclosure, by analyzing the peak and mean of the historical total container resource usage, the resource recommendation settings (i.e., resource recommendation values ​​and / or memory recommendation values) of the application and Pod replica are automatically generated to avoid the problem of over-allocation of resources or under-allocation of resources. The resource recommendation settings can guide users to optimize the allocation of container resources, compress the idle resources between the resource usage peak and the resource limit, thereby improving resource utilization without sacrificing application performance.

[0017] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0019] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0020] Figure 1 A schematic diagram of a process for processing a container provided by an embodiment of the present disclosure;

[0021] Figure 2 A diagram showing the effect of filling missing values ​​provided by an embodiment of the present disclosure;

[0022] Figure 3 This is a diagram showing the effect of outlier processing provided by an embodiment of the present disclosure;

[0023] Figure 4 A CPU usage sequence diagram provided by an embodiment of the present disclosure;

[0024] Figure 5 A power spectrum density estimation diagram provided by an embodiment of the present disclosure;

[0025] Figure 6 The first 50 frequency graphs of the power spectral density estimation provided by the embodiment of the present disclosure;

[0026] Figure 7 The autocorrelation function and main periodogram provided by the embodiments of the present disclosure;

[0027] Figure 8 A periodic and peak graph of CPU usage provided by an embodiment of the present disclosure;

[0028] Fig. 9 A CPU usage prediction graph provided by an embodiment of the present disclosure;

[0029] Fig.10 A schematic diagram of the structure of a container processing device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0030] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description. It should be noted that, in the absence of conflict, the embodiments of the present disclosure and the features in the embodiments may be combined with each other.

[0031] The container processing method, device and electronic device according to the embodiments of the present disclosure are described below with reference to the accompanying drawings.

[0032] The present disclosure provides a container processing method, device and electronic equipment, the main purpose of which is to solve the technical problem that improper resource allocation often occurs in practical applications, resulting in excessive or insufficient resource allocation and reduced resource utilization.

[0033] like Figure 1 As shown, an embodiment of the present disclosure provides a container processing method, the method comprising:

[0034] Step 101: Obtain the total container resource usage of the application, and generate a total resource usage time series of the total container resource usage.

[0035] The total container resource usage may be the total amount of resources jointly occupied by all Pod copies of the same application in a specific time period, which may include total CPU usage and total memory usage;

[0036] The total CPU usage can be the sum of the CPU usage generated by all Pod copies of the same application; the total memory usage can be the sum of the memory usage generated by all Pod copies of the same application;

[0037] The total resource usage time series can be a series of total container resource usage data points arranged in chronological order. Each data point represents the total container resource usage at a specific time point, which can be used to reflect the changes in application resource usage over time. The total resource usage time series can include the total CPU usage time series and the total memory usage time series.

[0038] The total CPU usage time series can be the total CPU usage data points arranged in chronological order, where each data point represents the sum of the CPU usage of all Pod replicas of the application at a specific point in time, and can be used to reflect the changes in the application's CPU resource usage over time.

[0039] The total memory usage can be the total memory usage data points arranged in chronological order, where each data point represents the sum of the memory usage of all Pod replicas of the application at a specific point in time, which can be used to reflect the changes in the application's memory resource usage over time.

[0040] For the embodiment of the present disclosure, obtaining the total container resource usage of the application and generating a total resource usage time series of the total container resource usage may specifically include:

[0041] Get the CPU usage and memory usage of multiple Pod copies of the application;

[0042] The CPU usage of multiple Pods is summed to obtain the total CPU usage, and the memory usage is summed to obtain the total memory usage. The total CPU usage is less than the sum of the CPU limits of multiple Pod replicas, and the total memory usage is less than the sum of the memory limits of multiple Pod replicas.

[0043] The total CPU usage is arranged in chronological order to generate a total CPU usage time series, and the total memory usage is arranged in chronological order to generate a total memory usage time series.

[0044] In specific application scenarios, the CPU usage and memory usage of each container can be collected from the Prometheus monitoring system of the Kubernetes container cluster.

[0045] Assume that the application has n Pod replicas, the CPU usage of each replica is marked as (pc0, pc1, ..., pcn-1), and the memory usage is marked as (pmo, pm1, ..., pmn-1).

[0046] Set the CPU limit of each Pod replica to pc limit , the memory limit is pm limit The total CPU usage of the application is c, the total memory usage is m, and the total CPU limit is c limit , total memory limit m limit , calculated by the following formula:

[0047]

[0048]

[0049] cl imit =n*pc limit

[0050] ml imit =n*pm limit

[0051] The total container resource usage of the application is collected every 30 seconds, and the total CPU usage time series () and the total memory usage time series () are generated. Among them, 0<=<, can be the index of the time series, can represent the number of sampling times, and can be the total length of the sequence.

[0052] (-1) and (-1) can represent the latest CPU usage sampling value and memory usage sampling value respectively; (-2) and (-2) can represent the previous sampling value, and so on.

[0053] Among them, in Kubernetes, Pod is the smallest deployable unit, which encapsulates one or more containers of the application and the container resources required by these containers (such as CPU, memory, storage, etc.). Because the reliability of a single Pod may not be sufficient to meet the needs of the production environment, Kubernetes allows the deployment of multiple Pod replicas to improve the availability and fault tolerance of the application.

[0054] The CPU usage may be the amount of CPU resources occupied by the Pod (or its container) in a specific time period; the memory usage may be the amount of memory resources occupied by the Pod (or its container) in a specific time period.

[0055] In Kubernetes, each Pod (or container in it) can set resource requests and resource limits. The CPU limit can be the maximum amount of CPU resources that a Pod (or container) can use. When the CPU usage of a Pod exceeds its limit, the system can limit or terminate it. The memory limit can be the maximum amount of memory resources that a Pod (or container) can use. When the memory resource usage of a Pod exceeds its limit, the system can limit or terminate it.

[0056] The total CPU limit can be the sum of the CPU limit values ​​of all Pod replicas of the same application, which represents the upper limit of CPU resources reserved by the system for the application.

[0057] The total memory limit can be the sum of the memory limit values ​​of all Pod replicas of the same application, which represents the upper limit of memory resources reserved by the system for the application.

[0058] For the embodiment of the present disclosure, after generating the total resource usage time series of the total container resource usage, the method further includes:

[0059] When there are one or more missing values ​​in the total resource usage time series, linear interpolation estimation is performed using the non-missing sample values ​​adjacent to the missing values ​​to obtain corresponding estimated missing values, and the missing values ​​are filled using the corresponding estimated missing values ​​to obtain the filled total resource usage time series; and / or,

[0060] When there are one or more outliers in the total resource usage time series, linear interpolation estimation is performed using the non-outlier sampling values ​​adjacent to the outliers to obtain corresponding estimated sampling values, and the outliers are filled with the corresponding estimated sampling values ​​to obtain the filled total resource usage time series, wherein the outliers are greater than the preset maximum sampling value or less than the preset minimum sampling value.

[0061] In the monitoring system, due to various reasons (such as system anomalies, data transmission problems, etc.), missing values ​​may appear in the total resource usage time series, that is, the data at certain time points are not recorded or transmitted. When there is a burst of traffic or the application starts, the resource usage often far exceeds the resources during normal operation, that is, outliers may appear in the total resource usage time series. Outliers can be values ​​in the data series that are significantly deviated from most data points. These missing values ​​and outliers will cause great interference to the algorithm and need to be removed.

[0062] For missing values, the non-missing sample values ​​before and after the missing value can be used to estimate the missing value.

[0063] Assuming that the CPU usage data between the kth sampling point and the mth sampling point is missing, the sampling values ​​of the missing points can be estimated using other non-missing sampling points (i.e., k+1, k+2, ..., m-1) between these two non-missing points (k and m), as shown in the following linear interpolation estimation formula:

[0064]

[0065] In the formula, () may be the estimated CPU usage sampling value at time i, () may be the non-missing CPU usage at time k, () may be the non-missing CPU usage at time m, and i may be a time point between k and m (i.e., k+1, k+2,..., m-1).

[0066] i=k+1,k+2,...,m - 1 are substituted into the linear interpolation estimation formula to obtain the following results:

[0067]

[0068]

[0069] like Figure 2 As shown in the figure, the effect of missing value filling is demonstrated, and the dots represent the values ​​after missing value filling.

[0070] For outliers in the sequence, the approach is to ignore the original sampling values ​​and then treat them as missing values. The missing value estimation formula (i.e., linear interpolation estimation formula) can be used to calculate new sampling values.

[0071] An outlier may be greater than P99.9 (ie, greater than 99.9% of the sample values ​​in the sequence) or less than P0.1 (ie, less than 0.1% of the sample values ​​in the sequence).

[0072] like Figure 3 As shown in the figure, the effect of outlier processing is shown, where dot 1 is the original outlier and dot 2 is the outlier after processing.

[0073] Step 102: Obtain a candidate period set of the total resource usage time series using a power spectrum density estimation algorithm, and determine a main period from the candidate period set using an autocorrelation function of the total resource usage time series.

[0074] Power spectral density is a method that can be used to describe the power distribution of a signal at different frequencies. If the sequence is periodic, the amplitude of the frequency corresponding to the period in the power spectral density will be significantly higher than other frequencies.

[0075] Specifically, the resource usage of an application generally shows periodicity as the business changes. After finding the period size, you can analyze and predict resource usage. If you use power spectrum density to describe the resource usage sequence, the frequency corresponding to the period of the sequence will have a significantly higher amplitude on the power spectrum density than other frequencies. Therefore, the period of the sequence can be obtained by calculating the power spectrum density.

[0076] A candidate period set of the total resource usage time series is obtained by using a power spectrum density estimation algorithm, and a main period is determined from the candidate period set by using an autocorrelation function of the total resource usage time series. Specifically, the following steps may be performed:

[0077] Performing discrete Fourier transform on the total CPU usage time series and / or the total memory usage time series using a power spectrum density estimation algorithm to obtain a transformation result;

[0078] Calculate the square of the transformed result and divide the square of the transform result by the number of samples to obtain the power spectral density estimate.

[0079] Determine a candidate period set of the total CPU usage time series and / or the total memory usage time series according to a peak value of the power spectrum density estimation;

[0080] The autocorrelation function of the total CPU usage time series and / or the total memory usage time series is calculated using fast Fourier transform and inverse fast Fourier transform respectively;

[0081] The similarity between the total CPU usage time series and / or the total memory usage time series and their corresponding delay series is calculated using the autocorrelation function, and the candidate period with the greatest similarity is selected from the candidate period set as the main period.

[0082] In a specific application scenario, the present application can use the periodogram method to estimate the power spectral density of a sequence. Suppose the sequence c(n) with N sampling points, where 0 <= n < N, is the CPU usage after preprocessing in the application. Then c(n) is an energy-limited signal. After performing the discrete Fourier transform on it and then taking the squared magnitude divided by N, it is the estimate of the true power spectral density.

[0083] Specifically, first calculate the discrete Fourier transform C(k) of c(n), where 0 <= n < N and 0 <= k < N.

[0084]

[0085] Then calculate the squared magnitude of C(k) divided by N, which is the power spectral density estimator S c (k).

[0086]

[0087] As Figure 4 shown is a time series of CPU usage in an application. As Figure 5 shown is its power spectral density estimate. Since this sequence is purely real, and the Fourier transform of a real signal has conjugate symmetry, the result of the power spectral density estimate is symmetric. Additionally, k = 0 is the DC component, which is independent of the period. So only need to focus on the power spectral density, and the corresponding frequency is frequency

[0088] Figure 6 can be Figure 5 the first 50 frequencies of the power spectral density. By magnifying and observing, it can be seen that the magnitudes of several frequencies are significantly higher than others. The periods corresponding to these frequencies are the candidate periods of the sequence.

[0089] To reduce the influence of noise, 20 frequencies with the largest magnitudes within the range of the power spectral density can be selected and added to the candidate period set, and then the time-domain analysis method is used to determine the main period T of the sequence.

[0090] For the periodic time series c(n), if its period is T, then the similarity between the sequence c(n) and its own delayed sequence c(n - T) is very high. Therefore, the resource usage time series can be delayed according to the candidate periods respectively, and the candidate period corresponding to the delayed sequence with the highest similarity to the original sequence is the main period of the sequence.

[0091] The autocorrelation function (ACF) of a sequence can reflect the similarity between the sequence and its delayed sequence. Suppose the sequence c(n) with N sampling points, where 0 <= n < N, is the preprocessed CPU usage. By performing a fast Fourier transform (FFT) on the sequence, taking the square of the magnitude, and then performing an inverse fast Fourier transform (IFFT), the autocorrelation function γ(n) of the sequence, where 0 <= n < N, can be obtained quickly.

[0092] γ(n) = IFFT[|FFT[c(n)]| 2 , 0 <= n < N

[0093] Among them, FFT and IFFT can be the fast Fourier transform and inverse transform, which are fast calculation methods for the discrete Fourier transform (DFT) and inverse discrete Fourier transform (IDFT). The formulas for the discrete Fourier transform and inverse transform are as follows:

[0094] Figure 7 Can be Figure 5 The autocorrelation function of the CPU usage sequence, the x-axis can represent the length of the delayed sequence, and the y-axis can represent the similarity between the sequence and its delayed sequence. Among them, n = 0 can represent the similarity of the sequence to itself, which is meaningless for period searching and needs to be ignored. The periods in the candidate period T' are judged respectively, and the period with the maximum similarity is taken as the main period T of the sequence.

[0095] Step 103: Determine the peak value and average value of the historical total container resource usage for a preset number of main periods based on the main period, and calculate the resource recommendation value and / or memory recommendation value for the application and Pod replicas based on the peak value and average value of the historical total container resource usage.

[0096] Among them, the historical total container resource usage can include the historical total CPU usage and the historical total memory usage.

[0097] Determining the peak value and average value of the historical total container resource usage for a preset number of main periods based on the main period, and calculating the resource recommendation value and / or memory recommendation value for the application and Pod replicas based on the peak value and average value of the historical total container resource usage can specifically include:

[0098] Determine the number of Pod replicas of the application, and determine the peak value of the historical total CPU usage for a preset number of main periods based on the main period, and / or determine the peak value of the historical total memory usage for a preset number of main periods based on the main period;

[0099] Calculate the average value of the peak values of the historical total CPU usage to obtain the average value of the historical total CPU usage, and / or calculate the average value of the peak values of the historical total memory usage to obtain the average value of the historical total memory usage. Take the average value of the historical total CPU usage as the resource recommendation value for the application, and / or take the average value of the historical total memory usage as the memory recommendation value for the application;

[0100] Divide the recommended resource value of the application by the number of Pod copies of the application to obtain the recommended resource value for each Pod copy, and / or divide the recommended memory value of the application by the number of Pod copies of the application to obtain the recommended memory value for each Pod copy.

[0101] For the embodiment of the present disclosure, after calculating the main period T of the resource usage time series, the time series of the past 5T length can be taken, and then the resource usage peak of each period is obtained, and then the average of the peak values ​​is taken as the recommended resource settings for the application. Finally, the application resource settings are divided by the number of application Pod copies expected by the user to obtain the recommended resource settings for a single copy.

[0102] Take CPU recommendation as an example. Figure 8 Can Figure 4 For the past five periods of the CPU usage time series, assuming that the application CPU usage peaks in the five periods are (peakc0, peakc1, peakc2, peakc3, peakc4), and the number of application Pod replicas specified by the user is n, the recommended CPU setting of the application recommendCPU and the recommended CPU setting of the Pod replica recommendPodCPU can be calculated using the following method:

[0103]

[0104] The same approach can be used to calculate the recommended memory settings for applications and Pod replicas.

[0105] Since resource setting adjustments involve all Pod replicas, in order to maintain application stability, frequent automatic adjustments are not performed. Instead, resource optimization plans are pushed to users, who initiate resource adjustments based on actual conditions. After optimizing according to the recommended resources, idle resources between the application's resource usage peak and resource limits are released, improving resource utilization and reducing hardware investment costs. After setting according to the recommended resource plan, there will be no waste of resources even if a single replica is running.

[0106] Step 104: Use the historical total container resource usage to predict the target total container resource usage, and perform dynamic expansion and contraction of containers according to the total container resource usage and the target total container resource usage.

[0107] The target total container resource usage may include a target total CPU usage and a target total memory usage.

[0108] For the embodiment of the present disclosure, the target total container resource usage is predicted by using the historical total container resource usage, and the dynamic expansion and contraction of the container is performed according to the total container resource usage and the target total container resource usage, which may specifically include:

[0109] The average of the historical total CPU usage is multiplied by the preset gain factor to obtain the target total CPU usage, and the average of the historical total memory usage is multiplied by the preset gain factor to obtain the target total memory usage;

[0110] The current CPU usage is determined based on the total CPU usage and the sum of the CPU limit, the current memory usage is determined based on the total memory usage and the sum of the memory limit, the target CPU usage peak is determined based on the target total CPU usage peak and the sum of the CPU limit, and the target memory usage peak is determined based on the target total memory usage peak and the sum of the memory limit;

[0111] Calculate the expected number of Pod replicas based on the current CPU usage, current memory usage, target CPU usage peak, and target memory usage peak.

[0112] The maximum value of multiple expected Pod replicas is selected as the target expected Pod replica number, so as to perform dynamic expansion and contraction of containers according to the target expected Pod replica number.

[0113] In specific application scenarios, after the recommended resource settings are adjusted, resource utilization is improved, but during business peak periods, resource usage will be close to resource limits. High resource utilization may affect application performance. To address this issue, this application proposes a dynamic container scaling algorithm, which increases the number of Pod copies of the application during business peak periods to ensure sufficient resources, and reduces the number of Pod copies of the application during business off-peak periods to recycle idle resources.

[0114] This application first predicts the resource usage in the next cycle based on the past five cycles of the resource usage sequence, and then uses the current CPU usage and memory resource usage to predict the peak CPU usage and memory usage in the next half hour, calculates the expected number of Pod copies of the application, and then takes the maximum value to scale the container.

[0115] Specifically, after calculating the main period T of the resource usage time series, take the time series of the past 5T length, then calculate the average of the resource usage at the same time in each period, and then multiply the average by a gain factor as the resource usage prediction for the same time in the next period. Assuming that the CPU usage corresponding to the first sampling point in the past 5 periods is (historyCPU0, historyCPU1, historyCPU2, historyCPU3, historyCPU4), and the gain factor is σ, then the CPU usage predictCPU of the first sampling point in the next period is predicted to be:

[0116]

[0117] The function of the gain factor σ is to appropriately increase the predicted resource usage so as to issue a resource high load warning in a timely manner. For example, σ=1.05 may indicate that the predicted resource usage is increased by 5%.

[0118] Since the sampling interval is 30 seconds, predictCPU is the predicted CPU usage in 30 seconds, and so on, the CPU usage of each sampling point in the next half hour is predicted. The same method is used to predict the memory usage in the next half hour.

[0119] Fig. 9 Can be based on Figure 4 The predicted future CPU resource usage can be plotted together with the actual resource usage for easy comparison.

[0120] Specifically, assuming that the current CPU usage of the application is c, the CPU limit is climit, the predicted CPU usage peak in the next half hour is predict_c, the memory usage is m, the memory limit is mlimit, and the predicted memory usage peak in the next half hour is predict_m, then the current CPU usage c_rate, the predicted CPU usage peak predict_c_rate, the current memory usage m_rate, and the predicted memory usage peak predict_m_rate of the application are calculated using the following formula:

[0121]

[0122] Assuming that the user specifies the expected CPU usage as exp_c_rate, the specified expected memory usage as exp_m_rate, and the current number of Pod copies of the application is n, the expected number of copies of the application that meets the current CPU usage, exp_n1, is:

[0123]

[0124] The ceil() function in the formula rounds up.

[0125] The expected number of replicas of the application that meets the predicted CPU usage, exp_n2, is:

[0126]

[0127] The expected number of replicas exp_n3 for the application that meets the current memory usage is:

[0128]

[0129]

[0130] The expected number of replicas of the application that meets the predicted memory usage exp_n4 is:

[0131]

[0132] The maximum value of the expected number of application replicas calculated by the four indicators is taken as the final expected number of application replicas exp_n.

[0133] exp_n=max(exp_n1,exp_n2,exp_n3,exp_n4)

[0134] To avoid too many or too few Pod copies of an application, you can set the maximum number of copies max_n and the minimum number of copies min_n, constrain the expected number of copies to this range, and get the final expected number of copies of the application new_exp_n. Finally, set the number of copies of the application to new_exp_n, call the k8s apiserver interface, and perform container expansion and contraction.

[0135] new_exp_n=min(max(exp_n,min_n),max_n)

[0136] The dynamic scaling algorithm for containers based on current and predicted resource usage can achieve capacity expansion before the peak of business, improve the timeliness of capacity expansion, ensure sufficient resources, and avoid application performance degradation. Combined with the recommended use of container resources, it can maximize resource compression and improve resource utilization while ensuring good application performance.

[0137] Based on power spectrum density estimation and autocorrelation function, the resource usage cycle is detected and future resource usage is predicted. This method does not require training, has fast calculation speed, requires less computing resources, and has high accuracy. It is particularly suitable for resource usage analysis and prediction scenarios with high real-time requirements.

[0138] In summary, compared with the prior art, the container processing method disclosed in the present invention obtains the total container resource usage of the application and generates a total resource usage time series of the total container resource usage; uses the power spectral density estimation algorithm to obtain a set of candidate periods of the total resource usage time series, and uses the autocorrelation function of the total resource usage time series to determine the main period from the candidate period set; based on the main period, the peak and mean of the historical total container resource usage of a preset number of main periods are determined, and the resource recommendation value and / or memory recommendation value of the application and Pod replica are calculated based on the peak and mean of the historical total container resource usage. For the disclosed solution, by analyzing the peak and mean of the historical total container resource usage, the resource recommendation settings (i.e., resource recommendation values ​​and / or memory recommendation values) of the application and Pod replica are automatically generated to avoid the problem of over-allocation of resources or insufficient allocation of resources. The resource recommendation settings can guide users to optimize the allocation of container resources, compress the idle resources between the resource usage peak and the resource limit, thereby improving resource utilization without sacrificing application performance.

[0139] Based on the above Figure 1 The present disclosure provides a container processing device, such as Fig.10 As shown, the device includes: an acquisition module 31, a determination module 32, a calculation module 33, and an execution module 34;

[0140] An acquisition module 31 is used to acquire the total container resource usage of the application and generate a total resource usage time series of the total container resource usage;

[0141] A determination module 32, configured to obtain a candidate period set of the total resource usage time series using a power spectrum density estimation algorithm, and determine a main period from the candidate period set using an autocorrelation function of the total resource usage time series;

[0142] The calculation module 33 is used to determine the peak value and average value of the historical total container resource usage of a preset number of main cycles based on the main cycle, and calculate the resource recommendation value and / or memory recommendation value of the application and Pod replica based on the peak value and average value of the historical total container resource usage.

[0143] In a specific application scenario, the device further includes: a first filling module 34, a second filling module 35;

[0144] A first filling module 34 is used to, when there are one or more missing values ​​in the total resource usage time series, use the non-missing sampling values ​​adjacent to the missing values ​​to perform linear interpolation estimation processing to obtain corresponding estimated missing values, and use the corresponding estimated missing values ​​to fill the missing values ​​to obtain the filled total resource usage time series; and / or,

[0145] The second filling module 35 is used to perform linear interpolation estimation processing using non-abnormal sampling values ​​adjacent to the abnormal values ​​to obtain corresponding estimated sampling values, and to fill the abnormal values ​​with the corresponding estimated sampling values ​​to obtain a filled total resource usage time series, when there are one or more abnormal values ​​in the total resource usage time series, wherein the abnormal value is greater than a preset maximum sampling value or less than a preset minimum sampling value.

[0146] In a specific application scenario, the total container resource usage includes total CPU usage and total memory usage; the total resource usage time series includes total CPU usage time series and total memory usage time series; the acquisition module 31 can be used to obtain the CPU usage and memory usage of multiple Pod copies of the application;

[0147] The multiple CPU usages are summed to obtain the total CPU usage, and the memory usages are summed to obtain the total memory usage, wherein the total CPU usage is less than the sum of the CPU limits of the multiple Pod replicas, and the total memory usage is less than the sum of the memory limits of the multiple Pod replicas;

[0148] The total CPU usage is arranged in chronological order to generate the total CPU usage time series, and the total memory usage is arranged in chronological order to generate the total memory usage time series.

[0149] In a specific application scenario, the determination module 32 may be used to perform a discrete Fourier transform on the total CPU usage time series and / or the total memory usage time series using the power spectrum density estimation algorithm to obtain a transformation result;

[0150] Calculating the square of the amplitude of the transformation result, and dividing the square of the amplitude by the number of samples to obtain a power spectral density estimate;

[0151] Determine a candidate period set of the total CPU usage time series and / or the total memory usage time series according to the peak value of the power spectrum density estimation;

[0152] Calculate the autocorrelation function of the total CPU usage time series and / or the total memory usage time series respectively by using fast Fourier transform and inverse fast Fourier transform;

[0153] The autocorrelation function is used to respectively calculate the similarity between the total CPU usage time series and / or the total memory usage time series and their corresponding delay series, and the candidate cycle with the greatest similarity is selected from the candidate cycle set as the main cycle.

[0154] In a specific application scenario, the historical total container resource usage includes the historical total CPU usage and the historical total memory usage; the calculation module 33 can be used to determine the number of Pod copies of the application, and determine the peak value of the historical total CPU usage of a preset number of main cycles based on the main cycle, and / or determine the peak value of the historical total memory usage of a preset number of main cycles based on the main cycle;

[0155] Calculating the average value of the peak values ​​of the historical total CPU usage to obtain the mean value of the historical total CPU usage, and / or calculating the average value of the peak values ​​of the historical total memory usage to obtain the mean value of the historical total memory usage, using the mean value of the historical total CPU usage as the resource recommendation value of the application, and / or using the mean value of the historical total memory usage as the memory recommendation value of the application;

[0156] The recommended resource value of the application is divided by the number of Pod copies of the application to obtain the recommended resource value of each Pod copy, and / or the recommended memory value of the application is divided by the number of Pod copies of the application to obtain the recommended memory value of each Pod copy.

[0157] In a specific application scenario, the device further includes: an execution module 36;

[0158] The execution module 36 is used to predict the target total container resource usage by using the historical total container resource usage, and perform dynamic expansion and contraction of containers according to the total container resource usage and the target total container resource usage.

[0159] In a specific application scenario, the target total container resource usage includes a target total CPU usage and a target total memory usage. The execution module 36 may be used to multiply the average of the historical total CPU usage by a preset gain factor to obtain the target total CPU usage, and to multiply the average of the historical total memory usage by the preset gain factor to obtain the target total memory usage.

[0160] Determine the CPU usage at the current moment according to the total CPU usage and the sum of the CPU limits, determine the memory usage at the current moment according to the total memory usage and the sum of the memory limits, determine the target CPU usage peak according to the peak value of the target total CPU usage and the sum of the CPU limits, and determine the target memory usage peak according to the peak value of the target total memory usage and the sum of the memory limits;

[0161] Calculate the corresponding expected number of Pod copies according to the current CPU usage, the current memory usage, the target CPU usage peak and the target memory usage peak;

[0162] The maximum value of the multiple expected Pod replica numbers is selected as the target expected Pod replica number, so as to perform dynamic expansion and contraction of the container according to the target expected Pod replica number.

[0163] It should be noted that for other corresponding descriptions of the functional units involved in the container processing device provided by the present disclosure, reference can be made to Figure 1 The corresponding description in will not be repeated here.

[0164] Based on the above Figure 1 The method shown in the present disclosure, accordingly, also provides a computer-readable storage medium having a computer program stored thereon, which implements the above-mentioned Figure 1 The method shown.

[0165] Based on this understanding, the technical solution of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.), and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods of various implementation scenarios of the present disclosure.

[0166] Based on the above Figure 1 The method shown, and Fig.10 In order to achieve the above-mentioned purpose, the embodiment of the present disclosure also provides an electronic device, which includes a storage medium and a processor; the storage medium is used to store a computer program; the processor is used to execute the computer program to achieve the above-mentioned Figure 1 The method shown.

[0167] Optionally, the above-mentioned physical device may also include a user interface, a network interface, a camera, a radio frequency (RF) circuit, a sensor, an audio circuit, a WI-FI module, etc. The user interface may include a display, an input unit such as a keyboard, etc., and the optional user interface may also include a USB interface, a card reader interface, etc. The network interface may optionally include a standard wired interface, a wireless interface (such as a WI-FI interface), etc.

[0168] Those skilled in the art will appreciate that the above-mentioned physical device structure provided by the present disclosure does not constitute a limitation on the physical device, and may include more or fewer components, or a combination of certain components, or different arrangements of components.

[0169] The storage medium may also include an operating system and a network communication module. The operating system is a program that manages the hardware and software resources of the above-mentioned physical device, and supports the operation of the information processing program and other software and / or programs. The network communication module is used to realize the communication between the components inside the storage medium, and the communication with other hardware and software in the information processing physical device.

[0170] Through the description of the above implementation methods, those skilled in the art can clearly understand that the present disclosure can be implemented by means of software plus necessary general hardware platforms, or by hardware. According to the container processing method, device and electronic device provided by the present disclosure, compared with the prior art, the present disclosure obtains the total container resource usage of the application and generates a total resource usage time series of the total container resource usage; obtains the candidate period set of the total resource usage time series using the power spectrum density estimation algorithm, and determines the main period from the candidate period set using the autocorrelation function of the total resource usage time series; determines the peak and mean of the historical total container resource usage of a preset number of main periods based on the main period, and calculates the resource recommendation value and / or memory recommendation value of the application and Pod replica based on the peak and mean of the historical total container resource usage. For the disclosed scheme, by analyzing the peak and mean of the historical total container resource usage, the resource recommendation settings (i.e., resource recommendation value and / or memory recommendation value) of the application and Pod replica are automatically generated to avoid the problem of over-allocation or under-allocation of resources. Resource recommendation settings can guide users to optimize the allocation of container resources and compress idle resources between resource usage peaks and resource limits, thereby improving resource utilization without sacrificing application performance.

[0171] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the term "comprising" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprising a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0172] The above are only specific embodiments of the present disclosure, so that those skilled in the art can understand or implement the present disclosure. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure will not be limited to these embodiments herein, but will conform to the widest scope consistent with the principles and novel features applied herein.

Claims

1. A container processing method, characterized in that: The method comprises: Obtaining total container resource usage of the application, and generating a total resource usage time series of the total container resource usage; Acquire a candidate period set of the total resource usage time series by using a power spectrum density estimation algorithm, and determine a main period from the candidate period set by using an autocorrelation function of the total resource usage time series; Based on the main cycle, the peak value and the average value of the historical total container resource usage of a preset number of main cycles are determined, and the resource recommendation value and / or the memory recommendation value of the application and the Pod copy are calculated based on the peak value and the average value of the historical total container resource usage.

2. The method according to claim 1, characterized in that After generating a total resource usage time series of the total container resource usage, the method further includes: When there are one or more missing values ​​in the total resource usage time series, linear interpolation estimation is performed using non-missing sampling values ​​adjacent to the missing values ​​to obtain corresponding estimated missing values, and the missing values ​​are filled using the corresponding estimated missing values ​​to obtain a filled total resource usage time series; and / or, When there are one or more abnormal values ​​in the total resource usage time series, linear interpolation estimation is performed using the non-abnormal sampling values ​​adjacent to the abnormal values ​​to obtain corresponding estimated sampling values, and the abnormal values ​​are filled with the corresponding estimated sampling values ​​to obtain a filled total resource usage time series, wherein the abnormal values ​​are greater than a preset maximum sampling value or less than a preset minimum sampling value.

3. The method according to claim 1, characterized in that The total container resource usage includes total CPU usage and total memory usage; the total resource usage time series includes total CPU usage time series and total memory usage time series; The obtaining of the total container resource usage of the application and generating a total resource usage time series of the total container resource usage includes: Get the CPU usage and memory usage of multiple Pod copies of the application; The multiple CPU usages are summed to obtain the total CPU usage, and the memory usages are summed to obtain the total memory usage, wherein the total CPU usage is less than the sum of the CPU limits of the multiple Pod replicas, and the total memory usage is less than the sum of the memory limits of the multiple Pod replicas; The total CPU usage is arranged in chronological order to generate the total CPU usage time series, and the total memory usage is arranged in chronological order to generate the total memory usage time series.

4. The method according to claim 3, characterized in that The step of acquiring a candidate period set of the total resource usage time series by using a power spectrum density estimation algorithm, and determining a main period from the candidate period set by using an autocorrelation function of the total resource usage time series, includes: Using the power spectrum density estimation algorithm, performing a discrete Fourier transform on the total CPU usage time series and / or the total memory usage time series to obtain a transformation result; Calculating the square of the amplitude of the transformation result, and dividing the square of the amplitude by the number of samples to obtain a power spectral density estimate; Determine a candidate period set of the total CPU usage time series and / or the total memory usage time series according to the peak value of the power spectrum density estimation; Calculate the autocorrelation function of the total CPU usage time series and / or the total memory usage time series respectively by using fast Fourier transform and inverse fast Fourier transform; The autocorrelation function is used to respectively calculate the similarity between the total CPU usage time series and / or the total memory usage time series and their corresponding delay series, and the candidate cycle with the greatest similarity is selected from the candidate cycle set as the main cycle.

5. The method according to claim 1, characterized in that The historical total container resource usage includes the historical total CPU usage and the historical total memory usage; The determining the peak value and the average value of the historical total container resource usage of a preset number of main cycles based on the main cycle, and calculating the resource recommendation value and / or the memory recommendation value of the application and the Pod replica based on the peak value and the average value of the historical total container resource usage, includes: Determine the number of Pod copies of the application, and determine the peak value of the historical total CPU usage of a preset number of main cycles based on the main cycle, and / or determine the peak value of the historical total memory usage of a preset number of main cycles based on the main cycle; Calculating the average value of the peak values ​​of the historical total CPU usage to obtain the mean value of the historical total CPU usage, and / or calculating the average value of the peak values ​​of the historical total memory usage to obtain the mean value of the historical total memory usage, using the mean value of the historical total CPU usage as the resource recommendation value of the application, and / or using the mean value of the historical total memory usage as the memory recommendation value of the application; The recommended resource value of the application is divided by the number of Pod copies of the application to obtain the recommended resource value of each Pod copy, and / or the recommended memory value of the application is divided by the number of Pod copies of the application to obtain the recommended memory value of each Pod copy.

6. The method according to claim 1, characterized in that The method further comprises: The target total container resource usage is predicted using the historical total container resource usage, and dynamic expansion and contraction of containers is performed according to the total container resource usage and the target total container resource usage.

7. The method according to claim 6, characterized in that The target total container resource usage includes a target total CPU usage and a target total memory usage; The method of using the historical total container resource usage to predict the target total container resource usage, and performing dynamic expansion and contraction of containers according to the total container resource usage and the target total container resource usage, includes: The target total CPU usage is obtained by multiplying the average of the historical total CPU usage by the preset gain factor, and the target total memory usage is obtained by multiplying the average of the historical total memory usage by the preset gain factor; Determine the CPU usage at the current moment according to the total CPU usage and the sum of the CPU limits, determine the memory usage at the current moment according to the total memory usage and the sum of the memory limits, determine the target CPU usage peak according to the peak value of the target total CPU usage and the sum of the CPU limits, and determine the target memory usage peak according to the peak value of the target total memory usage and the sum of the memory limits; Calculate the corresponding expected number of Pod copies according to the current CPU usage, the current memory usage, the target CPU usage peak and the target memory usage peak; The maximum value of the multiple expected Pod replica numbers is selected as the target expected Pod replica number, so as to perform dynamic expansion and contraction of the container according to the target expected Pod replica number.

8. A container processing device, the device comprising: An acquisition module, used to acquire total container resource usage of an application and generate a total resource usage time series of the total container resource usage; A determination module, configured to obtain a candidate period set of the total resource usage time series using a power spectrum density estimation algorithm, and determine a main period from the candidate period set using an autocorrelation function of the total resource usage time series; A calculation module is used to determine the peak value and average value of the historical total container resource usage of a preset number of main cycles based on the main cycle, and calculate the resource recommendation value and / or memory recommendation value of the application and Pod replica based on the peak value and average value of the historical total container resource usage.

9. An electronic device, comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 7.

10. A computer program product, characterized in that The invention comprises a computer program which, when executed by a processor, implements the method according to any one of claims 1 to 7.

Images