A method, system, medium and electronic terminal for adjusting container resources

Abstract

The present invention provides a method, system, medium and electronic terminal for adjusting container resources. The method includes: presetting a collection period, collecting the access times of the front-end containers of the service, and obtaining a service access times sequence; establishing an access times prediction model according to the service access times sequence; collecting the resource consumption of the containers of the service according to the collection period, and obtaining a resource consumption sequence; determining a resource consumption prediction model according to the service access times sequence and the resource consumption sequence; fitting the access times prediction model and the resource consumption prediction model to obtain a container resource prediction model; using the container resource prediction model to perform container resource prediction and adjustment. The container resource adjustment method in the present invention realizes the organic combination of access times and resource requirements, and at the same time, realizes the dynamic adjustment of the resource amount of microservice applications, effectively improving resource utilization rate.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a method, system, medium and electronic terminal for adjusting container resources. Background Art

[0002] K8s (Kubernetes) performs service quality management on pods (containers) through QoS (Quality of Service), and provides two types of requests (requirements) and limits (restrictions) for resource allocation and usage limits. Currently, the general dynamic resource adjustment methods are generally divided into two types. One is HPA (horizontal pod autoscaling), and the other is VPA (Vertical pod autoscaler). HAP dynamically adjusts the number of containers based on CPU utilization, and it will not play a good role in some scenarios that strongly depend on persistent storage. Because when scaling down occurs, the data persisted by the scaled-down containers cannot be automatically transferred to the running containers, which may cause data loss; VPA cannot effectively avoid memory OOM (out of memory) in certain cases. The suggestions of VPA may exceed the available resources, and in some specific scenarios, VPA will cause the deleted pods not to be recreated.

[0003] However, the above methods do not consider the periodic characteristics of application resource requirements, nor the relationship between the access times of microservices and the application's resource requirements. They only passively adjust the number of pods or the resource amount of pods, and cannot support the dynamic scaling and adjustment of container resources well, resulting in low resource utilization. Summary of the Invention

[0004] The present invention provides a method, system, medium and electronic terminal for adjusting container resources to solve the problems in the prior art that the adjustment of container resources does not consider the periodic characteristics of application resource requirements, nor the relationship between the access times of microservices and the application's resource requirements. It only passively adjusts the number of pods or the resource amount of pods, and cannot support the dynamic scaling and adjustment of container resources well, and the resource utilization rate is low.

[0005] The method for adjusting container resources provided by the present invention includes:

[0006] Pre-set a collection period, collect the access times of the front-end containers of the service, and obtain a service access times sequence;

[0007] According to the service access times sequence, establish an access times prediction model;

[0008] According to the collection period, collect the resource consumption of the container of the collection service, and obtain a resource consumption sequence;

[0009] According to the service access times sequence and the resource consumption sequence, determine a resource consumption prediction model;

[0010] Fit the access times prediction model and the resource consumption prediction model to obtain a container resource prediction model;

[0011] Use the container resource prediction model to perform container resource prediction and adjustment.

[0012] Optionally, the service is a microservice. The steps of establishing an access times prediction model according to the service access times sequence include:

[0013] Perform a first-order cumulative addition on the service access times sequence to obtain a first first-order cumulative addition sequence;

[0014] Establish a differential equation in white form for the first first-order cumulative addition sequence;

[0015] According to the service access times sequence and the first first-order cumulative addition sequence, use the least squares method to obtain the variable constants in the differential equation;

[0016] According to the variable constants and the differential equation, obtain a second first-order cumulative addition sequence;

[0017] Use the second first-order cumulative addition sequence to perform access times prediction to obtain an access times prediction sequence;

[0018] According to the service access times sequence and the access times prediction sequence, establish an access times prediction model.

[0019] Optionally, the steps of establishing an access times prediction model according to the service access times sequence and the access times prediction sequence include:

[0020] Obtain the difference between the access times prediction sequence and the service access times sequence to obtain a difference sequence;

[0021] Represent the difference sequence with a preset autoregressive integrated moving average model to obtain an autoregressive equation and a moving average equation;

[0022] According to the autoregressive equation and the moving average equation, obtain an autoregressive integrated moving average model;

[0023] Perform a difference processing on the difference sequence in the autoregressive integrated moving average model to obtain a difference sequence after difference processing;

[0024] According to the difference sequence after difference processing, obtain a difference prediction sequence;

[0025] Based on the access count prediction sequence and the difference prediction sequence, establish the access count prediction model.

[0026] Optionally, the steps of determining the resource consumption prediction model according to the service access count sequence and the resource consumption sequence include:

[0027] Construct a linear equation for resource consumption prediction, and the resource consumption prediction equation includes correlation parameters for representing the correlation relationship between service access counts and resource consumption;

[0028] Perform matrix transformation on the linear equation for resource consumption prediction to obtain a first resource consumption prediction matrix equation, and the first resource consumption prediction matrix equation is the matrix representation of the linear equation for resource consumption prediction;

[0029] Substitute the service access count sequence and the resource consumption sequence into the first resource consumption prediction matrix equation, and use the least squares method to determine the values of the correlation parameters;

[0030] According to the values of the correlation parameters, determine a second resource consumption prediction matrix equation to complete the construction of the resource consumption prediction model.

[0031] Optionally, the steps of fitting the access count prediction model and the resource consumption prediction model to obtain a container resource prediction model include:

[0032] According to the access count prediction model and the second resource consumption prediction matrix equation in the resource consumption prediction model, obtain a container resource prediction model based on access counts;

[0033] The mathematical expression of the container resource prediction model is:

[0034]

[0035] where, is the predicted value of the resource consumption at time and are the correlation parameters in the second resource consumption prediction matrix equation, is the predicted value of the access count at time is the access count prediction difference.

[0036] Optionally, the steps of performing container resource prediction and adjustment using the container resource prediction model include:

[0037] Use the container resource prediction model to predict the resource consumption of containers, and obtain the predicted average resource consumption value and the predicted peak resource consumption value of the service. The average resource consumption is the average value of the resources consumed by the service within a fixed time period, and the peak resource consumption is the maximum value of the resources consumed by the service within a fixed time period;

[0038] Take the predicted average resource consumption value as the resource demand quantity, and take the predicted peak resource consumption value as the resource limit quantity;

[0039] Adjust the container resources of the service according to the resource demand quantity and the resource limit quantity.

[0040] Optionally, the steps of using the container resource prediction model for container resource prediction and adjustment further include:

[0041] Use the container resource prediction model to predict the resource consumption of containers, and obtain the predicted average CPU resource consumption value, the predicted peak CPU resource consumption value, the predicted average memory resource consumption value, and the predicted peak memory resource consumption value of the service;

[0042] Obtain the CPU demand quantity, the CPU limit quantity, the memory demand quantity, and the memory limit quantity according to the predicted average CPU resource consumption value, the predicted peak CPU resource consumption value, the predicted average memory resource consumption value, and the predicted peak memory resource consumption value;

[0043] Adjust the container resources according to the CPU demand quantity, the CPU limit quantity, the memory demand quantity, and the memory limit quantity.

[0044] The present invention also provides a container resource adjustment system, including:

[0045] An access frequency prediction model construction module, configured to preset a collection period, collect the access frequencies of the front-end containers of the service, and obtain a service access frequency sequence; establish an access frequency prediction model according to the service access frequency sequence;

[0046] A resource consumption prediction model construction module, configured to collect the resource consumption amounts of the containers of the service according to the collection period, and obtain a resource consumption sequence; determine a resource consumption prediction model according to the service access frequency sequence and the resource consumption sequence;

[0047] A container resource prediction model acquisition module, configured to fit the access frequency prediction model and the resource consumption prediction model to obtain a container resource prediction model;

[0048] A resource adjustment module, configured to use the container resource prediction model to perform container resource prediction and adjustment.

[0049] The present invention also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the method described in any one of the above is implemented.

[0050] The present invention also provides an electronic terminal, including: a processor and a memory;

[0051] The memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory, so that the terminal executes the method described in any one of the above.

[0052] Advantages of the present invention: In the container resource adjustment method, system, medium and electronic terminal of the present invention, by collecting the access times of the front-end containers in the service according to a preset collection period, obtaining the service access times sequence, establishing an access times prediction model according to the service access times sequence, collecting the resource consumption of the containers in the service according to the collection period, obtaining the resource consumption sequence, determining the resource consumption prediction model according to the service access times sequence and the resource consumption sequence, and fitting the access times prediction model and the resource consumption prediction model, a container resource prediction model is obtained. Furthermore, the container resource prediction model is used for container resource prediction and adjustment, realizing the organic combination of access times and resource requirements. At the same time, the dynamic adjustment of the resource amount of the microservice application is realized, effectively improving the resource utilization rate. Description of the Drawings

[0053] Figure 1 It is a flowchart of the container resource adjustment method in an embodiment of the present invention.

[0054] Figure 2 It is a flowchart of establishing an access times prediction model in the container resource adjustment method in an embodiment of the present invention.

[0055] Figure 3 It is a flowchart of determining the resource consumption prediction model in the container resource adjustment method in an embodiment of the present invention.

[0056] Figure 4 It is a flowchart of obtaining the container resource prediction model in the container resource adjustment method in an embodiment of the present invention.

[0057] Figure 5 It is a flowchart of using the container resource prediction model for container resource prediction and adjustment in the container resource adjustment method in an embodiment of the present invention.

[0058] Figure 6 It is a schematic framework diagram of a general microservice in Embodiment 1 of the present invention.

[0059] Figure 7 It is a schematic structural diagram of the container resource adjustment system in an embodiment of the present invention. Detailed implementation manners

[0060] The following describes the implementation manners of the present invention through specific examples. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific implementation manners. Various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that, without conflict, the following embodiments and the features in the embodiments can be combined with each other.

[0061] It should be noted that the diagrams provided in the following embodiments only illustrate the basic concept of the present invention in a schematic manner. Therefore, only the components related to the present invention are shown in the diagrams, rather than being drawn according to the number, shape, and size of the components in actual implementation. The type, quantity, and proportion of each component in actual implementation can be arbitrarily changed, and the component layout type may also be more complex.

[0062] The inventor found that Kubernetes manages the service quality of pods (containers) through QoS (Quality of Service), and provides two types of requests and limits for resource allocation and usage limits. Requests define the minimum amount of resources required by the corresponding container, and limits define the maximum amount of resources that the corresponding container can consume. Currently, the common dynamic resource adjustment methods are generally divided into two types. One is HPA (horizontal pod autoscaling), and the other is VPA (Vertical pod autoscaler). HAP dynamically adjusts the number of containers based on CPU utilization, and it will not work well for some scenarios that strongly depend on persistent storage. Because when scaling down occurs, the data persisted by the scaled-down containers cannot be automatically transferred to the running containers, which may cause data loss; VPA cannot effectively avoid memory OOM in specific cases. The suggestions of VPA may exceed the available resources, and in some specific scenarios, VPA will cause the deleted pods not to be recreated. However, the above methods do not consider the periodic characteristics of application resource requirements, nor the relationship between the number of accesses to microservices and the application's resource requirements. They only passively adjust the number of pods or the resource amount of pods, and cannot support the dynamic scaling and adjustment of container resources well, resulting in low resource utilization. Therefore, the inventor proposes a container resource adjustment method, system, medium, and electronic terminal. By collecting the number of accesses to the front-end containers in the service according to a pre-set collection period, obtaining the service access count sequence, establishing an access count prediction model based on the service access count sequence, collecting the resource consumption of the containers in the service according to the collection period, obtaining the resource consumption sequence, determining the resource consumption prediction model based on the service access count sequence and the resource consumption sequence, and fitting the access count prediction model and the resource consumption prediction model to obtain the container resource prediction model. Then, using the container resource prediction model for container resource prediction and adjustment, it fully considers the periodic characteristics of application resource requirements, realizes the organic combination of access count and resource requirements, can dynamically predict the required resource amount of microservice applications at each stage. At the same time, it realizes the active adjustment of the resource amount of microservice applications, effectively improves resource utilization, has high feasibility, and low cost.

[0063] As Figure 1 shown, the container resource adjustment method in this embodiment includes:

[0064] S101: Preset a collection period, collect the access times of the front-end container of the service, and obtain a service access times sequence; the service is a microservice, and the collection period can be set according to the actual situation, such as counting the access times of the front-end container of the service every three minutes, etc., which will not be elaborated here. In a typical service framework, the front-end and back-end containers are usually in a separated state, that is, independent of each other. Therefore, by collecting the access times of the front-end container of the service, the access times of the service within a preset time period can be better obtained, which is convenient for subsequent prediction of the access times of the service. The above-mentioned collected access times of the front-end container of the service are the total access times within the cycle time.

[0065] S102: Establish an access times prediction model according to the service access times sequence; the access times prediction model is a fitting model that combines a metabolic grey model (MGM) and an autoregressive integrated moving average model (ARIMA). By establishing an access times prediction model according to the service access times sequence, the prediction of the access times of the service can be better realized, with a higher accuracy rate and stronger feasibility.

[0066] S103: According to the collection period, collect the resource consumption of the service container, and obtain a resource consumption sequence; by regularly collecting the resource consumption of the service container, it is convenient to count the resource consumption of the microservice and is also convenient for subsequent prediction of the resource consumption of the container.

[0067] S104: Determine a resource consumption prediction model according to the service access times sequence and the resource consumption sequence; that is, determine the correlation between the access times of the service and the resource consumption according to the service access times sequence and the resource consumption sequence, and then determine the resource consumption prediction model.

[0068] S105: Fit the access times prediction model and the resource consumption prediction model to obtain a container resource prediction model; by fitting the access times prediction model and the resource consumption prediction model, a container resource prediction model based on the access times can be better obtained, with a higher model prediction accuracy rate, realizing dynamic prediction of the required resource amount for service applications at each stage, and being more convenient to implement.

[0069] S106: Use the container resource prediction model for container resource prediction and adjustment. That is, use the container resource prediction model to predict and adjust the resource consumption and resource limit of the service container. By using the container resource prediction model for container resource prediction and adjustment, the active prediction and adjustment of the resources required by the microservice can be realized, effectively improving the resource utilization rate of the container in the service.

[0070] As Figure 2 shown, according to the service access frequency sequence, the steps of establishing an access frequency prediction model include:

[0071] S201: Perform a first-order cumulative addition on the service access frequency sequence to obtain a first first-order cumulative addition sequence;

[0072] For example: According to a pre-set collection period, regularly collect the access frequency of the front-end container of the service to obtain a service access frequency sequence. Suppose the access frequency sequence of a certain service collected is:

[0073]

[0074] wherein, is the service access frequency sequence, is the total access frequency of a certain service from the (n - 1)-th moment to the n-th moment.

[0075] Perform a first-order cumulative addition on the above service access frequency sequence to obtain a first first-order cumulative addition sequence. The mathematical expression of the first first-order cumulative addition sequence is:

[0076]

[0077] ,

[0078] wherein, is the first first-order cumulative addition sequence.

[0079] S202: Establish a differential equation in white form for the first first-order cumulative addition sequence; the mathematical expression of the differential equation is:

[0080]

[0081] wherein, is the data in the first first-order cumulative addition sequence, t is the time sequence, and a and b are variable constants in the differential equation.

[0082] S203: According to the service access frequency sequence and the first first-order cumulative addition sequence, use the least squares method to obtain the variable constants in the differential equation; that is, use the least squares method to obtain the values of the variable constants in the differential equation. The mathematical expression for obtaining the variable constants in the differential equation is:

[0083]

[0084] wherein,

[0085]

[0086] ,

[0087] From the above mathematical expressions, the values of a and b in the differential equation can be obtained.

[0088] S204: According to the variable constant and the differential equation, obtain a second-order cumulative sequence; that is, substitute the values of the variable constants a and b into the above differential equation to obtain a second-order cumulative sequence.

[0089] S205: Use the second-order cumulative sequence to perform access frequency prediction and obtain an access frequency prediction sequence; that is, use the second-order cumulative sequence to obtain an MGM model, and the mathematical expression of the MGM model is:

[0090]

[0091] where is the predicted access frequency value at the +1 moment, .

[0092] Use the MGM model to perform access frequency prediction and obtain an access frequency prediction sequence. The mathematical expression of the access frequency prediction sequence is: , where is the access frequency prediction sequence, is the predicted access frequency value at the nth moment.

[0093] S206: According to the service access frequency sequence and the access frequency prediction sequence, establish an access frequency prediction model. By establishing an access frequency prediction model, the prediction of the access frequency of microservices can be realized, and the degree of automation is relatively high.

[0094] In some embodiments, the steps of establishing an access frequency prediction model according to the service access frequency sequence and the access frequency prediction sequence include:

[0095] S2061: Obtain the difference between the access frequency prediction sequence and the service access frequency sequence to obtain a difference sequence; that is, subtract the access frequency prediction sequence predicted by the MGM model from the actually collected service access frequency sequence to obtain a difference sequence.

[0096] S2062: Represent the difference sequence with a preset autoregressive integrated moving average model ARIMA to obtain an autoregressive equation and a moving average equation; in the autoregressive integrated moving average model ARIMA(p, d, q), AR represents autoregression, I represents differencing, MA represents moving average, p is the number of autoregressive terms, q is the number of moving average terms, and d is the number of differences made when the time series becomes stationary.

[0097] The mathematical expression of the autoregressive equation AR(p) of order p is:

[0098]

[0099] where, represents the difference sequence , is a white noise sequence, is the autoregressive term, that is, the autoregressive order, are the autoregressive parameters.

[0100] The mathematical expression of the moving average equation MA(q) of order q is:

[0101]

[0102] where q is the moving average term, that is, the moving average order, are the estimated parameters.

[0103] S2063: Obtain an autoregressive moving average model according to the autoregressive equation and the moving average equation; that is, combine the above autoregressive equation and the moving average equation to obtain an autoregressive moving average model, and the mathematical expression of the autoregressive moving average model is:

[0104]

[0105] S2064: Perform a difference operation on the difference sequence in the autoregressive moving average model to obtain the difference sequence after the difference operation; by performing a difference operation on the difference sequence in the autoregressive moving average model, it is possible to achieve smoothing of the difference sequence .

[0106] S2065: Obtain a difference prediction sequence according to the difference sequence after the difference operation; that is, obtain a difference prediction sequence according to the difference sequence after the difference operation and a preset difference prediction rule, and the mathematical expression of the difference prediction sequence is:

[0107]

[0108]

[0109] where, is the difference prediction sequence of , the d value can be obtained by the existing ADF (Augmented Dickey-Fluller) method, and the p and q values can be determined by functions in the prior art, such as the arma_order_select_ic function in the statsmodels library in Python, etc., which will not be elaborated here.

[0110] S2066: According to the access count prediction sequence and the difference prediction sequence, establish the access count prediction model. That is, subtract the corresponding difference prediction sequence from the access count prediction sequence to complete the prediction of the access count of the service. The mathematical expression of the access count prediction model is:

[0111]

[0112] Among them, is the final access count prediction value at the moment, is the access count prediction value at the moment output by the MGM model, is the access count prediction difference at the moment. By representing the difference sequence with an autoregressive integrated moving average model and predicting a new difference sequence, that is, the difference prediction sequence, it is possible to calibrate the error of the predicted access count, reduce the error, and improve the accuracy of the access count prediction.

[0113] As Figure 3 shown, the steps to determine the resource consumption prediction model according to the service access count sequence and the resource consumption sequence include:

[0114] S301: Construct a resource consumption prediction linear equation, and the resource consumption prediction equation includes correlation parameters for representing the correlation relationship between the service access count and the resource consumption;

[0115] For example: According to a preset collection period, regularly collect the resource consumption of the service container within a preset time period to obtain a resource consumption sequence. The mathematical expression of the resource consumption sequence is: , where is the resource consumption of the service container at the nth moment. Since the access count of the service is positively correlated with its resource consumption, therefore, assume that the mathematical expression of its resource consumption prediction linear equation is:

[0116]

[0117] Among them, is the collected service access count, c is the resource consumption, and are correlation parameters.

[0118] S302: Perform matrix transformation on the resource consumption prediction linear equation to obtain a first resource consumption prediction matrix equation, where the first resource consumption prediction matrix equation is the matrix representation of the resource consumption prediction linear equation; the mathematical expression of the first resource consumption prediction matrix equation is:

[0119]

[0120] where, , .

[0121] S303: Substitute the service access times sequence and the resource consumption sequence into the first resource consumption prediction matrix equation, and use the least squares method to determine the value of the correlation parameter; the mathematical expression for using the least squares method to determine the value of the correlation parameter is: , that is , and the determination of the value of the correlation parameter is completed.

[0122] S304: Determine a second resource consumption prediction matrix equation according to the value of the correlation parameter, and complete the construction of the resource consumption prediction model, that is, substitute the value of the correlation parameter into the first resource consumption prediction matrix equation to obtain the second resource consumption prediction matrix equation, and complete the construction of the resource consumption prediction model.

[0123] As Figure 4 shown, the steps for fitting the access times prediction model and the resource consumption prediction model to obtain the container resource prediction model include:

[0124] S401: Obtain a container resource prediction model based on the access times according to the access times prediction model and the second resource consumption prediction matrix equation in the resource consumption prediction model; that is, substitute the second resource consumption prediction matrix equation into the access times prediction model to obtain a container resource prediction model based on the access times.

[0125] The mathematical expression of the container resource prediction model is:

[0126]

[0127] where, is the predicted value of the resource consumption at time and are the correlation parameters in the second resource consumption prediction matrix equation, is the predicted value of the access times at time is the access times prediction difference.

[0128] As Figure 5As shown, the steps of using the container resource prediction model to predict and adjust container resources include:

[0129] S501: Use the container resource prediction model to predict the resources consumed by containers, and obtain the predicted average resource consumption value and the predicted peak resource consumption value of the service. The average resource consumption is the average value of the resources consumed by the service within a fixed time period, and the peak resource consumption is the maximum value of the resources consumed by the service within a fixed time period;

[0130] In some embodiments, the steps of using the container resource prediction model to predict the resources consumed by containers and obtain the predicted average resource consumption value and the predicted peak resource consumption value of the service include:

[0131] According to a preset collection period, collect the average resource consumption and peak resource consumption of a certain service within a preset time period;

[0132] Input the average resource consumption and the peak resource consumption into the container resource prediction model respectively to obtain the predicted average resource consumption value and the predicted peak resource consumption value. For example: Regularly collect the average resource consumption and peak resource consumption , indicating the average resource consumption of the service from time to time, indicating the peak resource consumption of the service from time to time. Input and into the container resource prediction model respectively to obtain the predicted average resource consumption value and the predicted peak resource consumption value of the container resources at time k, that is, the predicted average resource consumption value and the predicted peak resource consumption value of the corresponding service resources.

[0133] S502: Use the predicted average resource consumption value as the resource demand quantity and the predicted peak resource consumption value as the resource limit quantity;

[0134] S503: Adjust the container resources of the service according to the resource demand quantity and the resource limit quantity.

[0135] In the kebenetes container system, the resources that need to be adjusted include the CPU and memory sizes. Distinguished by the demand quantity and the limit quantity, there are 4 dimensions that need to be adjusted, namely [CPU demand quantity, CPU limit quantity, memory demand quantity, memory limit quantity]. Therefore, the steps of using the container resource prediction model to predict and adjust container resources also include:

[0136] Use the container resource prediction model to predict the resource consumption of containers, and obtain the predicted average CPU resource consumption value, the predicted peak CPU resource consumption value, the predicted average memory resource consumption value, and the predicted peak memory resource consumption value of the service. For example: input the average CPU resource consumption, the peak CPU resource consumption, the average memory resource consumption, and the peak memory resource consumption of a certain service into the container resource prediction model respectively, and obtain the corresponding predicted average CPU resource consumption value, the predicted peak CPU resource consumption value, the predicted average memory resource consumption value, and the predicted peak memory resource consumption value.

[0137] According to the predicted average CPU resource consumption value, the predicted peak CPU resource consumption value, the predicted average memory resource consumption value, and the predicted peak memory resource consumption value, obtain the CPU demand, the CPU limit, the memory demand, and the memory limit; that is, take the predicted average CPU resource consumption value as the CPU demand, take the predicted peak CPU resource consumption value as the CPU limit, take the predicted average memory resource consumption value as the memory demand, and take the predicted peak memory resource consumption value as the memory limit.

[0138] According to the CPU demand, the CPU limit, the memory demand, and the memory limit, perform container resource adjustment. It better realizes the prediction and adjustment of the required resource amount of microservice applications or containers in each stage, actively adjusts the required resource amount of microservice applications, thereby improving resource utilization rate, with a relatively high degree of automation and a better resource adjustment effect.

[0139] Example 1:

[0140] Such as Figure 6 shown, the general microservice framework is generally divided into a business layer, a gateway layer, and internal services. Its main features are: First, separation of front and back ends; second, a unified API gateway. In this embodiment, by tracking the access times and resource consumption of a typical microservice, it is illustrated how to dynamically predict the required resource amount of microservice applications in each stage and actively adjust the required resource amount of the corresponding microservice applications or containers, thereby improving resource utilization rate at the system level.

[0141] Step 1: Through the acquisition module deployed in the API gateway of the gateway layer, regularly collect the access times of the front-end containers of microservices within a certain period of time to obtain the service access times sequence , where is the service access times sequence, and is the total access times of a certain microservice from the (n - 1)th moment to the nth moment. For example, the access times of microservices are counted every 5 minutes, etc.

[0142] Perform a first-order cumulative sum on the above service access times sequence to obtain the first first-order cumulative sum sequence. The mathematical expression of the first first-order cumulative sum sequence is:

[0143]

[0144] ,

[0145] Among them, is the first-order accumulated sequence.

[0146] Step 2: According to the service access frequency sequence, establish an access frequency prediction model, and the access frequency prediction model is a fitting model that combines a metabolic grey model (MGM) and an autoregressive integrated moving average model (ARIMA);

[0147] Specifically, establish a differential equation in white form for the first-order accumulated sequence; the mathematical expression of the differential equation is:

[0148]

[0149] Among them, are the data in the first-order accumulated sequence, t is the time sequence, and a and b are variable constants in the differential equation.

[0150] According to the service access frequency sequence and the first-order accumulated sequence, use the least squares method to obtain the variable constants in the differential equation; that is, use the least squares method to obtain the values of the variable constants in the differential equation. The mathematical expression for obtaining the variable constants in the differential equation is:

[0151]

[0152] Among them,

[0153]

[0154] ,

[0155] From the above mathematical expression, obtain the values of a and b in the differential equation.

[0156] According to the variable constants and the differential equation, obtain the second-order accumulated sequence; that is, substitute the values of the variable constants a and b into the above differential equation to obtain the second-order accumulated sequence; use the second-order accumulated sequence to predict the access frequency and obtain the access frequency prediction sequence; that is, use the second-order accumulated sequence to obtain the MGM model, and the mathematical expression of the MGM model is:

[0157]

[0158] Among them, is the predicted value of the access times at the +1 moment, .

[0159] Using the MGM model, perform access times prediction to obtain an access times prediction sequence. The mathematical expression of the access times prediction sequence is: , where is the access times prediction sequence, is the predicted value of the access times at the nth moment.

[0160] Obtain the difference between the access times prediction sequence and the service access times sequence to obtain a difference sequence; that is, subtract the access times prediction sequence predicted by the MGM model from the service access times sequence collected truly to obtain a difference sequence; represent the difference sequence with a preset autoregressive integrated moving average model ARIMA to obtain an autoregressive equation and a moving average equation; in the autoregressive integrated moving average model ARIMA(p, d, q), AR represents autoregression, I represents differencing, MA represents moving average, p is the autoregressive term, q is the moving average term, and d is the number of differencing times when the time series becomes stationary.

[0161] The mathematical expression of the pth-order autoregressive equation AR(p) is:

[0162]

[0163] where represents the difference sequence , is a white noise sequence, is the autoregressive term, that is, the autoregressive order, is the autoregressive parameter.

[0164] The mathematical expression of the qth-order moving average equation MA(q) is:

[0165]

[0166] where q is the moving average term, that is, the moving average order, is the estimated parameter.

[0167] According to the autoregressive equation and the moving average equation, obtain an autoregressive moving average model; that is, combine the above autoregressive equation and moving average equation to obtain an autoregressive moving average model. The mathematical expression of the autoregressive moving average model is:

[0168]

[0169] Differentiate the difference sequence in the autoregressive moving average model to obtain the difference sequence after differentiation processing; according to the difference sequence after differentiation processing, obtain a difference prediction sequence; that is, according to the difference sequence after differentiation processing and a preset difference prediction rule, obtain a difference prediction sequence. The mathematical expression of the difference prediction sequence is:

[0170]

[0171]

[0172] wherein, is the difference prediction sequence of , the d value can be obtained by the existing ADF (Augmented Dickey-Fluller) method, and the p and q values can be determined by functions in the prior art, such as the arma_order_select_ic function in the statsmodels library in Python, etc., which will not be elaborated here.

[0173] According to the access count prediction sequence and the difference prediction sequence, establish the access count prediction model. That is, subtract the corresponding difference prediction sequence from the access count prediction sequence to complete the prediction of the access count of the microservice. The mathematical expression of the access count prediction model is:

[0174]

[0175] wherein, is the final access count prediction value at the th moment, is the access count prediction value output by the MGM model at the th moment, is the access count prediction difference at the th moment.

[0176] Step 3: According to the collection period, collect the resource consumption of the microservice container to obtain a resource consumption sequence; according to the service access count sequence and the resource consumption sequence, determine a resource consumption prediction model;

[0177] Specifically, regularly collect the resource consumption of each microservice container within a certain period of time to obtain a resource consumption sequence , wherein, is the resource consumption of the microservice container at the nth moment. Since the access count of the microservice is positively correlated with its resource consumption, therefore, assume that the mathematical expression of its resource consumption prediction linear equation is:

[0178]

[0179] Among them, is the number of service accesses collected, c is the resource consumption, and are correlation parameters.

[0180] Perform matrix transformation on the resource consumption prediction linear equation to obtain the first resource consumption prediction matrix equation, and the first resource consumption prediction matrix equation is the matrix representation of the resource consumption prediction linear equation; the mathematical expression of the first resource consumption prediction matrix equation is:

[0181]

[0182] Among them, ,

[0183] Substitute the service access number sequence and the resource consumption sequence into the first resource consumption prediction matrix equation, and use the least squares method to determine the values of the correlation parameters; the mathematical expression for using the least squares method to determine the values of the correlation parameters is: , that is , and the determination of the values of the correlation parameters is completed; substitute the values of the correlation parameters into the first resource consumption prediction matrix equation to obtain the second resource consumption prediction matrix equation, and complete the construction of the resource consumption prediction model.

[0184] Step 3: Fit the access number prediction model and the resource consumption prediction model to obtain a container resource prediction model; use the container resource prediction model to perform container resource prediction and adjustment.

[0185] Specifically, substitute the second resource consumption prediction matrix equation into the access number prediction model to obtain a container resource prediction model based on the access number, and the mathematical expression of the container resource prediction model is:

[0186]

[0187] Among them, is the predicted value of the resource consumption at time and are the correlation parameters in the second resource consumption prediction matrix equation, is the predicted value of the access number at time is the access number prediction difference.

[0188] Predict the resource consumption of containers using the container resource prediction model to obtain the predicted average resource consumption value and the predicted peak resource consumption value of the microservice. The average resource consumption is the average value of the resources consumed by the microservice within a fixed time period, and the peak resource consumption is the maximum value of the resources consumed by the microservice within a fixed time period;

[0189] In some embodiments, the steps of predicting the resource consumption of containers using the container resource prediction model to obtain the predicted average resource consumption value and the predicted peak resource consumption value of the microservice include:

[0190] Collect the average resource consumption and the peak resource consumption of a certain microservice within a preset time period according to the preset collection period;

[0191] Input the average resource consumption and the peak resource consumption into the container resource prediction model respectively to obtain the predicted average resource consumption value and the predicted peak resource consumption value. For example: regularly collect the average resource consumption and the peak resource consumption , indicating the average resource consumption of the microservice from moment to moment, indicating the peak resource consumption of the microservice from moment to moment. Input and into the container resource prediction model respectively to obtain the predicted average resource consumption value and the predicted peak resource consumption value of the container resources at time k, that is, the predicted average resource consumption value and the predicted peak resource consumption value of the corresponding microservice resources.

[0192] Further, in the Kebenetes container system, the resources to be adjusted include the CPU and memory sizes, which are distinguished by the demand and limit amounts. There are a total of 4 dimensions to be adjusted, namely [CPU demand, CPU limit, memory demand, memory limit]. Therefore, the steps of using the container resource prediction model for container resource prediction and adjustment can also be: Using the container resource prediction model to predict the container consumption resources, and obtaining the average CPU resource consumption prediction value, the peak CPU resource consumption prediction value, the average memory resource consumption prediction value, and the peak memory resource consumption prediction value of the microservice; For example: Input the average CPU resource consumption, the peak CPU resource consumption, the average memory resource consumption, and the peak memory resource consumption of a certain microservice into the container resource prediction model respectively, and obtain the corresponding average CPU resource consumption prediction value, the peak CPU resource consumption prediction value, the average memory resource consumption prediction value, and the peak memory resource consumption prediction value. According to the average CPU resource consumption prediction value, the peak CPU resource consumption prediction value, the average memory resource consumption prediction value, and the peak memory resource consumption prediction value, obtain the CPU demand, the CPU limit, the memory demand, and the memory limit; That is, taking the average CPU resource consumption prediction value as the CPU demand, taking the peak CPU resource consumption prediction value as the CPU limit, taking the average memory resource consumption prediction value as the memory demand, and taking the peak memory resource consumption prediction value as the memory limit. According to the CPU demand, the CPU limit, the memory demand, and the memory limit, perform container resource adjustment to achieve dynamic adjustment of the container resources in the microservice, with relatively high accuracy and resource utilization rate.

[0193] As Figure 7 shown, this embodiment also provides a container resource adjustment system, including:

[0194] An access frequency prediction model construction module, configured to preset a collection period, collect the access frequencies of the front-end containers of the service, and obtain a service access frequency sequence; According to the service access frequency sequence, establish an access frequency prediction model;

[0195] A resource consumption prediction model construction module, configured to collect the resource consumption amounts of the containers of the service according to the collection period, and obtain a resource consumption sequence; According to the service access frequency sequence and the resource consumption sequence, determine a resource consumption prediction model;

[0196] A container resource prediction model acquisition module, configured to fit the access frequency prediction model and the resource consumption prediction model to obtain a container resource prediction model;

[0197] A resource adjustment module is used to predict and adjust container resources by using the container resource prediction model; the access frequency prediction model construction module, the resource consumption prediction model construction module, the container resource prediction model acquisition module, and the resource adjustment module are connected. In the container resource adjustment system of this embodiment, by collecting the access frequencies of the front-end containers in the service according to a preset collection period, obtaining a service access frequency sequence, establishing an access frequency prediction model based on the service access frequency sequence, collecting the resource consumption amounts of the containers in the service according to the collection period, obtaining a resource consumption sequence, determining a resource consumption prediction model based on the service access frequency sequence and the resource consumption sequence, and fitting the access frequency prediction model and the resource consumption prediction model to obtain a container resource prediction model, and then using the container resource prediction model to predict and adjust container resources, the organic combination of access frequencies and resource requirements is realized. At the same time, the dynamic adjustment of the resource amount of the microservice application is realized, the resource utilization rate is effectively improved, the cost is relatively low, and the degree of automation is relatively high.

[0198] In some embodiments, the service is a microservice. The steps of establishing an access frequency prediction model according to the service access frequency sequence include:

[0199] Performing first-order cumulative addition on the service access frequency sequence to obtain a first first-order cumulative addition sequence;

[0200] Establishing a differential equation in white form for the first first-order cumulative addition sequence;

[0201] According to the service access frequency sequence and the first first-order cumulative addition sequence, using the least squares method to obtain the variable constants in the differential equation;

[0202] According to the variable constants and the differential equation, obtaining a second first-order cumulative addition sequence;

[0203] Using the second first-order cumulative addition sequence to perform access frequency prediction to obtain an access frequency prediction sequence;

[0204] According to the service access frequency sequence and the access frequency prediction sequence, establishing an access frequency prediction model.

[0205] In some embodiments, the steps of establishing an access frequency prediction model according to the service access frequency sequence and the access frequency prediction sequence include:

[0206] Obtaining the difference between the access frequency prediction sequence and the service access frequency sequence to obtain a difference sequence;

[0207] Representing the difference sequence by using a preset autoregressive integrated moving average model to obtain an autoregressive equation and a moving average equation;

[0208] Obtain an autoregressive moving average model according to the autoregressive equation and the moving average equation;

[0209] Perform differencing processing on the difference sequence in the autoregressive moving average model to obtain the differenced difference sequence;

[0210] Obtain a difference prediction sequence according to the differenced difference sequence;

[0211] Complete the establishment of the access frequency prediction model according to the access frequency prediction sequence and the difference prediction sequence.

[0212] In some embodiments, the steps of determining a resource consumption prediction model according to the service access frequency sequence and the resource consumption sequence include:

[0213] Construct a resource consumption prediction linear equation, where the resource consumption prediction equation includes an association parameter for representing the association relationship between the service access frequency and the resource consumption;

[0214] Perform matrix transformation on the resource consumption prediction linear equation to obtain a first resource consumption prediction matrix equation, where the first resource consumption prediction matrix equation is the matrix representation of the resource consumption prediction linear equation;

[0215] Substitute the service access frequency sequence and the resource consumption sequence into the first resource consumption prediction matrix equation, and use the least squares method to determine the value of the association parameter;

[0216] Determine a second resource consumption prediction matrix equation according to the value of the association parameter to complete the construction of the resource consumption prediction model.

[0217] In some embodiments, the steps of fitting the access frequency prediction model and the resource consumption prediction model to obtain a container resource prediction model include:

[0218] Obtain a container resource prediction model based on the access frequency according to the access frequency prediction model and the second resource consumption prediction matrix equation in the resource consumption prediction model;

[0219] The mathematical expression of the container resource prediction model is:

[0220]

[0221] Among them, is the predicted value of the resource consumption at time and are the association parameters in the second resource consumption prediction matrix equation, is the predicted value of the access frequency at time It is the predicted difference in access times.

[0222] In some embodiments, the steps of using the container resource prediction model to perform container resource prediction and adjustment include:

[0223] Using the container resource prediction model to predict the resources consumed by the container, obtaining the predicted average resource consumption value and the predicted peak resource consumption value of the service, where the average resource consumption is the average value of the resources consumed by the service within a fixed time period, and the peak resource consumption is the maximum value of the resources consumed by the service within a fixed time period;

[0224] Taking the predicted average resource consumption value as the resource demand and taking the predicted peak resource consumption value as the resource limit;

[0225] Adjusting the container resources of the service according to the resource demand and the resource limit.

[0226] In some embodiments, the steps of using the container resource prediction model to perform container resource prediction and adjustment further include:

[0227] Using the container resource prediction model to predict the resources consumed by the container, obtaining the predicted average CPU resource consumption value, the predicted peak CPU resource consumption value, the predicted average memory resource consumption value, and the predicted peak memory resource consumption value of the service;

[0228] Obtaining the CPU demand, the CPU limit, the memory demand, and the memory limit according to the predicted average CPU resource consumption value, the predicted peak CPU resource consumption value, the predicted average memory resource consumption value, and the predicted peak memory resource consumption value;

[0229] Adjusting the container resources according to the CPU demand, the CPU limit, the memory demand, and the memory limit.

[0230] This embodiment also provides a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, it implements any one of the methods in this embodiment.

[0231] This embodiment also provides an electronic terminal, including: a processor and a memory;

[0232] The memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory so that the terminal executes any one of the methods in this embodiment.

[0233] For the computer-readable storage medium in this embodiment, those of ordinary skill in the art can understand that all or part of the steps for implementing the above method embodiments can be completed by hardware related to a computer program. The foregoing computer program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps including those of the above method embodiments; and the foregoing storage medium includes various media that can store program codes, such as ROM, RAM, magnetic disks, or optical discs.

[0234] The electronic terminal provided in this embodiment includes a processor, a memory, a transceiver, and a communication interface. The memory and the communication interface are connected to the processor and the transceiver and complete communication therebetween. The memory is used to store a computer program, the communication interface is used for communication, and the processor and the transceiver are used to run the computer program so that the electronic terminal executes each step of the above method.

[0235] In this embodiment, the memory may include a random access memory (Random Access Memory, abbreviated as RAM), and may also include a non-volatile memory, such as at least one disk memory.

[0236] The above-mentioned processor may be a general-purpose processor, including a central processing unit (Central Processing Unit, abbreviated as CPU), a network processor (Network Processor, abbreviated as NP), etc.; it may also be a digital signal processor (Digital Signal Processing, abbreviated as DSP), an application specific integrated circuit (Application Specific Integrated Circuit, abbreviated as ASIC), a field-programmable gate array (Field-Programmable Gate Array, abbreviated as FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0237] The above embodiments merely illustrate the principles and effects of the present invention, rather than limiting the present invention. Any person familiar with this technology can modify or change the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or changes completed by those with ordinary knowledge in the technical field without departing from the spirit and technical idea disclosed by the present invention should still be covered by the claims of the present invention.

Claims

1. A method for adjusting container resources, characterized in that, it includes: Preset a collection period, collect the access times of the front-end containers of the service, and obtain a service access times sequence; According to the service access times sequence, establish an access times prediction model; According to the collection period, collect the resource consumption of the containers of the service, and obtain a resource consumption sequence; According to the service access times sequence and the resource consumption sequence, determine a resource consumption prediction model; Fit the access times prediction model and the resource consumption prediction model to obtain a container resource prediction model; Use the container resource prediction model to perform container resource prediction and adjustment; The step of determining a resource consumption prediction model according to the service access times sequence and the resource consumption sequence includes: Construct a resource consumption prediction linear equation, and the resource consumption prediction equation includes an association parameter for representing the association relationship between the service access times and the resource consumption; Perform matrix transformation on the resource consumption prediction linear equation to obtain a first resource consumption prediction matrix equation, and the first resource consumption prediction matrix equation is the matrix representation of the resource consumption prediction linear equation; Substitute the service access times sequence and the resource consumption sequence into the first resource consumption prediction matrix equation, and use the least squares method to determine the value of the association parameter; According to the value of the association parameter, determine a second resource consumption prediction matrix equation to complete the construction of the resource consumption prediction model; The step of fitting the access times prediction model and the resource consumption prediction model to obtain a container resource prediction model includes: According to the access times prediction model and the second resource consumption prediction matrix equation in the resource consumption prediction model, obtain a container resource prediction model based on the access times; The mathematical expression of the container resource prediction model is: Among them, is the predicted value of resource consumption at time and is the correlation parameter in the second resource consumption prediction matrix equation, is the predicted value of the access times at time is the predicted difference in access times.

2. The container resource adjustment method according to claim 1, characterized in that, the service is a microservice, and the step of establishing an access times prediction model according to the service access times sequence includes: Perform first-order cumulative summation on the service access times sequence to obtain a first first-order cumulative summation sequence; Establish a differential equation in white form for the first first-order cumulative summation sequence; According to the service access times sequence and the first first-order cumulative summation sequence, use the least squares method to obtain the variable constant in the differential equation; According to the variable constant and the differential equation, obtain a second first-order cumulative summation sequence; Use the second first-order cumulative summation sequence to perform access times prediction to obtain an access times prediction sequence; According to the service access times sequence and the access times prediction sequence, establish an access times prediction model.

3. The container resource adjustment method according to claim 2, characterized in that, the step of establishing an access times prediction model according to the service access times sequence and the access times prediction sequence includes: Obtain the difference between the access times prediction sequence and the service access times sequence to obtain a difference sequence; Represent the difference sequence with a preset autoregressive integrated moving average model to obtain an autoregressive equation and a moving average equation; According to the autoregressive equation and the moving average equation, obtain an autoregressive integrated moving average model; Perform differencing processing on the difference sequence in the autoregressive moving average model to obtain the differenced difference sequence; Obtain a difference prediction sequence according to the differenced difference sequence; Complete the establishment of the access count prediction model according to the access count prediction sequence and the difference prediction sequence.

4. The container resource adjustment method according to claim 1, wherein, The steps of using the container resource prediction model for container resource prediction and adjustment include: Using the container resource prediction model to predict the resource consumption of the container, obtaining the average resource consumption prediction value and the peak resource consumption prediction value of the service, where the average resource consumption is the average value of the resources consumed by the service within a fixed time period, and the peak resource consumption is the maximum value of the resources consumed by the service within a fixed time period; Taking the average resource consumption prediction value as the resource demand and taking the peak resource consumption prediction value as the resource limit; Adjust the container resources of the service according to the resource demand and the resource limit.

5. The container resource adjustment method according to claim 1, wherein, The steps of using the container resource prediction model for container resource prediction and adjustment further include: Using the container resource prediction model to predict the resource consumption of the container, obtaining the average CPU resource consumption prediction value, the peak CPU resource consumption prediction value, the average memory resource consumption prediction value, and the peak memory resource consumption prediction value of the service; Obtaining the CPU demand, the CPU limit, the memory demand, and the memory limit according to the average CPU resource consumption prediction value, the peak CPU resource consumption prediction value, the average memory resource consumption prediction value, and the peak memory resource consumption prediction value; Adjust the container resources according to the CPU demand, the CPU limit, the memory demand, and the memory limit.

6. A container resource adjustment system, wherein, It includes: An access count prediction model construction module, configured to preset a collection period, collect the access counts of the front-end containers of the service, and obtain a service access count sequence; establish an access count prediction model according to the service access count sequence; A resource consumption prediction model construction module, configured to collect the resource consumption of the containers of the service according to the collection period, and obtain a resource consumption sequence; determine a resource consumption prediction model according to the service access count sequence and the resource consumption sequence; A container resource prediction model acquisition module, configured to fit the access count prediction model and the resource consumption prediction model to obtain a container resource prediction model; A resource adjustment module, configured to use the container resource prediction model for container resource prediction and adjustment; The resource consumption prediction model construction module is specifically configured to construct a resource consumption prediction linear equation, and the resource consumption prediction equation includes an association parameter for representing the association relationship between the service access count and the resource consumption; Perform matrix transformation on the resource consumption prediction linear equation to obtain a first resource consumption prediction matrix equation, and the first resource consumption prediction matrix equation is the matrix representation of the resource consumption prediction linear equation; Substitute the service access frequency sequence and the resource consumption sequence into the first resource consumption prediction matrix equation, and use the least squares method to determine the values of the correlation parameters; Determine the second resource consumption prediction matrix equation according to the values of the correlation parameters, and complete the construction of the resource consumption prediction model; The container resource prediction model acquisition module is specifically configured to obtain a container resource prediction model based on the access frequency according to the access frequency prediction model and the second resource consumption prediction matrix equation in the resource consumption prediction model; The mathematical expression of the container resource prediction model is: Among them, is the predicted value of resource consumption at the moment, and is the correlation parameter in the second resource consumption prediction matrix equation, is the predicted value of the access times at the moment, is the predicted difference in access times.

7. A computer-readable storage medium, on which a computer program is stored, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

8. An electronic terminal, characterized in that, comprising: a processor and a memory; The memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory, so that the terminal executes the method according to any one of claims 1 to 5.

Images