A container adjustment method and apparatus, and a computer-readable storage medium
By acquiring container resource utilization and historical data, and using predictive models to optimize container resource allocation, the problem of low container resource utilization in existing technologies is solved, achieving more efficient resource utilization.
Patent Information
- Application Number
- CN202210892379.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-07-27
- Publication Date
- 2026-11-17
- Estimated Expiration
- 2042-07-27
AI Technical Summary
Existing container scheduling methods result in inefficient and unreasonable resource allocation, low container resource utilization, and an inability to effectively cope with peak business pressure.
By acquiring container resource utilization and historical data, a predictive model is used to predict the actual number of containers. Combined with preset rules and real-time container count, scaling down is performed to optimize container resource allocation.
It improves the utilization rate of container resources, optimizes the allocation of container resources, and enhances the utilization efficiency of container resources.
Smart Images

Figure CN115220872B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of data applications, and in particular to a container adjustment method and apparatus, and a computer-readable storage medium. Background Technology
[0002] With the rapid development of business, technical indicators such as concurrency and complexity have been constantly increasing to meet the pressure of peak business. The most direct and effective method is to invest more resources, that is, to rapidly increase the number of containers. The launch of the Kubernetes management platform solved the problem of application containerization. However, the container scheduling method is relatively simple. It can only screen and schedule based on the remaining physical machine resources, the default policy of the native scheduler, the extension and configuration of the kube-scheduler scheduler, and whether the real-time resource usage scheduling and SLA scheduling policies meet the requirements. This results in unreasonable and inefficient allocation of container resources and low utilization of container resources. Summary of the Invention
[0003] This invention provides a container adjustment method and apparatus, and a computer-readable storage medium, which can improve the utilization rate of container resources.
[0004] The technical solution of this invention is implemented as follows:
[0005] This invention provides a container adjustment method, the method comprising: acquiring a first resource utilization rate corresponding to each of a plurality of containers; wherein the plurality of containers exist in at least one data center; the containers in the plurality of containers are used for virtual resources running on a server; determining a first target container from the plurality of containers whose first resource utilization rate is less than a first preset threshold; and acquiring a target application corresponding to the first target container; acquiring a second resource utilization rate corresponding to each target application; filtering by the second resource utilization rate and a preset container setting rule to determine a second target container; wherein the preset container setting rule represents a preset number of containers containing the same application in a data center; acquiring first historical resource data of the target application; and predicting the actual number of containers based on the first historical resource data using a preset prediction model; wherein the preset prediction model represents a method for predicting future resource data based on historical resource data; and scaling down the second target container based on the actual number of containers and the acquired real-time number of containers.
[0006] This invention provides a container adjustment device, which includes an acquisition unit, a determination unit, a prediction unit, and a reduction unit; wherein,
[0007] The acquisition unit is configured to acquire the first resource utilization rate corresponding to each of the multiple containers; wherein the multiple containers exist in at least one data center; the containers in the multiple containers are used for virtual resources running on the server; from the multiple containers, a first target container with the first resource utilization rate being less than a first preset threshold is determined; and the target application corresponding to the first target container is acquired;
[0008] The determining unit is used to obtain the second resource utilization rate corresponding to each target application; and to filter and determine the second target container by using the second resource utilization rate and the preset container setting rules; wherein, the preset container setting rules represent the minimum preset number of containers containing the same application in a data center;
[0009] The prediction unit is used to acquire first historical resource data of the target application; and based on the first historical resource data, predict the actual number of containers using a preset prediction model; wherein, the preset prediction model is used to predict future resource data based on historical resource data;
[0010] The shrinking unit is used to shrink the second target container according to the actual number of containers and the acquired real-time number of containers.
[0011] This invention provides a container adjustment device, the container adjustment device comprising:
[0012] Memory, used to store executable instructions;
[0013] A processor is configured to execute executable instructions stored in the memory, and when the executable instructions are executed, the processor executes the container adjustment method.
[0014] This invention provides a computer-readable storage medium, characterized in that it stores executable instructions, which, when executed by one or more processors, cause the processors to perform the container adjustment method.
[0015] This invention provides a container adjustment method and apparatus, and a computer-readable storage medium. The method includes: acquiring a first resource utilization rate corresponding to each of a plurality of containers; wherein the plurality of containers exist in at least one data center; the containers in the plurality of containers are used for virtual resources running on a server; determining a first target container from the plurality of containers whose first resource utilization rate is less than a first preset threshold; and acquiring a target application corresponding to the first target container; acquiring a second resource utilization rate corresponding to each target application; filtering by the second resource utilization rate and a preset container setting rule to determine a second target container; wherein the preset container setting rule represents a preset number of containers containing the same application in a data center; acquiring first historical resource data of the target application; and predicting the actual number of containers based on the first historical resource data using a preset prediction model; wherein the preset prediction model represents a method for predicting future resource data based on historical resource data; and scaling down the second target container based on the actual number of containers and the acquired real-time number of containers. In the above scheme, the server can obtain the first resource utilization rate corresponding to each of multiple containers, and determine the first target container based on the first resource utilization rate; a range of containers to be scaled down can be obtained; based on the second resource utilization rate corresponding to the target application of the first target container; the second target container is determined by filtering based on the second resource utilization rate and preset container setting rules; the range of containers to be scaled down can be narrowed, improving the accuracy and efficiency of container scaling down; based on the first historical resource data of the target application, the actual number of containers is predicted through a preset prediction model; based on the actual number of containers and the obtained real-time number of containers, the second target container is scaled down, which can reduce containers with low resource utilization, thereby improving the overall container resource utilization rate. Attached Figure Description
[0016] Figure 1 This invention provides an optional flowchart of a container adjustment method according to an embodiment of the present invention. Figure 1 ;
[0017] Figure 2 This invention provides an optional flowchart of a container adjustment method according to an embodiment of the present invention. Figure 2 ;
[0018] Figure 3 This is an optional clustering result diagram for a container adjustment method provided in an embodiment of the present invention;
[0019] Figure 4 This invention provides an optional flowchart of a container adjustment method according to an embodiment of the present invention. Figure 3 ;
[0020] Figure 5This invention provides an optional flowchart of a container adjustment method according to an embodiment of the present invention. Figure 4 ;
[0021] Figure 6 This invention provides an optional flowchart of a container adjustment method according to an embodiment of the present invention. Figure 5 ;
[0022] Figure 7 This invention provides an optional flowchart of a container adjustment method according to an embodiment of the present invention. Figure 6 ;
[0023] Figure 8 A schematic diagram of the structure of a container adjustment device provided in an embodiment of the present invention. Figure 1 ;
[0024] Figure 9 A schematic diagram of the structure of a container adjustment device provided in an embodiment of the present invention. Figure 2 . Detailed Implementation
[0025] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the protection scope of the present invention.
[0026] To enable those skilled in the art to better understand the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. Figure 1 This is an optional flowchart illustrating a container adjustment method provided by an embodiment of the present invention. Figure 1 , will combine Figure 1 The steps shown are explained.
[0027] S101. Obtain the first resource utilization rate for each of the multiple containers.
[0028] In some embodiments of the present invention, multiple containers exist in at least one data center; the containers within the multiple containers are used for virtual resources running on a server.
[0029] In some embodiments of the present invention, the server can obtain second historical resource data corresponding to each of multiple containers; and determine the first resource utilization rate corresponding to each of the multiple containers by clustering the second historical resource data.
[0030] In some embodiments of the present invention Figure 2 This is an optional flowchart illustrating a container adjustment method provided by an embodiment of the present invention. Figure 2 ,like Figure 2 As shown, S101 can be implemented through S1011 and S1012, as follows:
[0031] S1011. Obtain the second historical resource data corresponding to each of the multiple containers.
[0032] In some embodiments of the present invention, the second historical resource data includes the second historical processor utilization rate and the second historical memory utilization rate.
[0033] S1012. By clustering the second historical resource data, determine the first resource utilization rate corresponding to each of the multiple containers.
[0034] In some embodiments of the present invention, the server can determine the cluster center of the containers by clustering the containers corresponding to the second historical processor utilization rate and the second historical memory utilization rate; and determine the first resource utilization rate corresponding to multiple containers based on the distribution of the cluster center, the second historical processor utilization rate and the second historical memory utilization rate.
[0035] Understandably, the server can obtain the second historical resource data corresponding to each of the multiple containers; by clustering the second historical resource data, the first resource utilization rate corresponding to each of the multiple containers can be determined; by determining the first resource utilization rate corresponding to each of the multiple containers, the resource utilization rate of each container can be analyzed, which facilitates the subsequent screening of containers.
[0036] In some embodiments of the present invention, S1012 can be implemented by S10121 and S10122, as follows:
[0037] S10121. Cluster the containers corresponding to the second historical processor utilization rate and the second historical memory utilization rate to determine the cluster center of the containers.
[0038] In some embodiments of the present invention, the server can cluster containers based on the second historical processor usage rate and the second historical memory usage rate corresponding to each container using a clustering algorithm to determine the cluster center of the containers.
[0039] It should be noted that the clustering algorithm can be K-means clustering algorithm, and the embodiments of the present invention do not limit this.
[0040] S10122. Based on the distribution of the cluster center, the second historical processor utilization rate, and the second historical memory utilization rate, determine the first resource utilization rate corresponding to multiple containers.
[0041] In some embodiments of the present invention, the server can compare and analyze the second historical processor utilization rate and the second historical memory utilization rate corresponding to the container with preset thresholds based on the distribution of the cluster center, the second historical processor utilization rate, and the second historical memory utilization rate, to determine the first resource utilization rate corresponding to multiple containers. Here, the preset threshold is set to 15.
[0042] For example, the second historical processor utilization rate (equivalent to CPU utilization) and the second historical memory utilization rate (equivalent to memory utilization). In the initial stage of clustering, based on the feature category division, there are four possible scenarios: high CPU utilization, high memory utilization; low CPU utilization, high memory utilization; high CPU utilization, low memory utilization; and low CPU utilization, low memory utilization. Therefore, in the clustering process, the K-means algorithm is used with the number of categories k=4, and Euclidean distance is used as the metric to cluster the container resource utilization, ultimately resulting in... Figure 3 The results shown; from Figure 3 The clustering results show that in part A, both CPU and memory usage are less than 15%. Overall, part A exhibits low CPU and low memory usage. Parts B and C show CPU usage less than 15% and memory usage greater than 15%. Overall, parts B and C exhibit low CPU and high memory usage. Part D shows both CPU and memory usage greater than 15%. Overall, part D exhibits high CPU and high memory usage. Based on the cluster centers and resource usage, group A is marked as having abnormal CPU and memory usage (i.e., low primary resource usage). Group B is marked as having abnormal CPU usage (i.e., low primary resource usage). Groups C and D can be considered to have normal usage (since the memory usage of part C is much higher than that of part B, part C is determined to be a storage-type container) and are marked as normal (i.e., high primary resource usage). Output the coordinates of the center of each cluster, mapping the cluster center coordinates to the four labels mentioned above, thus fixing the clustering as a classification problem. Classify the containers on the real-time line and output the classification labels. Simultaneously, score the containers based on their resource utilization, obtaining a container score. The container score is obtained by feeding the CPU and memory utilization of all applications on the container into the k-means clustering algorithm, resulting in (0,0), (0,1), (1,0), and (1,1) categories. These categories are then fed into the Isolation Forest algorithm to calculate the resource utilization anomalies of each application, converting them into scores from 0 to 100, ensuring that applications belonging to category (0,0) have the highest resource utilization score and applications belonging to category (1,1) have the lowest.
[0043] Understandably, the server clusters the containers corresponding to the second historical processor utilization rate and the second historical memory utilization rate to determine the cluster center of the containers; based on the cluster center, the second historical processor utilization rate, and the second historical memory utilization rate, the first resource utilization rate corresponding to multiple containers is determined; this can ensure the objectivity and accuracy of the first resource utilization rate.
[0044] S102. From multiple containers, determine the first target container whose first resource utilization rate is less than the first preset threshold; and obtain the target application corresponding to the first target container.
[0045] In some embodiments of the present invention, the server can filter out a first target resource utilization rate that is less than a first preset threshold from multiple containers based on the first resource utilization rates corresponding to each container; determine a first target container based on the first target resource utilization rate; and obtain the target application corresponding to the first target container based on the first target container.
[0046] For example, from the four types of containers ABCD, type A is marked as having abnormal CPU and memory usage, that is, the first resource utilization rate of type A is low; type B is marked as having abnormal CPU usage, that is, the first resource utilization rate of type B is low; types C and D can be considered to be in normal use and are marked as normal, that is, the first resource utilization rate of types C and D is high; since the first resource utilization rate of types AB is low, types AB are used as the first target container; and the application corresponding to the first table container is obtained.
[0047] It should be noted that multiple applications will be deployed on the container, so there is a relationship between the container and the application; the first target container includes multiple containers with a first resource utilization rate of less than a first preset threshold.
[0048] S103. Obtain the second resource utilization rate corresponding to each target application; filter by the second resource utilization rate and preset container setting rules to determine the second target container.
[0049] In some embodiments of the present invention, the preset container setting rule represents the minimum preset number of containers containing the same application in a data center. The preset container setting rule states that since containers are deployed in data centers, each data center must ensure at least n containers, and the minimum resource allocation of the data center must be guaranteed during scaling down.
[0050] In some embodiments of the present invention, the server can obtain the third processor utilization rate, third memory utilization rate, and timing fluctuation characteristics corresponding to each target application; and calculate the second resource utilization rate corresponding to each target application based on the third processor utilization rate, third memory utilization rate, and timing fluctuation characteristics. From multiple target applications, a first application with a second resource utilization rate greater than a second preset threshold is determined; from the first target container, containers corresponding to the first application, newly launched applications, and periodically changing applications are removed to determine the second initial target container; the data center information corresponding to the second initial target container is obtained; and the second initial target container is filtered using preset container setting rules and data center information to determine the second target container.
[0051] In some embodiments of the present invention Figure 4 This is an optional flowchart illustrating a container adjustment method provided by an embodiment of the present invention. Figure 3 ,like Figure 4 As shown, obtaining the second resource utilization rate corresponding to each target application in S103 can be achieved through S201 and S202, as follows:
[0052] S201. Obtain the third processor utilization rate, third memory utilization rate, and timing fluctuation characteristics for each target application.
[0053] In some embodiments of the present invention, the server can obtain the third processor utilization rate, third memory utilization rate, and timing fluctuation characteristics corresponding to each target application for each first target container.
[0054] S202. Calculate the second resource utilization rate for each target application based on the third processor utilization rate, the third memory utilization rate, and timing fluctuation characteristics.
[0055] In some embodiments of the present invention, the server can calculate the second resource utilization rate for each target application based on the third processor utilization rate, third memory utilization rate, and timing fluctuation characteristics corresponding to each target application, thereby obtaining the second resource utilization rate for each target application. Simultaneously, an application score is determined based on the second resource utilization rate for each target application.
[0056] Understandably, the server obtains the third processor utilization rate, third memory utilization rate, and timing fluctuation characteristics for each target application; based on the third processor utilization rate, third memory utilization rate, and timing fluctuation characteristics, it calculates the second resource utilization rate for each target application; calculating the second resource utilization rate through the third processor utilization rate and third memory utilization rate can ensure the objectivity and accuracy of the second resource utilization rate.
[0057] In some embodiments of the present invention Figure 5This is an optional flowchart illustrating a container adjustment method provided by an embodiment of the present invention. Figure 4 ,like Figure 5 As shown, the process in S103, which filters based on the second resource utilization rate and preset container setting rules to determine the second target container, can be implemented through S301-S304, as follows:
[0058] S301. From multiple target applications, identify the first application whose second resource utilization rate is greater than the second preset threshold.
[0059] In some embodiments of the present invention, the first application represents an application that works normally.
[0060] In some embodiments of the present invention, the server can filter out a second target resource utilization rate that is greater than a second preset threshold from the second resource utilization rates corresponding to multiple target applications; and determine the first application corresponding to the second target resource utilization rate based on the second target resource utilization rate.
[0061] S302. Based on the first target container, remove the containers corresponding to the first application, newly launched applications, and periodically changing applications to determine the second initial target container.
[0062] In some embodiments of the present invention, a newly launched application refers to an application for which there is no historical weekly data when comparing the usage rate of multiple target applications over the same period last week; an application with periodic changes refers to an application for which the usage rate of multiple target applications is compared over the week or day to determine whether a low usage rate on a given day indicates a periodic anomaly. If the resource usage of an application shows periodic changes, then the application is considered a periodic change application.
[0063] In some embodiments of the present invention, the server can determine the containers corresponding to the first application, the newly launched application, and the periodically changing application based on the first application; and remove the containers corresponding to the first application, the newly launched application, and the periodically changing application from the first target container to obtain the second initial target container.
[0064] It should be noted that the second initial target container includes multiple containers.
[0065] S303. Obtain the data center information corresponding to the second initial target container.
[0066] In some embodiments of the present invention, the server can determine the data center information corresponding to the second initial target container based on the second initial target container.
[0067] S304. By using preset container setting rules and data center information, the second initial target container is filtered to determine the second target container.
[0068] In some embodiments of the present invention, the server can divide the second initial target container into multiple container groups based on the data center information; for each of the multiple container groups, a fixed container corresponding to each container group is determined by a preset container setting rule; and the fixed container is removed from the second initial target container to determine the second target container.
[0069] Understandably, the server can prioritize the target application with the highest resource utilization among the target applications, ensuring that the remaining target applications all have low resource utilization. It can then remove the containers corresponding to the first target application, newly launched applications, and periodically changing applications from the first target container to obtain the second initial target container. This narrows down the range of containers to be scaled up, improving the accuracy of container scaling. The server also obtains the data center information corresponding to the second initial target container. By using preset container setting rules and data center information, the server filters the second initial target container to determine the second target container. This further narrows down the range of containers to be scaled up, further improving the accuracy of container scaling.
[0070] In some embodiments of the present invention, S304 can be implemented by S3041, S3042 and S3043, as follows:
[0071] S3041. Based on the data center information, the second initial target container is divided into multiple container groups.
[0072] In some embodiments of the present invention, a container group represents a container within a computer room.
[0073] In some embodiments of the present invention, the server can divide the second initial target container according to the data center information corresponding to the second initial target container, and divide the second initial target containers located in the same data center into a container group, thereby obtaining multiple container groups.
[0074] S3042. For multiple container groups, the fixed containers corresponding to each container group are determined by the preset container setting rules.
[0075] In some embodiments of the present invention, the server can reserve a fixed number of containers in a data center (i.e., a container group) according to preset container setting rules for multiple container groups, thereby obtaining fixed containers corresponding to each of the multiple container groups.
[0076] It should be noted that fixed containers are containers that must be set up to ensure the normal operation of the computer room.
[0077] S3043. Based on the second initial target container, remove the fixed container and determine the second target container.
[0078] In some embodiments of the present invention, the server may obtain a second target container by eliminating a fixed container based on a second initial target container.
[0079] Understandably, the server divides the second initial target container into multiple container groups based on the data center information; for each container group, a fixed container is determined according to preset container setting rules; based on the second initial target container, the fixed container is removed to determine the second target container; this further narrows down the range of containers to be scaled down, thus further improving the accuracy of container scaling down.
[0080] S104. Obtain the first historical resource data of the target application; and based on the first historical resource data, predict the actual number of containers using a preset prediction model.
[0081] In some embodiments of the present invention, a preset prediction model is characterized for predicting future resource data based on historical resource data; the first historical resource data includes: historical resource data of the first month and historical resource data of the first week; the preset prediction model includes a first prediction model and a second prediction model.
[0082] In some embodiments of the present invention, the server may preprocess the first historical resource data to obtain feature data; the feature data may be judged, and if the feature data belongs to daily feature data, the feature data may be input into a first prediction model for prediction to determine the actual number of containers. Alternatively, if the feature data belongs to promotional event feature data, the feature data may be input into a second prediction model for prediction to determine the actual number of containers.
[0083] For example, the preset prediction model uses a neural network MQ-RNN, which includes two MLP branches. The first is a global MLP that aggregates the encoder output and all future inputs into two contexts: a set of horizon-specific contexts c. t+k For each of the k future points, and for capturing the horizon-agnostic context c of common information. a The second local MLP combines the context of the corresponding future input with that of the global MLP, and then outputs all the quantiles required for that particular future time step: the local MLP is used for future seasonality and to generate sharp predictions. Adding this structure to the model can improve the stability of learning and the smoothness of the generated predictions.
[0084] In some embodiments of the present invention, S104 can be implemented by S1041, S1042 and S1043, as follows:
[0085] S1041. Preprocess the historical resource data for the first month and the historical resource data for the first week to obtain monthly feature data and weekly feature data.
[0086] S1042. Input the monthly feature data into the preset prediction model for prediction and obtain the first prediction result.
[0087] In some embodiments of the present invention, the first prediction result represents the core value of the processor corresponding to the monthly data.
[0088] In some embodiments of the present invention, the server can determine the monthly feature data. If the monthly feature data belongs to daily feature data, the monthly feature data is input into the first prediction model for prediction to obtain the first prediction result; or, if the monthly feature data belongs to promotional feature data, the monthly feature data is input into the second prediction model for prediction to obtain the first prediction result.
[0089] S1043. Input the weekly feature data into the preset prediction model for prediction to obtain the second prediction result.
[0090] In some embodiments of the present invention, the second prediction result characterizes the core value of the processor corresponding to the weekly data.
[0091] In some embodiments of the present invention, the server can determine the weekly feature data. If the weekly feature data belongs to daily feature data, the weekly feature data is input into the first prediction model for prediction to obtain a second prediction result; or, if the weekly feature data belongs to promotional feature data, the weekly feature data is input into the second prediction model for prediction to obtain a second prediction result.
[0092] S1044. The first prediction result and the second prediction result are weighted and summed to obtain the final core value of the processor; and the actual number of containers is determined based on the final core value of the processor.
[0093] In some embodiments of the present invention, the server can perform a weighted summation of the first prediction result and the second prediction result to obtain the final core value of the processor; and use the preset number of the final core value as the actual number of containers.
[0094] For example, the final number of processor cores = a*y1 + b*y2, where y1 is the first prediction result, y2 is the second prediction result, and a and b are the weighting coefficients of y1 and y2, respectively, and a and b satisfy a + b = 1. The preset number is generally half of the final number of cores.
[0095] It should be noted that the values of a and b are updated during each model training (using RMSE as the loss function, i.e., obtaining the combination of a and b values that minimizes RMSE).
[0096] Understandably, the server can preprocess the historical resource data for the first month and the historical resource data for the first week to obtain monthly feature data and weekly feature data, respectively. The monthly feature data is then input into a preset prediction model for prediction to obtain a first prediction result. The weekly feature data is then input into the preset prediction model for prediction to obtain a second prediction result. The first and second prediction results are then weighted and summed to obtain the final core value of the processor. Based on the final core value of the processor, the actual number of containers can be determined. This prediction of the actual number of containers facilitates the subsequent determination of the number of containers to be scaled down, thus improving the accuracy of container scaling down.
[0097] S105. Based on the actual number of containers and the acquired real-time number of containers, reduce the size of the second target container.
[0098] In some embodiments of the present invention, the server can obtain the real-time number of containers; perform a difference operation based on the actual number of containers and the real-time number of containers to determine the number of containers to be scaled down; and delete the containers corresponding to the second target container to complete the scaling down.
[0099] Understandably, the server can obtain the first resource utilization rate corresponding to multiple containers, determine the first target container based on the first resource utilization rate, and obtain a range of containers to be scaled down. Based on the second resource utilization rate of the target application corresponding to the first target container, and using the second resource utilization rate and preset container setting rules, the server can filter and determine the second target container. This narrows down the range of containers to be scaled down, improving the accuracy and efficiency of container scaling down. Based on the first historical resource data of the target application, and through a preset prediction model, the server predicts the actual number of containers. Based on the actual number of containers and the obtained real-time number of containers, the server scales down the second target container, reducing containers with low resource utilization, thereby improving the overall container resource utilization rate.
[0100] In some embodiments of the present invention, S105 can be implemented by S1051, S1052 and S1053, as follows:
[0101] S1051, Get the real-time number of containers.
[0102] S1052. Based on the actual number of containers and the real-time number of containers, perform a difference operation to determine the number of containers to be reduced in size.
[0103] In some embodiments of the present invention, the server can perform a difference operation based on the actual number of containers and the real-time number of containers to obtain the calculation result; the calculation result is used as the number of containers to be scaled down.
[0104] S1053. Based on the second target container, delete the container corresponding to the number of containers to complete the shrinking.
[0105] In some embodiments of the present invention, the server can sum the scores of the application on the container, the container's own score, and the difference between the container's actual core value and the ideal core value to obtain the comprehensive container score corresponding to the second target container; based on the comprehensive container score, the low-scoring containers corresponding to the number of containers are deleted to complete the scaling down.
[0106] Understandably, the server can obtain the real-time container count; based on the actual container count and the real-time container count, it performs a difference operation to determine the number of containers to be scaled down; based on the second target container, it deletes the containers corresponding to the number of containers, thus completing the scaling down; this reduces the number of containers with low resource utilization and improves the overall resource utilization of the container group.
[0107] In some embodiments of the present invention, S106 and S107 are executed before S104, as follows:
[0108] S106. Obtain the third historical resource data and the number of processor cores used in the history of the first application.
[0109] In some embodiments of the present invention, the first application represents multiple containers that are other than the target application.
[0110] S107. Preprocess the third historical resource data to obtain historical feature data; and use the historical feature data to train the initial prediction model to determine the preset prediction model.
[0111] In some embodiments of the present invention, the historical feature data includes first historical feature data and second historical feature data.
[0112] In some embodiments of the present invention, the server can determine the initial prediction model as a first initial prediction model and a second initial prediction model based on different parameter values; input the first historical feature data into the initial first prediction model, perform regression function fitting training, and determine the first prediction model; input the second historical feature data into the initial second prediction model, perform regression function fitting training, and determine the second prediction model.
[0113] Understandably, the server can obtain the third historical resource data and the number of processor cores used in the first application; preprocess the third historical resource data to obtain historical feature data; and train the initial prediction model using the historical feature data to determine the preset prediction model, which can improve the accuracy of the preset prediction model.
[0114] In some embodiments of the present invention, the process of training the initial prediction model using historical feature data in step S107 to determine the preset prediction model can be implemented through steps S1071, S1072, and S1073, as follows:
[0115] S1071. Based on different parameter values, the initial prediction model is determined as the first initial prediction model and the second initial prediction model.
[0116] In some embodiments of the present invention, the server can determine the initial prediction model as a first initial prediction model and a second initial prediction model based on different parameter values.
[0117] S1072. Input the first historical feature data into the first initial prediction model, perform regression function fitting training, and determine the first prediction model.
[0118] In some embodiments of the present invention, the first historical feature data represents the historical feature data of daily capacity; the first prediction model represents the prediction used for daily capacity.
[0119] In some embodiments of the present invention, the server can input the first historical feature data into the first initial prediction model, perform regression function fitting training to obtain the first fitting function; compare the output result of the first fitting function with the first preset result until the output result of the first fitting function reaches the first preset result, stop training, and obtain the first prediction model.
[0120] S1073. Input the second historical feature data into the second initial prediction model, perform regression function fitting training, and determine the second prediction model.
[0121] In some embodiments of the present invention, the second prediction model represents the prediction for the capacity of the promotion; the second historical feature data represents the historical feature data of the capacity of the promotion.
[0122] In some embodiments of the present invention, the server can input the second historical feature data into the second initial prediction model, perform regression function fitting training to obtain the second fitting function; compare the output of the second fitting function with the second preset result until the output of the second fitting function reaches the second preset result, stop training, and obtain the second prediction model.
[0123] Understandably, the server can determine the initial prediction model as a first initial prediction model and a second initial prediction model based on different parameter values; inputting the first historical feature data into the first initial prediction model and performing regression function fitting training to determine the first prediction model can improve the accuracy of the first prediction model; inputting the second historical feature data into the second initial prediction model and performing regression function fitting training to determine the second prediction model can improve the accuracy of the second prediction model.
[0124] An optional flowchart of a container adjustment method provided in this embodiment of the invention. Figure 5 ,like Figure 6As shown, the steps for executing the container adjustment method are as follows:
[0125] S1. Obtain the processor utilization and memory utilization of each of the multiple containers.
[0126] In some embodiments of the present invention, the processor utilization and memory utilization of each of the multiple containers are respectively (equivalent to the second historical processor utilization and the second historical memory utilization).
[0127] S2. Determine the first resource utilization rate by using processor utilization rate and memory utilization rate.
[0128] S3. Filter the first resource utilization rate to determine the initial list of shrinking containers.
[0129] In some embodiments of the present invention, the containers in the initial shrinkage container list (equivalent to the first target container).
[0130] S4. The initial list of shrinkable containers is filtered by preset data center setting criteria, newly launched applications, and periodically changing applications to obtain a new list of shrinkable containers.
[0131] In some embodiments of the present invention, the container in the shrink container list (equivalent to the second target container).
[0132] In some embodiments of the present invention Figure 7 A schematic diagram of an optional process for a container adjustment method. Figure 6 ,like Figure 7 As shown, S4 can be implemented through S401-S404, as follows:
[0133] S401. Obtain the target application corresponding to the container in the initial shrinkable container list to form a shrinkable application list.
[0134] S402. Calculate the second resource utilization rate for each target application in the shrinking application list.
[0135] S403. Remove the container containing the target application with the second highest resource utilization from the initial shrink container list to obtain the intermediate shrink container list.
[0136] In some embodiments of the present invention, the containers in the intermediate shrinkage container list (equivalent to the second initial target container).
[0137] S404. Obtain the data center information of the containers in the intermediate shrinking container list, and filter the intermediate shrinking container list according to the preset data center setting rules and data center information to obtain the shrinking container list.
[0138] In some embodiments of the present invention, the container in the shrink container list (equivalent to the second target container).
[0139] S5. Obtain the historical processor usage and historical memory usage of the application corresponding to the container in the list of shrinking containers.
[0140] In some embodiments of the present invention, historical processor utilization and historical memory utilization (equivalent to first historical resource data) are used.
[0141] S6. Preprocess the historical processor usage rate and historical memory usage rate to obtain historical feature data.
[0142] S7. By using a preset prediction model, historical feature data is predicted to obtain the actual kernel value.
[0143] S8. Get the real-time number of containers.
[0144] In some embodiments of the present invention, S5 and S8 are executed after S4, and there is no particular order between them.
[0145] S9. Determine the actual number of containers based on the actual number of cores; determine the number of containers to be reduced in size based on the actual number of containers and the real-time number of containers.
[0146] S10. Perform scaling down based on the list of scaling down containers and the number of containers to be scaled down.
[0147] Understandably, the server can filter based on the first resource utilization rate to determine the initial list of containers to be scaled down; this narrows down the range of containers to be scaled down, improving the accuracy and efficiency of container scaling down; the initial list of containers to be scaled down is further filtered using preset data center setting criteria, newly launched applications, and periodically changing applications to obtain the list of containers to be scaled down; the range of containers to be scaled down is further narrowed down by using a preset prediction model to predict historical feature data to obtain the actual core value, and based on the actual core value, the actual number of containers is determined; based on the actual number of containers and the obtained real-time number of containers, the second target containers are scaled down, which can reduce containers with low resource utilization, thereby improving the overall container resource utilization rate.
[0148] Based on the container adjustment method described in the above embodiments, this invention also provides a container adjustment device, such as... Figure 8 As shown, Figure 8 A schematic diagram of the structure of a container adjustment device provided in an embodiment of the present invention. Figure 1 The container adjustment device 8 includes: an acquisition unit 801, a determination unit 802, a prediction unit 803, and a reduction unit 804;
[0149] The acquisition unit 801 is used to acquire the first resource utilization rate corresponding to each of the multiple containers; wherein the multiple containers exist in at least one data center; the containers in the multiple containers are used for virtual resources running on the server; from the multiple containers, a first target container with the first resource utilization rate being less than a first preset threshold is determined; and the target application corresponding to the first target container is acquired;
[0150] The determining unit 802 is used to obtain the second resource utilization rate corresponding to each target application; and to filter and determine the second target container by using the second resource utilization rate and the preset container setting rules; wherein, the preset container setting rules represent the minimum preset number of containers containing the same application in a data center;
[0151] The prediction unit 803 is used to acquire the first historical resource data of the target application; and based on the first historical resource data, predict the actual number of containers using a preset prediction model; wherein, the preset prediction model is used to predict future resource data based on historical resource data;
[0152] The shrinkage unit 804 is used to shrink the second target container according to the actual number of containers and the acquired real-time number of containers.
[0153] In some embodiments of the present invention, the acquisition unit 801 is further configured to acquire the second historical resource data corresponding to each of the plurality of containers;
[0154] The determining unit 802 is further configured to determine the first resource utilization rate corresponding to each of the plurality of containers by clustering the second historical resource data.
[0155] In some embodiments of the present invention, the second historical resource data includes: a second historical processor utilization rate and a second historical memory utilization rate; the determining unit 802 is further configured to cluster the containers corresponding to the second historical processor utilization rate and the second historical memory utilization rate to determine the cluster center of the containers; and determine the first resource utilization rate corresponding to the plurality of containers based on the distribution of the cluster center, the second historical processor utilization rate and the second historical memory utilization rate.
[0156] In some embodiments of the present invention, the acquisition unit 801 is further configured to acquire the third processor utilization rate, the third memory utilization rate, and the timing fluctuation characteristics corresponding to each target application; and calculate the second resource utilization rate corresponding to each target application based on the third processor utilization rate, the third memory utilization rate, and the timing fluctuation characteristics.
[0157] In some embodiments of the present invention, the determining unit 802 is further configured to determine, from multiple target applications, a first application whose second resource utilization rate is greater than a second preset threshold; wherein, the first application represents an application that is working normally; based on the first target container, the containers corresponding to the first application, newly launched applications, and periodically changing applications are removed to determine a second initial target container;
[0158] The acquisition unit 801 is also used to acquire the data center information corresponding to the second initial target container;
[0159] The determining unit 802 is further configured to filter the second initial target container and determine the second target container by means of the preset container setting rules and the data center information.
[0160] In some embodiments of the present invention, the acquisition unit 801 is further configured to divide the second initial target container according to the data center information to obtain multiple container groups; wherein, a container group represents a container in a data center;
[0161] The determining unit 802 is further configured to, for the plurality of container groups, respectively determine the fixed container corresponding to each of the plurality of container groups through the preset container setting rules; and based on the second initial target container, remove the fixed container to determine the second target container.
[0162] In some embodiments of the present invention, before predicting the actual number of containers based on the first historical resource data and a preset prediction model, the acquisition unit 801 is further configured to acquire the third historical resource data and the historical number of processor cores used by the first application; wherein, the first application represents multiple applications other than the target application corresponding to the containers;
[0163] The determining unit 802 is further configured to preprocess the third historical resource data to obtain historical feature data; and use the historical feature data to train the initial prediction model to determine the preset prediction model.
[0164] In some embodiments of the present invention, the historical feature data includes: first historical feature data and second historical feature data; the preset prediction model includes: a first prediction model and a second prediction model; the determining unit 802 is further configured to determine the initial prediction model as a first initial prediction model and a second initial prediction model according to different parameter values; input the first historical feature data into the first initial prediction model, perform regression function fitting training, and determine the first prediction model; wherein, the first historical feature data represents historical feature data of daily capacity; the first prediction model represents prediction for daily capacity; input the second historical feature data into the second initial prediction model, perform regression function fitting training, and determine the second prediction model; wherein, the second historical feature data represents historical feature data of peak sales capacity; the second prediction model represents prediction for peak sales capacity.
[0165] In some embodiments of the present invention, the first historical resource data includes: first monthly historical resource data and first weekly historical resource data; the acquisition unit 801 is further configured to preprocess the first monthly historical resource data and the first weekly historical resource data respectively to obtain monthly feature data and weekly feature data; input the monthly feature data into the preset prediction model for prediction to obtain a first prediction result; the first prediction result represents the core value of the processor corresponding to the monthly data; input the weekly feature data into the preset prediction model for prediction to obtain a second prediction result; the second prediction result represents the core value of the processor corresponding to the weekly data; perform a weighted summation of the first prediction result and the second prediction result to obtain the final core value of the processor; and determine the actual number of containers based on the final core value of the processor.
[0166] In some embodiments of the present invention, the acquisition unit 801 is further configured to acquire the real-time number of containers;
[0167] The determining unit 802 is further configured to perform a difference operation based on the actual number of containers and the real-time number of containers to determine the number of containers to be reduced in size;
[0168] The shrinking unit 804 is also used to delete containers corresponding to the number of containers based on the second target container, thereby completing the shrinking.
[0169] It should be noted that the above-described division of program modules is only used as an example when adjusting the container. In practical applications, the above processing can be assigned to different program modules as needed, that is, the internal structure of the device can be divided into different program modules to complete all or part of the processing described above. Furthermore, the container adjustment device and container adjustment method embodiments provided in the above embodiments belong to the same concept, and their specific implementation process and beneficial effects are detailed in the method embodiments, which will not be repeated here. For technical details not disclosed in this device embodiment, please refer to the description of the method embodiments of this invention for understanding.
[0170] Based on the container adjustment method described in the above embodiments, this invention also provides a container adjustment device, such as... Figure 9 As shown, Figure 9 A schematic diagram of the structure of a container adjustment device provided in an embodiment of the present invention. Figure 2 The device 9 includes a processor 901 and a memory 902; the memory 902 stores one or more programs executable by the processor, and when one or more programs are executed, the processor 901 executes any of the container adjustment methods described in the previous embodiments.
[0171] This invention also provides a computer-readable storage medium storing executable instructions that, when executed, cause a processor to perform the container adjustment method as described in this invention.
[0172] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of hardware embodiments, software embodiments, or embodiments combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage and optical storage) containing computer-usable program code.
[0173] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.
[0174] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.
[0175] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.
[0176] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention.
Claims
1. A container adjustment method characterized by, include: Obtain the first resource utilization rate corresponding to each of multiple containers; wherein, the multiple containers exist in at least one data center; the containers among the multiple containers are used for virtual resources running on the server; From the plurality of containers, determine a first target container whose first resource utilization rate is less than a first preset threshold; and obtain the target application corresponding to the first target container; Obtain the second resource utilization rate corresponding to each target application; filter the applications based on the second resource utilization rate and preset container setting rules to determine the second target containers; wherein, the preset container setting rules represent the minimum preset number of containers containing the same application in a data center; Obtain first historical resource data of the target application; and based on the first historical resource data, predict the actual number of containers using a preset prediction model; wherein the preset prediction model is characterized as a method for predicting future resource data based on historical resource data; Based on the actual number of containers and the acquired real-time number of containers, the second target container is scaled down; The step of filtering and determining the second target container based on the second resource utilization rate and preset container setting rules includes: From multiple target applications, a first application is identified whose second resource utilization rate is greater than a second preset threshold; wherein, the first application represents an application that is working normally; Based on the first target container, the containers corresponding to the first application, newly launched applications, and periodically changing applications are removed to determine the second initial target container; Obtain the data center information corresponding to the second initial target container; The second initial target container is selected by filtering the preset container setting rules and the data center information.
2. The method of claim 1, wherein, The step of obtaining the first resource utilization rate corresponding to each of the multiple containers includes: Obtain the second historical resource data corresponding to each of the multiple containers; By clustering the second historical resource data, the utilization rate of the first resource corresponding to each of the multiple containers is determined.
3. The method of claim 2, wherein, The second historical resource data includes: second historical processor utilization and second historical memory utilization; The step of clustering the second historical resource data to determine the utilization rate of the first resource corresponding to each of the multiple containers includes: Cluster the containers corresponding to the second historical processor usage rate and the second historical memory usage rate to determine the cluster centers of the containers; Based on the distribution of the cluster center, the second historical processor usage rate, and the second historical memory usage rate, the first resource usage rate corresponding to the plurality of containers is determined.
4. The method of claim 1, wherein, The step of obtaining the second resource utilization rate for each target application includes: Obtain the third processor utilization, third memory utilization, and timing fluctuation characteristics for each target application; The second resource utilization rate for each target application is calculated based on the third processor utilization rate, the third memory utilization rate, and the timing fluctuation characteristics.
5. The method according to claim 1, characterized in that, The step of filtering the second initial target container and determining the second target container by using the preset container setting rules and the data center information includes: Based on the data center information, the second initial target container is divided into multiple container groups; wherein, a container group represents a container within a data center; For the multiple container groups, the fixed container corresponding to each of the multiple container groups is determined according to the preset container setting rules; Based on the second initial target container, the fixed container is eliminated to determine the second target container.
6. The method according to any one of claims 1-5, characterized in that, Before predicting the actual number of containers based on the first historical resource data using a preset prediction model, the method further includes: Obtain the third historical resource data and the historical number of processor cores used by the first application; wherein, the first application represents multiple applications other than the target application corresponding to containers; The third historical resource data is preprocessed to obtain historical feature data; and the historical feature data is used to train the initial prediction model to determine the preset prediction model.
7. The method according to claim 6, characterized in that, The historical feature data includes: first historical feature data and second historical feature data; the preset prediction model includes: first prediction model and second prediction model; The step of training the initial prediction model using the historical feature data to determine the preset prediction model includes: Based on different parameter values, the initial prediction model is determined as a first initial prediction model and a second initial prediction model; The first historical feature data is input into the first initial prediction model for regression function fitting training to determine the first prediction model; wherein, the first historical feature data represents the historical feature data of daily capacity; and the first prediction model represents the prediction used for daily capacity. The second historical feature data is input into the second initial prediction model, and regression function fitting training is performed to determine the second prediction model; wherein, the second historical feature data represents the historical feature data of the large-scale promotion capacity, and the second prediction model represents the prediction used for the large-scale promotion capacity.
8. The method according to claim 1, characterized in that, The first historical resource data includes: historical resource data for the first month and historical resource data for the first week; The step of predicting the actual number of containers based on the first historical resource data and using a preset prediction model includes: The historical resource data of the first month and the historical resource data of the first week are preprocessed to obtain monthly feature data and weekly feature data, respectively. The monthly feature data is input into the preset prediction model for prediction to obtain a first prediction result; the first prediction result represents the core value of the processor corresponding to the monthly data. The weekly feature data is input into the preset prediction model for prediction to obtain a second prediction result; the second prediction result represents the core value of the processor corresponding to the weekly data. The first prediction result and the second prediction result are weighted and summed to obtain the final core value of the processor; and the actual number of containers is determined based on the final core value of the processor.
9. The method according to any one of claims 1-5, characterized in that, The step of scaling down the second target container based on the actual number of containers and the acquired real-time number of containers includes: Obtain the real-time number of containers; Based on the actual number of containers and the real-time number of containers, a difference operation is performed to determine the number of containers to be reduced in size; Based on the second target container, delete the container corresponding to the number of containers to complete the reduction.
10. A container adjusting device, characterized in that, It includes an acquisition unit, a determination unit, a prediction unit, and a scaling-down unit; among which, The acquisition unit is configured to acquire the first resource utilization rate corresponding to each of the multiple containers; wherein the multiple containers exist in at least one data center; the containers in the multiple containers are used for virtual resources running on the server; from the multiple containers, a first target container with the first resource utilization rate being less than a first preset threshold is determined; and the target application corresponding to the first target container is acquired; The determining unit is used to obtain the second resource utilization rate corresponding to each target application; and to filter and determine the second target container by using the second resource utilization rate and the preset container setting rules; wherein, the preset container setting rules represent the minimum preset number of containers containing the same application in a data center; The prediction unit is used to acquire first historical resource data of the target application; and based on the first historical resource data, predict the actual number of containers using a preset prediction model; wherein, the preset prediction model is used to predict future resource data based on historical resource data; The shrinking unit is used to shrink the second target container according to the actual number of containers and the acquired real-time number of containers; The determining unit is specifically used to determine, from multiple target applications, a first application whose second resource utilization rate is greater than a second preset threshold; wherein, the first application represents an application that is working normally; based on the first target container, remove the containers corresponding to the first application, newly launched applications, and periodically changing applications to determine a second initial target container; obtain the data center information corresponding to the second initial target container; and filter the second initial target container through the preset container setting rules and the data center information to determine the second target container.
11. A container adjusting device, characterized in that, include: Memory, used to store executable instructions; A processor, when executing executable instructions stored in the memory, implements the container adjustment method according to any one of claims 1-9.
12. A computer-readable storage medium, characterized in that, The storage medium stores executable instructions, which, when executed, cause the processor to perform the container adjustment method as described in any one of claims 1-9.
Citation Information
Patent Citations
Container resource adjustment method and device, computer device and storage medium
CN112181649A
Resource dynamic allocation method and device and server
CN114168248A
