Container group adjustment method and device, electronic equipment and storage medium

Abstract

The invention provides a container group adjustment method and device, electronic equipment and a storage medium, and relates to the technical field of computers.The method comprises the steps that at least two kinds of historical resource usage amount generated by a container group in a preset time period are obtained; determining a preset application type corresponding to the container group according to the historical resource usage amount, and determining a predicted resource usage amount of the container group in a future preset time period according to the historical resource usage amount; whether the predicted resource usage amount meets a preset capacity adjustment condition corresponding to the preset application type or not is judged, and when it is determined that the predicted resource usage amount meets the preset capacity adjustment condition, the container number of the container group is adjusted according to a container group adjustment mode corresponding to the preset capacity adjustment condition; according to the embodiment of the invention, the future predicted resource usage amount can be determined according to the historical resource usage amount generated by the container group, and capacity expansion or capacity reduction can be performed in advance according to the predicted resource usage amount under the condition that the resource usage demand of the preset application type is met, so that container resources can be fully utilized.

Description

Technical Field

[0001] The present invention relates to the field of storage, and in particular to a container group adjustment method, device, electronic equipment and storage medium. Background Art

[0002] A pod is a virtual device node installed within a physical device node, used to deploy business service programs. A container group is a group of containers deployed with the same business service. Containers in the same container group collectively provide business services.

[0003] In real-world applications, the volume of user requests fluctuates constantly, and the resources consumed by the container group to process them also fluctuate. To avoid over- or under-resourced container groups, timely scaling up or down is crucial. However, prior art methods scale container groups up or down based solely on their current resource availability, making them difficult to handle sudden traffic bursts. Summary of the Invention

[0004] The present invention aims to provide a container group adjustment method, apparatus, electronic device, and storage medium. These methods can determine the preset application type and future predicted resource usage corresponding to the container group based on the container group's corresponding historical resource usage, and can also expand or shrink the container group based on the preset capacity adjustment conditions and predicted resource usage corresponding to the preset application type, thereby improving the effectiveness of container group expansion and shrinkage.

[0005] To solve the above technical problems, the present invention provides a container group adjustment method, comprising:

[0006] Obtain at least two historical resource usages generated by the container group within a preset time period;

[0007] Determining a preset application type corresponding to the container group based on the historical resource usage, and determining a predicted resource usage of the container group within a future preset time period based on the historical resource usage;

[0008] Determine whether the predicted resource usage satisfies a preset capacity adjustment condition corresponding to the preset application type, and if it is determined that the predicted resource usage satisfies the preset capacity adjustment condition, adjust the number of containers in the container group according to a container group adjustment method corresponding to the preset capacity adjustment condition.

[0009] Optionally, determining the preset application type corresponding to the container group according to the historical resource usage includes:

[0010] converting the historical resource usage into data points;

[0011] Performing clustering processing on the data points to obtain aggregated data points, and determining distribution areas corresponding to the aggregated data points;

[0012] Determining, in at least two preset areas, the proportion of each of the preset areas in the distribution area; wherein the preset areas have corresponding preset application types;

[0013] The preset application type corresponding to the preset area with the largest occupancy ratio is determined as the preset application type corresponding to the container group.

[0014] Optionally, the preset areas are divided according to different numerical ratios between resource usage types.

[0015] Optionally, the preset capacity adjustment condition includes a preset weight, a preset capacity expansion condition, and a preset capacity reduction condition;

[0016] The determining whether the predicted resource usage satisfies a preset capacity adjustment condition corresponding to the preset application type, and adjusting the number of containers in the container group according to a container group adjustment method corresponding to the preset capacity adjustment condition when it is determined that the predicted resource usage satisfies the preset capacity adjustment condition, includes:

[0017] Determine a predicted remaining amount of resources corresponding to the container group according to the predicted resource usage;

[0018] Performing weighted fusion on the predicted remaining resources according to the preset weights to obtain a capacity adjustment evaluation value of the container group;

[0019] If the capacity adjustment evaluation value is greater than a first preset threshold, determining that the preset capacity expansion condition is met, determining the number of containers to be increased according to the predicted resource usage, and increasing the number of containers according to the number of containers to be increased;

[0020] If the capacity adjustment evaluation value is less than a second preset threshold, it is determined that the preset shrinking condition is met, the number of containers to be reduced is determined based on the predicted resource usage, and the number of containers is reduced based on the number of containers to be reduced; the first preset threshold is greater than the second preset threshold.

[0021] Optionally, increasing the number of containers according to the increased number of containers includes:

[0022] Obtaining, from at least two device nodes, a remaining amount of hardware resources of each of the device nodes and a historical hardware load value generated within a preset time period;

[0023] Determining a predicted hardware load value of the device node within a future preset time period based on the historical hardware load value;

[0024] Determining an evaluation score for the device node based on the predicted hardware load value and the remaining amount of hardware resources; wherein the predicted hardware load data is negatively correlated with the evaluation score, and the remaining amount of hardware resources is positively correlated with the evaluation score;

[0025] In the device node with the largest evaluation score, a container is created according to the container increase quantity.

[0026] Optionally, determining the predicted resource usage of the container group within a future preset time period based on the historical resource usage includes:

[0027] The historical resource usage is input into a pre-trained machine learning model so that the pre-trained machine learning model predicts the predicted resource usage corresponding to the historical resource usage based on the learned resource usage change sequence.

[0028] Optionally, after adjusting the number of containers in the container group according to the container group adjustment method corresponding to the preset capacity adjustment condition, the method further includes:

[0029] Obtaining a current resource usage of the container group, and determining whether a difference between the current resource usage and the predicted resource usage is greater than a third preset threshold;

[0030] If it is greater than the third preset threshold, the current resource usage is used to generate supplementary training data, and the supplementary training data is used to perform supplementary training on the pre-trained machine learning model.

[0031] The present invention may also provide a container group adjustment device, comprising:

[0032] An acquisition module, configured to acquire at least two historical resource usages generated by a container group within a preset time period;

[0033] a processing module, configured to determine a preset application type corresponding to the container group based on the historical resource usage, and determine a predicted resource usage of the container group within a future preset time period based on the historical resource usage;

[0034] The container adjustment module is configured to determine whether the predicted resource usage satisfies a preset capacity adjustment condition corresponding to the preset application type, and, if it is determined that the predicted resource usage satisfies the preset capacity adjustment condition, adjust the number of containers in the container group according to a container group adjustment method corresponding to the preset capacity adjustment condition.

[0035] The present invention may also provide an electronic device, comprising:

[0036] memory for storing computer programs;

[0037] A processor is configured to implement the container group adjustment method described above when executing the computer program.

[0038] The present invention may also provide a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions. When the computer-executable instructions are loaded and executed by a processor, the container group adjustment method described above is implemented.

[0039] The present invention provides a container group adjustment method, comprising: obtaining at least two historical resource usages generated by a container group within a preset time period; determining a preset application type corresponding to the container group based on the historical resource usages, and determining a predicted resource usage of the container group within a future preset time period based on the historical resource usages; determining whether the predicted resource usages meet a preset capacity adjustment condition corresponding to the preset application type, and if it is determined that the predicted resource usages meet the preset capacity adjustment condition, adjusting the number of containers in the container group according to a container group adjustment method corresponding to the preset capacity adjustment condition.

[0040] The present invention provides the following advantages: it obtains at least two historical resource usages generated by a container group within a preset time period; determines the preset application type corresponding to the container group based on the historical resource usages; and determines the predicted resource usage of the container group within a future preset time period based on the historical resource usages. Different preset application types correspond to different resource usage requirements; and the predicted resource usage can represent the future resource consumption of the container group. Subsequently, it determines whether the predicted resource usage meets the preset capacity adjustment conditions corresponding to the preset application type. If it is determined that the predicted resource usage meets the preset capacity adjustment conditions, the number of containers in the container group is adjusted according to the container group adjustment method corresponding to the preset capacity adjustment conditions. In other words, the present invention can determine the preset application type actually corresponding to the container group, and estimate whether the container group will need to be expanded or reduced in the future based on the preset capacity adjustment conditions corresponding to the preset application type. It can also adjust the number of containers in advance, thereby enabling pre-expansion or reduction to be performed while meeting the resource usage requirements of the preset application type. This improves the effectiveness of container expansion and reduction, thereby helping to improve the utilization of container resources. The present invention also provides a container group adjustment device, electronic device, and computer-readable storage medium, all of which have the aforementioned advantages. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0042] Figure 1 A flowchart of a container group adjustment method provided by an embodiment of the present invention;

[0043] Figure 2 This is an architectural diagram of a container group adjustment system provided by an embodiment of the present invention;

[0044] Figure 3 A flowchart of another container group adjustment method provided by an embodiment of the present invention;

[0045] Figure 4 A flow chart of load prediction provided by an embodiment of the present invention;

[0046] Figure 5 A flowchart of horizontal elastic capacity expansion provided by an embodiment of the present invention;

[0047] Figure 6 A structural block diagram of a container group adjustment device provided by an embodiment of the present invention;

[0048] Figure 7 This is a structural block diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0049] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0050] A pod is a virtual device node installed within a physical device node, used to deploy business service programs. A container group is a collection of containers deployed with the same business service. Containers in the same container group collectively provide business services. In real-world applications, the volume of user-initiated requests fluctuates constantly, and the resources consumed by the container group to process them also fluctuate. To avoid excessive or insufficient remaining resources in the container group, it is crucial to promptly scale the container group up or down. However, in related technologies, scaling a container group up or down is performed solely based on the group's real-time remaining resources, which is not conducive to handling sudden traffic bursts and can easily lead to resource waste.

[0051] In view of this, the present invention provides a container group adjustment method to better adjust the number of containers in a container group. This method can determine the preset application type corresponding to the container group and the future predicted resource usage based on the container group's corresponding historical resource usage. Furthermore, the container group can be expanded or reduced in capacity based on the preset capacity adjustment conditions corresponding to the preset application type and the predicted resource usage, thereby improving the effectiveness of container group expansion and reduction.

[0052] For easier understanding, please refer to Figure 1 , Figure 1 This is a flowchart of a container group adjustment method provided by an embodiment of the present invention. The method may include:

[0053] S101: Obtain at least two historical resource usages generated by a container group within a preset time period.

[0054] In this step, historical resource usage refers to the resource usage information generated by the container group's use of hardware resources during operation. Historical resource usage can be of at least two types, such as CPU usage, memory usage, and bandwidth usage. This step collects the historical resource usage generated by the container group within a preset time period and sorts it chronologically to generate a historical resource usage change sequence. This sequence reflects the temporal changes in historical resource usage and facilitates prediction of the container group's future resource usage.

[0055] It should be noted that this embodiment does not limit the length of the preset time period, which may be 30 minutes, for example. This embodiment also does not limit the specific type of historical resource usage, which may be set according to actual application requirements.

[0056] S102: Determine a preset application type corresponding to the container group based on the historical resource usage, and determine a predicted resource usage of the container group within a future preset time period based on the historical resource usage.

[0057] In this step, the preset application type corresponding to the container group is first determined based on historical resource usage. The preset application type is used to reflect the resource usage characteristics corresponding to the business services in the container group. For example, three preset application types can be set: CPU-intensive, memory-intensive, and balanced. The CPU-intensive type consumes the most CPU resources, the memory-intensive type consumes the most memory resources, and the balanced type consumes CPU and memory resources more evenly. It can be understood that different preset application types correspond to different scaling conditions. For example, CPU-intensive applications should prioritize sufficient CPU resources, memory-intensive applications should prioritize sufficient memory resources, and balanced applications need to comprehensively consider the use of CPU and memory resources. In addition, the resource usage characteristics corresponding to the business services in the container group can be reflected by historical resource usage. For example, if the historical resource usage shows that the container group's consumption of CPU resources is significantly greater than its consumption of memory resources, it can be determined that the container group corresponds to CPU-intensive applications. Therefore, this embodiment can use historical resource usage to determine the preset application type corresponding to the container group, so as to expand or shrink the container according to the resource usage requirements corresponding to the preset application type, thereby ensuring that the expansion or contraction of the container is more in line with actual application requirements.

[0058] Furthermore, the key to determining the preset application type corresponding to a container group lies in determining the ratio between the various historical resource usages generated by the container group. For example, if the CPU resource usage ratio is the largest, it can be determined to be CPU-intensive, and if the memory resource usage ratio is the largest, it can be determined to be memory-intensive. Of course, in actual applications, different ratios between historical resource usages can be pre-set, and the preset application type corresponding to the container group can be determined based on the relationship between the actual ratio between historical resource usages and the preset ratio. For example, CPU intensive, memory intensive, and balanced can be determined based on the following formula:

[0059] 1. If the CPU usage / memory usage ratio is greater than 2, the system is classified as CPU intensive.

[0060] 2. If the memory usage / CPU usage ratio is greater than 2, the system is classified as memory intensive.

[0061] 3. Other situations are classified as balanced.

[0062] Furthermore, to determine the preset application type corresponding to a container group based on all historical resource usage generated within a preset time period, a numerical range space can be determined based on the numerical range of resource usage, and the numerical range space can be divided into at least two preset regions, each of which corresponds to a preset application type. Subsequently, the historical resource usage can be converted into data points, and clustering can be used to identify clustered data points within the data points to filter out discrete data points, and the distribution regions corresponding to the clustered data points can be determined. Furthermore, the proportion of each preset region within the distribution region can be determined, and the preset application type corresponding to the preset region with the largest proportion can be determined as the preset application type corresponding to the container group. The preset regions are divided based on the different numerical proportions between resource usage types.

[0063] Based on this, determining the preset application type corresponding to the container group according to the historical resource usage may include:

[0064] Step 11: Convert the historical resource usage into data points.

[0065] Step 12: Cluster the data points to obtain clustered data points, and determine the distribution areas corresponding to the clustered data points.

[0066] It should be noted that this embodiment does not limit how to perform clustering processing, and reference may be made to related clustering technologies, for example, the K-means algorithm may be used for clustering.

[0067] Step 13: Determine, in at least two preset areas, the proportion of each preset area in the distribution area; wherein the preset areas have corresponding preset application types.

[0068] For example, the first preset area corresponding to CPU intensive type can be set according to CPU usage / memory usage > 2, the second preset area corresponding to memory intensive type can be set according to memory usage / CPU usage > 2, and the third preset area corresponding to balanced type can be set according to CPU usage / memory usage ≤ 2 and memory usage / CPU usage ≤ 2.

[0069] Step 14: Determine the preset application type corresponding to the preset area with the largest occupancy ratio as the preset application type corresponding to the container group.

[0070] In this step, the proportion of each preset area in the distribution area is determined, and the preset application type corresponding to the preset area with the largest proportion is determined as the preset application type corresponding to the container group. For example, if the first preset area occupies 60% of the distribution area and the third preset area occupies 40% of the distribution area, the preset application type corresponding to the container group can be determined to be CPU-intensive.

[0071] Furthermore, step S102 may also determine the predicted resource usage of the container group within a preset future time period based on the historical resource usage. For example, this embodiment may include a pre-trained machine learning model that has been trained using a number of resource usage change sequences, where the resource usage change sequence is a time-ordered sequence of resource usage changes. Furthermore, this step may input the historical resource usage into the pre-trained machine learning model, so that the pre-trained machine learning model predicts the predicted resource usage corresponding to the historical resource usage based on the learned resource usage change sequences.

[0072] Based on this, determining the predicted resource usage of the container group in a future preset time period based on the historical resource usage may include:

[0073] Step 21: Input the historical resource usage into a pre-trained machine learning model so that the pre-trained machine learning model predicts the predicted resource usage corresponding to the historical resource usage based on the learned resource usage change sequence.

[0074] It should be noted that this embodiment does not limit the specific model type of the pre-trained machine learning model, and it can be set according to actual application requirements. For example, considering that the LSTM model (Long Short-Term Memory) has strong processing capabilities for time series, the structure of the LSTM model can be used as the model structure of the pre-trained machine learning model in this embodiment.

[0075] S103: Determine whether the predicted resource usage satisfies a preset capacity adjustment condition corresponding to the preset application type, and if it is determined that the predicted resource usage satisfies the preset capacity adjustment condition, adjust the number of containers in the container group according to a container group adjustment method corresponding to the preset capacity adjustment condition.

[0076] In this step, after determining the predicted resource usage of the container group and the corresponding preset application type, the preset capacity adjustment conditions corresponding to the preset application type are obtained and a determination is made as to whether the predicted resource usage meets the preset capacity adjustment conditions. The preset capacity adjustment conditions are set based on the resource usage characteristics of the preset application type. If the preset capacity adjustment conditions are met, this step involves adjusting the number of containers in the container group, either by adding new containers or removing existing ones.

[0077] Specifically, the preset capacity adjustment conditions include preset weights, preset expansion conditions, and preset reduction conditions. The preset weights are used to weight the remaining amounts of various resources, and are set based on the resource usage characteristics corresponding to the preset application types. The preset expansion conditions refer to conditions that require expansion of a container group, and the preset reduction conditions refer to conditions that require reduction of a container group. In this embodiment, a predicted remaining amount of resources corresponding to a container group can be first determined based on the predicted resource usage, that is, the predicted remaining amount of resources that the container group will correspond to in the future given its current capacity. Subsequently, the predicted remaining amounts of resources can be weighted and combined based on the preset weights to obtain a capacity adjustment assessment value for the container group. The predicted remaining amount of resources and the capacity adjustment assessment value are negatively correlated; that is, the smaller the predicted remaining amount of resources, the larger the capacity adjustment assessment value. Subsequently, if the capacity adjustment assessment value is determined to be greater than a first preset threshold, the preset expansion condition can be determined to be satisfied. If the capacity adjustment assessment value is determined to be less than a second preset threshold, the preset reduction condition can be determined to be satisfied, where the first preset threshold is greater than the second preset threshold. In this way, this embodiment can predetermine whether the container group needs to be expanded or reduced in capacity based on the predicted remaining resources of the container group in the future and the resource usage characteristics of the preset application type corresponding to the container group, thereby effectively ensuring the rational use of container resources.

[0078] Furthermore, when expanding or shrinking capacity, this embodiment can also determine the number of containers to add or reduce based on the predicted resource usage. For example, a conversion function can be pre-set and the predicted resource usage can be input into the conversion function to determine the number of containers to add or reduce. It should be noted that this embodiment does not limit the specific conversion function and can be set based on actual application requirements.

[0079] Based on this, the determining whether the predicted resource usage satisfies a preset capacity adjustment condition corresponding to the preset application type, and when it is determined that the predicted resource usage satisfies the preset capacity adjustment condition, adjusting the number of containers in the container group according to a container group adjustment method corresponding to the preset capacity adjustment condition, includes:

[0080] Step 31: Determine the predicted remaining amount of resources corresponding to the container group according to the predicted resource usage;

[0081] Step 32: performing weighted fusion on the predicted remaining resources according to the preset weights to obtain a capacity adjustment evaluation value of the container group;

[0082] Step 33: If the capacity adjustment evaluation value is greater than a first preset threshold, determining that the preset capacity expansion condition is met, determining the number of containers to be added based on the predicted resource usage, and increasing the number of containers based on the number of containers to be added;

[0083] Step 34: If the capacity adjustment evaluation value is less than a second preset threshold, it is determined that the preset shrinking condition is met, the number of containers to be reduced is determined based on the predicted resource usage, and the number of containers is reduced based on the number of containers to be reduced; the first preset threshold is greater than the second preset threshold.

[0084] It should be noted that this embodiment does not limit the specific first and second preset thresholds, and they can be set according to actual application requirements. Thus, by weighting and fusing various predicted remaining resource amounts, this embodiment can expand or shrink capacity based on the resource usage characteristics of the preset application type, taking into account the remaining resource amounts of various types, thereby improving the expansion and shrinkage effect.

[0085] The following describes three specific preset capacity adjustment conditions. This embodiment can dynamically select a scaling strategy based on the application type, such as a memory priority strategy, a CPU priority strategy, or a balance strategy.

[0086] 1. Memory priority strategy (for memory-intensive applications):

[0087] Expansion conditions: Memory usage > 80%.

[0088] Scaling condition: Memory usage < 30%.

[0089] In this case, CPU usage has a weight of 0, and memory usage has a weight of 1. The weighting is: 0 * CPU usage + 1 * memory usage. Memory usage is the dominant indicator, and CPU usage is only for reference.

[0090] 2. CPU priority strategy (suitable for CPU-intensive applications):

[0091] Expansion conditions: CPU usage > 70%.

[0092] Scaling condition: CPU usage < 20%.

[0093] In this case, CPU usage has a weight of 1, and memory usage has a weight of 0. The weighting is: 1 * CPU usage + 0 * memory usage. CPU usage is the dominant indicator, and memory usage is only for reference.

[0094] 3. Balance strategy (applicable to balanced applications):

[0095] Expansion condition: (0.5*CPU usage + 0.5*memory usage) > 75%.

[0096] Scaling condition: (0.5*CPU usage + 0.5*memory usage) < 25%.

[0097] In this case, the weight of CPU usage is 0.5, and the weight of memory usage is 0.5, with the weighting being: 0.5 * CPU usage + 0.5 * memory usage. CPU and memory metrics are comprehensively considered to ensure resource balance.

[0098] Furthermore, considering that the container group needs to be added to the physical device node, in order to balance the burden of each device node, the container can also be created according to the burden of each device node. Specifically, this embodiment can obtain the remaining hardware resources of each device node and the historical hardware load value generated in a preset time period in at least two device nodes. The historical hardware load values ​​in the preset time period can form a historical hardware load value sequence, which is used to predict the predicted hardware load value of the device node in the future. Subsequently, the predicted hardware load value of the device node in the future preset time period can be determined based on the historical hardware load value. Subsequently, the evaluation score of the device node can be determined based on the predicted hardware load value and the remaining hardware resources, so as to comprehensively consider the future load situation of the device node and the current remaining hardware resources. The predicted hardware load data is negatively correlated with the evaluation score, and the remaining hardware resources is positively correlated with the evaluation score. Finally, a container can be created in the device node with the largest evaluation score based on the number of containers added.

[0099] Based on this, the step of increasing the number of containers according to the increased number of containers includes:

[0100] Step 41: From at least two device nodes, obtain the remaining amount of hardware resources of each device node and its historical hardware load value generated within a preset time period.

[0101] It should be noted that this embodiment does not limit the specific type of hardware resource remaining amount, and may be, for example, CPU remaining amount, memory remaining amount, network bandwidth remaining amount, etc. This embodiment also does not limit the method for determining the hardware load value, and may, for example, be determined by weighting CPU usage, memory usage, and network bandwidth usage.

[0102] Step 42: Determine a predicted hardware load value of the device node within a future preset time period based on the historical hardware load value.

[0103] In this step, a pre-trained machine learning model (such as an LSTM model) can also be used to predict the predicted hardware load value of the device node in a preset time period in the future.

[0104] Step 43: Determine an evaluation score of the device node based on the predicted hardware load value and the remaining amount of hardware resources; wherein the predicted hardware load data is negatively correlated with the evaluation score, and the remaining amount of hardware resources is positively correlated with the evaluation score;

[0105] Step 44: In the device node with the largest evaluation score, create a container according to the container increase quantity.

[0106] Specifically, the calculation method of the evaluation score can be expressed as:

[0107] NodeScore = α×RemainingCapacity + β×PredictedLoad.

[0108] NodeScore represents the evaluation score, RemainingCapacity represents the remaining amount of hardware resources, PredictedLoad represents the predicted hardware load value, and α and β are preset weights.

[0109] Finally, after the container quantity adjustment is completed, since the container quantity adjustment is based on the predicted resource usage output by the pre-trained machine learning model, this predicted resource usage may not match the actual situation, thereby affecting the effect of expansion or contraction. Therefore, this embodiment can also monitor the current resource usage of the container group and determine whether the difference between the current resource usage and the predicted resource usage is greater than a third preset threshold, that is, determine whether the difference between the current resource usage and the predicted resource usage is too large. If it is too large, it is necessary to use the current resource usage to generate supplementary training data, and use this supplementary training data to supplement the pre-trained machine learning model to improve the accuracy of the pre-trained machine learning model.

[0110] Based on this, after adjusting the number of containers in the container group according to the container group adjustment method corresponding to the preset capacity adjustment condition, the method may further include:

[0111] Step 51: Obtain the current resource usage of the container group, and determine whether the difference between the current resource usage and the predicted resource usage is greater than a third preset threshold;

[0112] Step 52: If it is greater than the third preset threshold, the current resource usage is used to generate supplementary training data, and the pre-trained machine learning model is supplemented with the supplementary training data.

[0113] It should be noted that this embodiment does not limit the specific third preset threshold value, and it can be set according to actual application requirements.

[0114] Based on the above embodiments, the present invention obtains at least two historical resource usages generated by a container group within a preset time period; determines the preset application type corresponding to the container group based on the historical resource usage; and determines the predicted resource usage of the container group within a future preset time period based on the historical resource usage. Different preset application types correspond to different resource usage requirements; and the predicted resource usage can represent the future resource consumption of the container group. Subsequently, it is determined whether the predicted resource usage meets the preset capacity adjustment conditions corresponding to the preset application type. If it is determined that the predicted resource usage meets the preset capacity adjustment conditions, the number of containers in the container group is adjusted according to the container group adjustment method corresponding to the preset capacity adjustment conditions. In other words, the present application can determine the preset application type actually corresponding to the container group, and estimate whether the container group will need to be expanded or reduced in the future based on the preset capacity adjustment conditions corresponding to the preset application type. It can also adjust the number of containers in advance, thereby allowing pre-expansion or reduction to be performed while meeting the resource usage requirements of the preset application type. This can improve the effectiveness of container expansion and reduction, thereby helping to improve the utilization of container resources.

[0115] Based on the above embodiment, the container group adjustment method is fully introduced below with reference to a specific schematic diagram. Figure 2 、 Figure 3 , Figure 2 This is an architectural diagram of a container group adjustment system provided by an embodiment of the present invention. Figure 3 This is a flow chart of another container group adjustment method provided by an embodiment of the present invention. This method may specifically include:

[0116] 1. Load prediction algorithm:

[0117] This solution first uses machine learning algorithms to predict future load based on historical load data and real-time traffic trends. The prediction results are used to pre-adjust the scaling strategy to cope with possible future traffic fluctuations. Common load prediction algorithms include time series analysis and regression analysis. Figure 4 , Figure 4 A flow chart of load prediction provided by an embodiment of the present invention.

[0118] 2. Adaptive threshold adjustment:

[0119] Traditional HPA mechanisms use fixed thresholds (such as CPU usage or memory usage) to determine capacity expansion or contraction. In contrast, the adaptive adjustment scheme of the present invention dynamically adjusts the expansion and contraction thresholds based on real-time load conditions. By comparing the current load with the predicted load, the system can react quickly to load fluctuations, avoiding unnecessary or delayed expansion.

[0120] 3. Multi-dimensional resource analysis:

[0121] This method doesn't rely solely on a single resource metric (such as CPU or memory) to make decisions. Instead, it comprehensively considers multiple resource dimensions, including CPU, memory, and network bandwidth, to assess pod scaling requirements. This comprehensive analysis of multi-dimensional metrics improves decision-making accuracy and avoids unnecessary scaling due to fluctuations in a single resource.

[0122] For example, you can set a weighted scoring model: Score = w1×CPU + w2×Memory + w3×Network + w4×ResponseTime + w5×RequestRate, where weights w1-w5 can be dynamically adjusted based on the application type. If the score exceeds the threshold, capacity expansion is triggered.

[0123] 4. Application type identification and classification:

[0124] The present invention can identify the resource usage pattern of an application by automated means and classify it as memory intensive, CPU intensive or balanced.

[0125] Specific content:

[0126] 1) Use machine learning algorithms (such as K-means clustering) to analyze historical application CPU and memory usage data and automatically classify application types.

[0127] Classification by:

[0128] a. If the CPU usage / memory usage ratio is greater than 2, the system is classified as CPU-intensive.

[0129] b. If the ratio of memory usage to CPU usage is greater than 2, the system is classified as memory-intensive.

[0130] c. Other situations are classified as balanced.

[0131] This function provides accurate application characteristic basis for subsequent differentiated scaling rules.

[0132] 2) Differentiated scaling rules:

[0133] Dynamically select scaling policies based on application type, such as memory priority, CPU priority, or balanced policy.

[0134] Specific content:

[0135] a. Memory priority strategy (suitable for memory-intensive applications):

[0136] Expansion conditions: Memory usage > 80%.

[0137] Scaling condition: Memory usage < 30%.

[0138] Memory usage is the dominant indicator, and CPU usage is only for reference.

[0139] b.CPU priority policy (suitable for CPU-intensive applications):

[0140] Expansion condition: CPU usage > 70%.

[0141] Scaling down condition: CPU usage < 20%.

[0142] CPU usage is the dominant indicator, and memory usage is only for reference.

[0143] c.Balancing strategy (applicable to balanced applications):

[0144] Expansion condition: (0.5 * CPU usage + 0.5 * memory usage) > 75%.

[0145] Scaling condition: (0.5 * CPU usage + 0.5 * memory usage) < 25%.

[0146] Comprehensively consider CPU and memory indicators to ensure resource balance.

[0147] It can be seen that compared with the traditional scaling method based on a single indicator (such as CPU usage alone), differentiated rules are more in line with the actual needs of the application and avoid misjudgment.

[0148] 5. Intelligent scheduling:

[0149] This technical solution further optimizes Pod scheduling strategies. When the system decides to expand capacity, the intelligent scheduling module rationally allocates newly created Pods based on the cluster's load and expected load, ensuring balanced resource utilization across nodes. This minimizes resource waste and improves system processing capacity.

[0150] 6. Adaptive strategy adjustment:

[0151] The present invention also dynamically adjusts the expansion and contraction strategy according to resource usage changes during application runtime.

[0152] Specific content:

[0153] By monitoring resource usage patterns in real time, it can detect changes in application types (such as from CPU-intensive to memory-intensive) and automatically switch strategies.

[0154] Implementation: Use sliding window technology to analyze resource data from the last 30 minutes, reassess application types every hour, and adjust policies.

[0155] It can be seen that the present invention can enhance the system's ability to adapt to dynamic loads and ensure that the scaling rules always match the current state of the application.

[0156] Finally, please refer to Figure 5 , Figure 5 A flowchart of horizontal elastic capacity expansion is provided in an embodiment of the present invention. The flowchart may include:

[0157] 1. PositiveHPAController creates HorizontalPodAutoscaler and TimeSeriesFrocastion objects;

[0158] 2. Predictor obtains historical metrics from Prometheus, calculates them through the prediction algorithm, and records the results in TimeSeriesForcastion;

[0159] 3. HPAController reads metric data from KubeApiServer through metricclient

[0160] 4. KubeApiServer routes the request to MetricAdapter;

[0161] 5. HPAController calculates the results returned by all metrics to obtain the final elastic replica recommendation;

[0162] 6. HPAController calls the scaleAPI to scale the target application up or down.

[0163] The following describes a container group adjustment device, an electronic device, and a computer-readable storage medium provided in embodiments of the present invention. The container group adjustment device, electronic device, and computer-readable storage medium described below can be referenced in correspondence with the container group adjustment method described above.

[0164] Please refer to Figure 6 , Figure 6 This is a structural block diagram of a container group adjustment device provided by an embodiment of the present invention. The device may include:

[0165] An acquisition module 601 is configured to acquire at least two historical resource usages generated by a container group within a preset time period;

[0166] Processing module 602, configured to determine a preset application type corresponding to the container group based on the historical resource usage, and determine a predicted resource usage of the container group within a future preset time period based on the historical resource usage;

[0167] The container adjustment module 603 is configured to determine whether the predicted resource usage satisfies a preset capacity adjustment condition corresponding to the preset application type, and, if it is determined that the predicted resource usage satisfies the preset capacity adjustment condition, adjust the number of containers in the container group according to a container group adjustment method corresponding to the preset capacity adjustment condition.

[0168] Optionally, the processing module 602 includes:

[0169] a conversion submodule, configured to convert the historical resource usage into data points;

[0170] A clustering submodule, configured to perform clustering processing on the data points to obtain clustered data points, and determine the distribution areas corresponding to the clustered data points;

[0171] An occupancy ratio determination submodule, configured to determine, among at least two preset areas, an occupancy ratio of each of the preset areas in the distribution area; wherein the preset areas have corresponding preset application types;

[0172] The application type determination submodule is configured to determine the preset application type corresponding to the preset area with the largest occupancy ratio as the preset application type corresponding to the container group.

[0173] Optionally, the preset areas are divided according to different numerical ratios between the resource usage types.

[0174] Optionally, the preset capacity adjustment condition includes a preset weight, a preset capacity expansion condition, and a preset capacity reduction condition;

[0175] The container adjustment module 603 includes:

[0176] a remaining amount determining submodule, configured to determine a predicted remaining amount of resources corresponding to the container group according to the predicted resource usage;

[0177] An evaluation value determination submodule, configured to perform weighted fusion on each of the predicted remaining resources according to the preset weights to obtain a capacity adjustment evaluation value for the container group;

[0178] a capacity expansion determination submodule, configured to determine that the preset capacity expansion condition is satisfied if the capacity adjustment evaluation value is greater than a first preset threshold, determine a number of containers to be increased based on the predicted resource usage, and increase the number of containers based on the number of containers to be increased;

[0179] A capacity reduction determination submodule is configured to determine that the preset capacity reduction condition is met if the capacity adjustment evaluation value is less than a second preset threshold, determine the number of containers to be reduced based on the predicted resource usage, and reduce the number of containers based on the number of containers to be reduced; and the first preset threshold is greater than the second preset threshold.

[0180] Optionally, the expansion determination submodule includes:

[0181] An acquiring unit, configured to acquire, from at least two device nodes, a remaining amount of hardware resources of each of the device nodes and a historical hardware load value generated within a preset time period;

[0182] A prediction unit, configured to determine a predicted hardware load value of the device node within a future preset time period based on the historical hardware load value;

[0183] An evaluation unit, configured to determine an evaluation score of the device node based on the predicted hardware load value and the remaining amount of hardware resources; wherein the predicted hardware load data is negatively correlated with the evaluation score, and the remaining amount of hardware resources is positively correlated with the evaluation score;

[0184] The capacity expansion unit is configured to create a container in the device node with the largest evaluation score according to the increased number of containers.

[0185] Optionally, the processing module 602 includes:

[0186] A prediction submodule is used to input the historical resource usage into a pre-trained machine learning model so that the pre-trained machine learning model predicts the predicted resource usage corresponding to the historical resource usage based on the learned resource usage change sequence.

[0187] Optionally, the device may further include:

[0188] a difference detection module, configured to obtain a current resource usage of the container group and determine whether a difference between the current resource usage and the predicted resource usage is greater than a third preset threshold;

[0189] A supplementary training module is used to generate supplementary training data using the current resource usage if it is greater than the third preset threshold, and use the supplementary training data to supplement the pre-trained machine learning model.

[0190] Please refer to Figure 7 , Figure 7 This is a structural block diagram of an electronic device provided in an embodiment of the present invention. The embodiment of the present invention provides an electronic device 70, including a processor 71 and a memory 72; wherein the memory 72 is used to store a computer program; and the processor 71 is used to execute the container group adjustment method provided in the aforementioned embodiment when executing the computer program.

[0191] For the specific process of the above container group adjustment method, reference may be made to the corresponding content provided in the above embodiments, which will not be described in detail here.

[0192] Furthermore, the memory 72 as a carrier for resource storage may be a read-only memory, a random access memory, a magnetic disk or an optical disk, etc., and the storage method may be temporary storage or permanent storage.

[0193] In addition, the electronic device 70 also includes a power supply 73, a communication interface 74, an input / output interface 75 and a communication bus 76; wherein, the power supply 73 is used to provide operating voltage for each hardware device on the electronic device 70; the communication interface 74 can create a data transmission channel between the electronic device 70 and external devices, and the communication protocol it follows is any communication protocol that can be applied to the technical solution of the present invention, and is not specifically limited here; the input / output interface 75 is used to obtain external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs and is not specifically limited here.

[0194] An embodiment of the present invention further provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the container group adjustment method described in the above embodiment.

[0195] Since the embodiments of the computer program product part correspond to the embodiments of the container group adjustment method part, the embodiments of the computer program product part refer to the description of the embodiments of the container group adjustment method part, and are not repeated here.

[0196] An embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the container group adjustment method described in the above embodiment is implemented.

[0197] Since the embodiments of the computer-readable storage medium portion correspond to the embodiments of the container group adjustment method portion, the embodiments of the storage medium portion are described in detail in the description of the embodiments of the container group adjustment method portion, and are not repeated here.

[0198] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Reference can be made to the common and similar parts between the various embodiments. For the devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the method description.

[0199] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the components and steps of each example according to their functions. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present invention.

[0200] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0201] The above describes in detail the container group adjustment method, device, electronic device, and storage medium provided by the present invention. This article uses specific examples to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only intended to help understand the method and core concept of the present invention. It should be noted that for those skilled in the art, without departing from the principles of the present invention, various improvements and modifications can be made to the present invention, and these improvements and modifications also fall within the scope of protection of the present invention.

Claims

1. A container group adjustment method, characterized in that: include: Obtain at least two historical resource usages generated by the container group within a preset time period; Determining a preset application type corresponding to the container group based on the historical resource usage, and determining a predicted resource usage of the container group within a future preset time period based on the historical resource usage; Determine whether the predicted resource usage satisfies a preset capacity adjustment condition corresponding to the preset application type, and if it is determined that the predicted resource usage satisfies the preset capacity adjustment condition, adjust the number of containers in the container group according to a container group adjustment method corresponding to the preset capacity adjustment condition.

2. The container group adjustment method according to claim 1, characterized in that: The determining, based on the historical resource usage, the preset application type corresponding to the container group includes: converting the historical resource usage into data points; Performing clustering processing on the data points to obtain aggregated data points, and determining distribution areas corresponding to the aggregated data points; Determining, in at least two preset areas, the proportion of each of the preset areas in the distribution area; wherein the preset areas have corresponding preset application types; The preset application type corresponding to the preset area with the largest occupancy ratio is determined as the preset application type corresponding to the container group.

3. The container group adjustment method according to claim 2, characterized in that: The preset areas are divided according to different numerical ratios between resource usage types.

4. The container group adjustment method according to claim 1, characterized in that: The preset capacity adjustment conditions include preset weights, preset expansion conditions, and preset reduction conditions; The determining whether the predicted resource usage satisfies a preset capacity adjustment condition corresponding to the preset application type, and adjusting the number of containers in the container group according to a container group adjustment method corresponding to the preset capacity adjustment condition when it is determined that the predicted resource usage satisfies the preset capacity adjustment condition, includes: Determine a predicted remaining amount of resources corresponding to the container group according to the predicted resource usage; Performing weighted fusion on the predicted remaining resources according to the preset weights to obtain a capacity adjustment evaluation value of the container group; If the capacity adjustment evaluation value is greater than a first preset threshold, determining that the preset capacity expansion condition is met, determining the number of containers to be increased according to the predicted resource usage, and increasing the number of containers according to the number of containers to be increased; If the capacity adjustment evaluation value is less than a second preset threshold, it is determined that the preset shrinking condition is met, the number of containers to be reduced is determined based on the predicted resource usage, and the number of containers is reduced based on the number of containers to be reduced; the first preset threshold is greater than the second preset threshold.

5. The container group adjustment method according to claim 4, characterized in that: Increasing the number of containers according to the increased number of containers includes: Obtaining, from at least two device nodes, a remaining amount of hardware resources of each of the device nodes and a historical hardware load value generated within a preset time period; Determining a predicted hardware load value of the device node within a future preset time period based on the historical hardware load value; Determining an evaluation score for the device node based on the predicted hardware load value and the remaining amount of hardware resources; wherein the predicted hardware load data is negatively correlated with the evaluation score, and the remaining amount of hardware resources is positively correlated with the evaluation score; In the device node with the largest evaluation score, a container is created according to the container increase quantity.

6. The container group adjustment method according to claim 1, characterized in that: Determining the predicted resource usage of the container group within a future preset time period based on the historical resource usage includes: The historical resource usage is input into a pre-trained machine learning model so that the pre-trained machine learning model predicts the predicted resource usage corresponding to the historical resource usage based on the learned resource usage change sequence.

7. The container group adjustment method according to claim 6, characterized in that: After adjusting the number of containers in the container group according to the container group adjustment method corresponding to the preset capacity adjustment condition, the method further includes: Obtaining a current resource usage of the container group, and determining whether a difference between the current resource usage and the predicted resource usage is greater than a third preset threshold; If it is greater than the third preset threshold, the current resource usage is used to generate supplementary training data, and the supplementary training data is used to perform supplementary training on the pre-trained machine learning model.

8. A container group adjustment device, characterized in that: include: An acquisition module, configured to acquire at least two historical resource usages generated by a container group within a preset time period; a processing module, configured to determine a preset application type corresponding to the container group based on the historical resource usage, and determine a predicted resource usage of the container group within a future preset time period based on the historical resource usage; The container adjustment module is configured to determine whether the predicted resource usage satisfies a preset capacity adjustment condition corresponding to the preset application type, and, if it is determined that the predicted resource usage satisfies the preset capacity adjustment condition, adjust the number of containers in the container group according to a container group adjustment method corresponding to the preset capacity adjustment condition.

9. An electronic device, characterized in that: include: memory for storing computer programs; A processor, configured to implement the container group adjustment method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are loaded and executed by the processor, the container group adjustment method according to any one of claims 1 to 7 is implemented.

Images