Method, device, electronic equipment and storage medium for resource dynamic scaling and quota limiting of cloud load balancer

Abstract

The method, device, electronic equipment and storage medium provided by the cloud load balancer resource dynamic scaling and quota limiting method provided by the embodiments of the present application comprise: periodically polling to obtain service data and resource usage for a maximum number of replicas of a current computing Pod, the service data comprising first index data of a first weight index and second index data of a second weight index; determining a basic maximum number of replicas based on the first index data and the resource usage; dynamically adjusting the basic maximum number of replicas based on the second index data by means of a hierarchical threshold triggering mechanism to determine a final maximum number of replicas; and updating the maximum number of replicas of an HPA object by means of the final maximum number of replicas. The scheme calculates the final maximum number of replicas based on the data of the service indexes, and can dynamically update the maximum number of replicas in the HPA object, and the entire process completely skips the intermediate adaptation link of converting the service indexes into HPA native indexes, not only eliminating the development and maintenance cost of the adapter, but also avoiding causing architecture redundancy.

Description

Technical Field

[0001] This application relates to the field of Internet technology, and in particular to a method, apparatus, electronic device, and storage medium for dynamic resource scaling and quota limiting of a cloud load balancer. Background Technology

[0002] The current mainstream containerized elastic scaling solutions mainly rely on the HPA (Horizontal Pod Autoscaler) controller to trigger scaling up and down based on a single-dimensional metric.

[0003] Because the HPA controller only supports basic resource metrics such as CPU (Central Processing Unit) and memory by default, business metrics (such as QPS (Queries Per Second), CPS (Connections Per Second), and ACC (Active Concurrent Connections)) need to be converted into resource request values ​​through an adapter. This tightly coupled design means that a separate adapter needs to be developed for each new business metric, resulting in architectural redundancy and significantly increasing deployment, configuration, and version compatibility maintenance costs. Summary of the Invention

[0004] In view of this, in order to effectively alleviate the above-mentioned technical problems, embodiments of this application provide a method, apparatus, electronic device and storage medium for dynamic resource scaling and quota limiting of a cloud load balancer.

[0005] In a first aspect, embodiments of this application provide a method for dynamic resource scaling and quota limiting of a cloud load balancer, the method comprising:

[0006] Periodically poll to obtain business data and resource utilization for the current maximum number of replicas of the compute Pod. The business data includes first indicator data of a first weight indicator and second indicator data of a second weight indicator. The first weight indicator and the second weight indicator are business indicators that do not need to be adapted to the HPA controller.

[0007] The basic maximum number of replicas is determined based on the first indicator data and the resource utilization rate;

[0008] Based on the second indicator data, the basic maximum number of replicas is dynamically adjusted through a tiered threshold triggering mechanism to determine the final maximum number of replicas.

[0009] Update the maximum number of HPA replicas in the HPA object using the final maximum number of replicas.

[0010] Optionally, as described above, the periodic polling to obtain business data and resource utilization for the current maximum number of replicas of the compute Pod includes:

[0011] Obtain the first resource utilization rate sequence collected using the current time window;

[0012] Extract the start and end timestamps of the time window;

[0013] Determine whether the time span of the start and end timestamps is less than a preset time span;

[0014] If it is determined that the time span of the start and end timestamps is less than the preset time span, the current first data sequence is obtained by querying the cluster monitoring system based on the preset time span;

[0015] If it is determined that the time span of the start and end timestamps is equal to or greater than the preset time span, the current first data sequence is obtained by querying the cluster monitoring system based on the time span of the start and end timestamps.

[0016] Time-align the first resource utilization rate sequence and the first data sequence to determine the second resource utilization rate sequence and the second data sequence;

[0017] The resource utilization rate is obtained based on the second resource utilization rate sequence, and the first indicator data of the first weight indicator and the second indicator data of the second weight indicator are obtained based on the second data sequence.

[0018] Optionally, as described above, the resource utilization includes the current number of Pods, current memory utilization, and current CPU utilization.

[0019] The step of obtaining the resource utilization rate based on the second resource utilization rate sequence includes:

[0020] Obtain the latest Pod count from the second resource utilization sequence;

[0021] The average number of Pods and the standard deviation of Pods are determined based on the second resource utilization sequence.

[0022] Determine whether the noise filtering mechanism based on 2σ standard deviation is satisfied based on the latest number of Pods, the average number of Pods, and the standard deviation of Pods.

[0023] If the noise filtering mechanism based on 2σ standard deviation is satisfied, the latest number of Pods is determined as the current number of Pods;

[0024] If the noise filtering mechanism based on 2σ standard deviation is not satisfied, the latest Pod count at the previous running time is determined as the current Pod count;

[0025] Obtain the latest memory utilization rate from the second resource utilization rate sequence, and determine the latest memory utilization rate as the current memory utilization rate;

[0026] The latest CPU utilization is obtained from the CPU utilization sequence, and the latest CPU utilization is determined as the current CPU utilization.

[0027] Optionally, as described above, the first indicator data of the first weight indicator and the second indicator data of the second weight indicator are obtained based on the second data sequence;

[0028] Obtain the latest first indicator data from the second data sequence, and determine the latest first indicator data as the first indicator data of the first weight indicator;

[0029] Obtain the latest second indicator data from the second data sequence, and determine the latest second indicator data as the second indicator data of the second weight indicator.

[0030] Optionally, as described above, determining the basic maximum number of replicas based on the first indicator data and the resource utilization rate includes:

[0031] Obtain the maximum threshold, memory utilization threshold, and CPU utilization threshold of the first weighted index;

[0032] The maximum number of CPU replicas is determined based on the current number of Pods, the maximum metric threshold, the first metric data, the CPU utilization threshold, and the current CPU utilization.

[0033] The maximum number of memory replicas is determined based on the current number of Pods, the maximum metric threshold, the first metric data, the memory utilization threshold, and the current memory utilization.

[0034] Select the maximum value between the maximum number of CPU replicas and the maximum number of memory replicas, and round the maximum value up to obtain the basic maximum number of replicas.

[0035] Optionally, as described above, the step of dynamically adjusting the basic maximum replica count based on the second indicator data through a tiered threshold triggering mechanism to determine the final maximum replica count includes:

[0036] The value per core is determined based on the second indicator data and the current number of Pods;

[0037] The system detects whether the value per core exceeds a preset index data threshold range; wherein the preset index data threshold range is a data range consisting of a pre-defined pressure value and a limit value corresponding to each core.

[0038] If the value per core is detected to be within the preset index data threshold range, the real-time change rate of the first weight index is obtained;

[0039] Detect whether the real-time rate of change exceeds a preset rate of change threshold;

[0040] If the real-time change rate is detected to exceed the preset change rate threshold, the basic maximum replica count is adjusted according to the preset adjustment Pod count, and the adjusted basic maximum replica count is determined as the final maximum replica count.

[0041] If the value per core unit is detected to exceed the preset indicator data threshold range, the basic maximum replica count is modified based on the second indicator data, and the modified basic maximum replica count is determined as the final maximum replica count.

[0042] If the value per core is detected to be lower than the preset index data threshold range, or if the real-time change rate is detected to be less than the preset change rate threshold, the basic maximum replica count is determined as the final maximum replica count.

[0043] Optionally, as described above, modifying the base maximum replica count based on the second indicator data includes:

[0044] The required number of cores is determined based on the per-core value and the limit value;

[0045] The number of replicas to be added is determined based on the required number of cores;

[0046] The base maximum number of replicas is modified based on the number of replicas added.

[0047] Optionally, as described above, before updating the maximum number of HPA replicas in the HPA object using the final maximum number of replicas, the method further includes:

[0048] Based on the first metric data, the current number of Pods, the current CPU utilization, the current memory utilization, the maximum metric threshold, the CPU utilization threshold, the memory utilization threshold, and the maximum number of HPA replicas, determine the scaling flag value corresponding to the final maximum number of replicas;

[0049] The final maximum number of replicas and the corresponding scaling flag value are stored in the recommendation array; wherein, the recommendation array also includes the final maximum number of replicas determined at multiple historical moments, and the scaling flag value corresponding to each final maximum number of replicas.

[0050] Optionally, as described above, determining the scaling flag value corresponding to the final maximum replica count based on the first metric data, the current number of Pods, the current CPU utilization, the current memory utilization, the maximum metric threshold, the CPU utilization threshold, the memory utilization threshold, and the maximum HPA replica count includes:

[0051] The single Pod CPU utilization is determined based on the first indicator data, the current number of Pods, and the current CPU utilization; and the single Pod memory utilization is determined based on the first indicator data, the current number of Pods, and the current memory utilization.

[0052] The incremental demand value for memory Pods is determined based on the single Pod memory utilization rate, the first indicator data, and the maximum indicator threshold; and the incremental demand value for CPU Pods is determined based on the single Pod CPU utilization rate, the first indicator data, and the maximum indicator threshold.

[0053] The remaining available Pod capacity value for CPU is determined based on the CPU utilization threshold, the current CPU utilization, the current number of Pods, and the maximum number of HPA replicas; and the remaining available Pod capacity value for memory is determined based on the memory utilization threshold, the current memory utilization, the current number of Pods, and the maximum number of HPA replicas.

[0054] The scaling flag value corresponding to the final maximum number of replicas is determined based on the final maximum number of replicas, the maximum number of HPA replicas, the incremental demand value of memory Pods, the incremental demand value of CPU Pods, the remaining available Pod capacity of CPU, the remaining available Pod capacity of memory, the first indicator data, the current CPU utilization, the current number of Pods, and the latest number of Pods.

[0055] Optionally, as described above, the method further includes:

[0056] The single-pod CPU utilization and the maximum number of CPU replicas are respectively used as smoothing objects, and the following is performed on each of the smoothing objects:

[0057] Retrieve multiple historical data values ​​and the current data value;

[0058] The mean and standard deviation are determined based on the multiple historical data values ​​and the current data value;

[0059] Based on the current data value, the mean, and the standard deviation, determine whether the noise filtering mechanism based on the standard deviation is satisfied.

[0060] If the current data value is determined to meet the standard deviation-based noise filtering mechanism, it is retained.

[0061] If the noise filtering mechanism based on standard deviation is not satisfied, the historical data value from the previous moment is retained.

[0062] Optionally, as described above, determining the scaling flag value corresponding to the final maximum replica count based on the final maximum replica count, the maximum HPA replica count, the incremental memory Pod requirement value, the incremental CPU Pod requirement value, the remaining available CPU Pod capacity value, the remaining available memory Pod capacity value, the first metric data, the current CPU utilization, the current number of Pods, and the latest number of Pods includes:

[0063] Check whether the final maximum number of replicas is the same as the latest number of Pods;

[0064] If the final maximum number of replicas is found to be the same as the latest number of Pods, it is determined whether the final maximum number of replicas is selected from the maximum number of CPU replicas.

[0065] If the final maximum number of replicas is detected to be selected from the maximum number of CPU replicas, it is determined whether the first condition or the second condition is met based on the CPUPod incremental demand value and the CPU remaining available Pod capacity value.

[0066] If either the first condition or the second condition is met, the value of the telescoping flag is set to the first value;

[0067] If the final maximum number of replicas is detected to be selected from the maximum number of replicas in memory, it is determined whether the third or fourth condition is met based on the incremental memory Pod requirement value and the remaining available memory Pod capacity value.

[0068] If the third or fourth condition is met, the value of the telescoping flag is set to the first value;

[0069] If the final maximum number of replicas is found to be different from the latest number of Pods, the average data of the first indicator is determined based on the second data sequence.

[0070] Determine whether the fifth condition is met based on the first indicator data and the average data of the first indicator, or determine whether the sixth condition is met based on the current number of Pods and the current number of Pods at the previous running time.

[0071] If the fifth or sixth condition is met, the value of the telescoping flag is set to the second value;

[0072] If the fifth and sixth conditions are not met, check whether the current CPU utilization is less than a preset pressure threshold.

[0073] If the current CPU utilization is detected to be less than the preset pressure threshold, the scaling flag value is set to the third value;

[0074] If the current CPU utilization is not less than the preset pressure threshold, it is detected whether the final maximum number of replicas is less than the latest number of Pods;

[0075] If the final maximum number of replicas is detected to be less than the latest number of Pods, the scaling flag value is set to the fourth value;

[0076] If the final maximum number of replicas is detected to be greater than the latest number of Pods, it is detected whether the first metric data is less than a preset initial threshold.

[0077] If the first indicator data is detected to be less than the preset initial threshold, the scaling flag value is set to the fifth value;

[0078] If the first indicator data is detected to be not less than the preset initial threshold, it is detected whether the final maximum number of replicas is greater than the maximum number of HPA replicas;

[0079] If it is detected that the final maximum number of replicas is not greater than the maximum number of replicas of HPA, the scaling flag value is set to the sixth value;

[0080] If the final maximum number of replicas is detected to be greater than the maximum number of replicas of the HPA, the scaling flag value is set to the seventh value.

[0081] Optionally, as described above, updating the maximum number of HPA replicas in the HPA object using the final maximum number of replicas includes:

[0082] Iterate through and detect the scaling flag values ​​corresponding to each of the final maximum replica counts in the recommendation array;

[0083] If it is detected that the scaling flag values ​​corresponding to each of the final maximum replica numbers in the recommendation array are all greater than the preset value, or if there are values ​​greater than and equal to the preset value at the same time, the smallest final maximum replica number is selected from the recommendation array as the final target maximum replica number, and the HPA maximum replica number in the HPA object is updated using the final target maximum replica number.

[0084] If it is detected that the scaling flag values ​​corresponding to each of the final maximum replica counts in the recommendation array are all less than the preset value, or if there are values ​​that are both less than and equal to the preset value, the largest final maximum replica count is selected from the recommendation array as the final target maximum replica count, and the maximum replica count of HPA in the HPA object is updated using the final target maximum replica count.

[0085] Optionally, as described above, updating the maximum number of HPA replicas in the HPA object using the final target maximum number of replicas includes:

[0086] Check whether the final target maximum number of replicas is the same as the maximum number of replicas of the HPA;

[0087] If the maximum number of replicas of the final target is detected to be the same as the maximum number of replicas of the HPA, the maximum number of replicas of the HPA in the HPA object will not be updated;

[0088] If the maximum number of replicas in the final target is different from the maximum number of replicas in the HPA, it is detected whether the maximum number of replicas in the final target exceeds the replica number threshold; wherein, the replica number threshold is a preset upper limit of the latest number of Pods;

[0089] If the maximum number of replicas of the final target is not exceeded, it is detected whether the maximum number of replicas of the final target is less than the latest number of Pods;

[0090] If the maximum number of replicas of the final target is detected to be not less than the latest number of Pods, the maximum number of replicas of HPA in the HPA object is replaced and updated based on the maximum number of replicas of the final target;

[0091] If the maximum number of replicas of the final target is less than the latest number of Pods, obtain the tag value for enabling the scaling down label;

[0092] Detect whether the label value is a preset value;

[0093] If the detected tag value is the preset value, the maximum number of HPA replicas in the HPA object is replaced and updated based on the final target maximum number of replicas;

[0094] If the detected tag value is not the preset value, the maximum number of HPA replicas in the HPA object is replaced and updated based on the latest Pod count;

[0095] If the maximum number of replicas of the final target is detected to exceed the replica number threshold, the maximum number of replicas of the HPA in the HPA object is replaced and updated based on the replica number threshold.

[0096] Optionally, as described above, the method further includes:

[0097] The target scaling strategy attribute value corresponding to the first indicator data is retrieved from the scaling strategy attribute value lookup table; wherein, the scaling strategy attribute value lookup table pre-stores the correspondence between the indicator data range of the first weight indicator and the scaling strategy attribute value, and the scaling strategy attribute value is used to characterize the scaling speed.

[0098] Update the scaling-down strategy attribute value in the HPA object using the target scaling-down strategy attribute value.

[0099] Secondly, embodiments of this application provide an apparatus for dynamic resource scaling and quota limiting of a cloud load balancer, the apparatus comprising:

[0100] The acquisition module is used to periodically poll and acquire business data and resource utilization for the current maximum number of replicas of the computing Pod. The business data includes first indicator data of a first weight indicator and second indicator data of a second weight indicator. The first weight indicator and the second weight indicator are business indicators that do not need to be adapted to the HPA controller.

[0101] The first determining module is used to determine the basic maximum number of replicas based on the first indicator data and the resource utilization rate;

[0102] The second determining module is used to dynamically adjust the basic maximum number of replicas based on the second indicator data through a hierarchical threshold triggering mechanism to determine the final maximum number of replicas.

[0103] An update module is used to update the maximum number of HPA replicas in the HPA object using the final maximum number of replicas.

[0104] Thirdly, embodiments of this application provide an electronic device, comprising: a processor and a memory, wherein the processor is configured to execute a program stored in the memory for dynamic scaling and quota limiting of resources of a cloud load balancer, so as to implement the above-described method for dynamic scaling and quota limiting of resources of a cloud load balancer.

[0105] Fourthly, embodiments of this application provide a storage medium storing one or more programs, which can be executed by one or more processors to implement the above-mentioned method for dynamic resource scaling and quota limiting of a cloud load balancer.

[0106] This application provides a method, apparatus, electronic device, and storage medium for dynamic resource scaling and quota limiting in a cloud load balancer. The method includes: periodically polling to obtain business data and resource utilization for currently calculating the maximum replica count of a Pod; the business data includes first indicator data of a first weight indicator and second indicator data of a second weight indicator; determining the basic maximum replica count based on the first indicator data and resource utilization; dynamically adjusting the basic maximum replica count according to the second indicator data through a tiered threshold triggering mechanism to determine the final maximum replica count; and updating the maximum replica count of the HPA object using the final maximum replica count. This solution directly uses business indicator data to participate in HPA replica count decision-making, completely avoiding the architectural limitations of traditional solutions that rely on adapters. Its core technological breakthrough lies in: calculating the final maximum replica count based on business indicator data such as the first and second weight indicators, and dynamically updating the maximum replica count of the HPA object. The entire process completely skips the intermediate adaptation step of converting business indicators into native HPA indicators, achieving seamless integration between business indicators and the elastic scaling system. This direct-connect design not only eliminates the cost of adapter development and maintenance, but also effectively avoids architectural redundancy, significantly reducing deployment configuration and version compatibility maintenance costs.

[0107] Furthermore, the calculation of the maximum number of replicas is optimized through a hierarchical decision-making mechanism. First, a basic maximum number of replicas is generated based on resource utilization and a first-weighted indicator, and then this value is dynamically adjusted using a second-weighted indicator. This multi-indicator collaborative scheduling design can effectively overcome the problem of frequent fluctuations in the number of Pods caused by the independent triggering of each indicator in traditional methods, significantly improving the stability of scaling up and down, and further optimizing business performance. Attached Figure Description

[0108] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0109] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0110] Figure 1 A flowchart illustrating an embodiment of a method for dynamic resource scaling and quota limiting of a cloud load balancer provided in this application;

[0111] Figure 2 A flowchart illustrating an embodiment of another method for dynamic resource scaling and quota limiting of a cloud load balancer provided in this application;

[0112] Figure 3 A flowchart illustrating an embodiment of another method for dynamic resource scaling and quota limiting of a cloud load balancer provided in this application;

[0113] Figure 4 A flowchart illustrating an embodiment of another method for dynamic resource scaling and quota limiting of a cloud load balancer provided in this application;

[0114] Figure 5 A flowchart illustrating an embodiment of another method for dynamic resource scaling and quota limiting of a cloud load balancer provided in this application;

[0115] Figure 6 A flowchart illustrating an embodiment of another method for dynamic resource scaling and quota limiting of a cloud load balancer provided in this application;

[0116] Figure 7 A flowchart illustrating an embodiment of another method for dynamic resource scaling and quota limiting of a cloud load balancer provided in this application;

[0117] Figure 8 A block diagram illustrating an embodiment of a device for dynamic resource scaling and quota limiting of a cloud load balancer provided in this application.

[0118] Figure 9 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0119] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0120] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0121] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0122] The embodiments of this application can acquire and process relevant data based on artificial intelligence technology. Artificial intelligence is the theory, method, technology, and application system that uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results.

[0123] Foundational technologies in artificial intelligence generally include sensors, dedicated AI chips, cloud computing, storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies mainly encompass computer vision, robotics, biometrics, speech processing, natural language processing, and machine learning / deep learning.

[0124] It should be understood that the sequence number of each step in the following embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0125] To facilitate understanding of the embodiments of this application, the following will provide further explanation and description with reference to the accompanying drawings and specific embodiments. These embodiments do not constitute a limitation on the embodiments of this application.

[0126] This application provides a method for dynamic resource scaling and quota limiting of a cloud load balancer. This method is applied to the HPAPLUS controller. (See also...) Figure 1 , Figure 1 This is a flowchart illustrating an embodiment of a method for dynamic resource scaling and quota limiting of a cloud load balancer provided in this application. Figure 1 The process shown may include the following steps:

[0127] Step 101: Periodically poll to obtain business data and resource utilization for the current maximum number of replicas of the computing Pod. The business data includes the first indicator data of the first weight indicator and the second indicator data of the second weight indicator.

[0128] The primary weighted metric directly reflects service traffic pressure and is the main basis for triggering scaling up or down. Typical candidate metrics include QPS or CPS (Connections Per Second). QPS reflects the actual request throughput processed by the system, is linearly related to user access behavior, and directly reflects business load fluctuations. CPS, on the other hand, characterizes the server's connection establishment capability and is particularly important for handling short-connection services. In this embodiment, QPS or CPS should be selected as the primary weighted metric based on the product characteristics (short-connection request-intensive or long-connection maintenance-intensive).

[0129] Resource utilization is a comprehensive indicator for evaluating the resource consumption of a Pod on a node. It mainly includes the following three dimensions: CPU utilization, which reflects the real-time consumption of computing resources by the Pod; memory utilization, which characterizes the memory consumption level of the Pod during runtime; and the number of Pods, which reflects the scale of Pod instances deployed on the node. Together, these three constitute the quantitative basis for resource pressure.

[0130] The second weighted metric serves as an auxiliary evaluation basis for service quality. Typical candidate metrics include ACC (Accuracy of Access) or BPS (Bits Per Second). ACC reflects continuous connection pressure but has a lag (e.g., connection pool reuse can distort the metric); BPS reflects network bandwidth usage but is significantly affected by data compression. Similarly, ACC or BPS should be selected as the second weighted metric based on product characteristics. In this embodiment, both the first and second weighted metrics are business metrics that do not require adaptation to the HPA controller. That is, in this application, there is no need to use an adapter to adapt the business metrics to the HPA controller, and the maximum number of Pod replicas can be dynamically determined through the collaborative analysis of the three types of data, so as to update and correct the maximum number of replicas in the HPA object.

[0131] Step 102: Determine the basic maximum number of replicas based on the first indicator data and resource utilization rate;

[0132] As described above, resource utilization, as a core quantitative indicator of node load, directly reflects the real-time pressure status of instances and is a key basis for triggering replica number adjustments to achieve load balancing. Meanwhile, the primary weight indicator, as the highest priority measure of business health, will trigger scaling operations first when it becomes abnormal. Therefore, by comprehensively analyzing the primary weight indicator's primary indicator data and resource utilization, the basic maximum number of replicas can be calculated more accurately, thereby improving the accuracy of scaling decisions.

[0133] The aforementioned basic maximum replica count is a theoretical expansion limit calculated jointly by the first metric data (such as QPS / CPS) and resource utilization. Simply put, this basic maximum replica count is calculated in real-time by the HPA PLUS controller based on the current workload (QPS / CPS) and node resource availability (CPU utilization, memory utilization, number of Pods), and is an intermediate result without correction by the second weighted metric (ACC / BPS).

[0134] Step 103: Based on the second indicator data, dynamically adjust the basic maximum number of replicas through a tiered threshold triggering mechanism to determine the final maximum number of replicas;

[0135] Since the basic maximum replica count is calculated only considering traffic and resources, there may be the following biases: connection pool bottleneck (ACC) is not taken into account or network bandwidth limitation (BPS) is ignored. Therefore, the basic maximum replica count needs to be dynamically adjusted through the hierarchical threshold mechanism of the second weight indicator to form a more accurate expansion decision and obtain the final maximum replica count.

[0136] As described above, since ACC and BPS, as the second weighted indicators, focus on connection status and traffic dimensions respectively, and both have certain limitations, such indicator data can usually help correct the decision bias of the maximum replica count.

[0137] Specifically, a multi-level threshold mechanism is set up for the second weighting metric. When different levels of thresholds are triggered, the system will adopt corresponding basic maximum replica count adjustment strategies. It should be noted that the default behavior of this strategy is to maintain the basic maximum replica count unchanged if no threshold is triggered, that is, to determine the basic maximum replica count as the final maximum replica count. The specific implementation details of this strategy will be described in detail in the subsequent implementation examples section, and will not be discussed further here.

[0138] Step 104: Update the maximum number of HPA replicas in the HPA object using the final maximum number of replicas.

[0139] The maximum number of HPA replicas is determined by the HPA controller through elastic scaling based on the thresholds set by the CPU / memory.

[0140] In this embodiment, the process of updating the maximum replica count of the HPA object using the final maximum replica count is as follows: The HPAPLUS controller directly modifies the maxReplicas field of the HPA object through the Kubernetes (k8s) API (Application Programming Interface) to correct the maximum replica count of the HPA determined by the HPA controller. When it is confirmed that the new configuration has taken effect and the scaling behavior is as expected, the HPAPLUS controller will immediately recalculate the final maximum replica count according to the method provided in this embodiment.

[0141] This solution directly incorporates business metrics into HPA replica count decisions, completely bypassing the architectural limitations of traditional solutions that rely on adapters. Its core technological breakthrough lies in calculating the final maximum replica count based on business metrics such as the first and second weighted metrics, and dynamically updating the maximum replica count of HPA objects. The entire process completely skips the intermediate adaptation step of converting business metrics into native HPA metrics, achieving seamless integration between business metrics and the elastic scaling system. This direct-connection design not only eliminates adapter development and maintenance costs but also effectively avoids architectural redundancy, significantly reducing deployment, configuration, and version compatibility maintenance costs.

[0142] Furthermore, the aforementioned technical solution optimizes the calculation of the maximum number of replicas through a hierarchical decision-making mechanism. First, a basic maximum number of replicas is generated based on resource utilization and a primary weighting indicator. Then, this value is dynamically adjusted using a secondary weighting indicator. This multi-indicator collaborative scheduling design effectively overcomes the problem of frequent Pod count fluctuations caused by independent triggering of various indicators in traditional methods, significantly improving scaling stability and further optimizing business performance.

[0143] like Figure 2 As shown, as an optional implementation, the method described above, step 101, which involves periodically polling to obtain business data and resource utilization for the current maximum number of replicas of the computing Pod, includes the following steps:

[0144] Step 201: Obtain the first resource utilization rate sequence collected using the current time window;

[0145] A sliding time window mechanism is required (the default window size is 5 sampling points, which can be configured) to collect the latest data as follows:

[0146] 1. Record CPU utilization and memory utilization at fixed intervals to form a time series; 2. Synchronously obtain the number of Pods in the Ready state in the Kubernetes cluster at each sampling time.

[0147] The CPU utilization and memory utilization mentioned above are spatiotemporally aligned with the number of Pods at the corresponding time points to construct a first resource utilization sequence (ResourceUtilizationRatio1[Ti....Tj]) containing multi-dimensional indicator data, so that the HPAPLUS controller can obtain it.

[0148] Step 202: Extract the start and end timestamps of the time window;

[0149] In order to query and obtain the first and second indicator data of the sequence, in this embodiment, it is necessary to obtain the start and end timestamps accurate to the second.

[0150] Step 203: Determine whether the time span of the start and end timestamps is less than the preset time span;

[0151] The time span is determined by the time difference between the start timestamp (startTime) and the end timestamp (endTime). The preset time span can be set according to actual needs and is not limited here, such as 300 seconds, 350 seconds, 400 seconds, etc.

[0152] If the time span of the start and end timestamps is less than the preset time span, to avoid monitoring values ​​lagging and resulting in too little data for the first and second indicators, step 204 can be executed to obtain the first data sequence; if the time span of the start and end timestamps is equal to or greater than the preset time span, step 205 can be executed to obtain the first data sequence.

[0153] Step 204: Query the cluster monitoring system based on a preset time span to obtain the current first data sequence;

[0154] If endTime - startTime < 300 (preset time span), the time query range is directly extended to 300 seconds, i.e., startTime = endTime - 300. Based on the extended time span, i.e., the preset time span, the cluster monitoring system (such as Prometheus) is queried to obtain multiple first indicator data and multiple second indicator data. Among the obtained multiple first indicator data and second indicator data, monitoring points that are less than the initial threshold (e.g., the default value of QPS: 1000) are filtered out. This threshold is determined by the initial parameter and is generally kept at the default value. At the same time, if there are too many first indicator data and second indicator data, the last 8 (default value, adjustable) time points of first indicator data and second indicator data can be selected to construct the current first data sequence (Value1[Tn....Tm]) for the HPA PLUS controller to obtain.

[0155] Step 205: Query the cluster monitoring system based on the time span of the start and end timestamps to obtain the current first data sequence;

[0156] After retrieving multiple first and second indicator data based on the time span of the start and end timestamps, threshold filtering and excessive data removal are also required to construct the current first data sequence for the HPAPLUS controller to obtain.

[0157] Step 206: Time-align the first resource utilization rate sequence and the first data sequence to determine the second resource utilization rate sequence and the second data sequence;

[0158] Currently, two sequences, ResourceUtilizationRatio1[Ti....Tj] and Value1[Tn....Tm], are obtained. The time points of the two sequences are intersected to obtain a common time range [Tx...Ty]. The resource utilization rate sequence within this range is taken as the second resource utilization rate sequence ResourceUtilizationRatio2 for subsequent calculations. Similarly, multiple first indicator data and multiple second indicator data close to this range are selected from Value1 as the second data sequence Value2 for subsequent calculations.

[0159] Step 207: Obtain resource utilization rate based on the second resource utilization rate sequence, and obtain first indicator data of the first weight indicator and second indicator data of the second weight indicator based on the second data sequence.

[0160] As described above, resource utilization includes the current number of Pods, current memory utilization, and current CPU utilization. The process of obtaining resource utilization based on the second resource utilization sequence can be achieved through steps A1 to A7:

[0161] Step A1: Obtain the latest Pod count from the second resource utilization sequence;

[0162] Specifically, the latest Pod count can be obtained from the second resource utilization sequence using the following call statement:

[0163] `lastPodNum = resourceUtilizationRatio2[len(resourceUtilizationRatio2)-1].countPods`. Here, `len(resourceUtilizationRatio2)-1` represents the index of the last element of the second resource utilization sequence, `resourceUtilizationRatio2[...]` represents accessing the last element of the second resource utilization sequence by index, and `.countPods` represents accessing the `countPods` attribute or field of this second resource utilization sequence to get the latest Pod count.

[0164] Step A2: Determine the average number of Pods and the standard deviation of Pods based on the second resource utilization sequence;

[0165] The average number of Pods can reflect the long-term load level, while the standard deviation of Pods can quantify the volatility of the number of Pods.

[0166] Specifically, when determining the average number of Pods, it is necessary to obtain the total number of Pods (totalPods) and the resource statistics dimension (resourceLength, e.g., the length of a time period) from the second resource utilization sequence, and use the formula averagePods = totalPods / resourceLength to calculate the average number of Pods for each unit dimension (e.g., per hour).

[0167] When determining the standard deviation of a Pod, you can use the following statement: stdevPods = STDEV(resourceUtilizationRatio2.countPods).

[0168] Here, `resourceUtilizationRatio2.countPods` represents the `countPods` field values ​​extracted from all elements of the second resource utilization rate sequence, forming a sequence of Pod counts. Example: If the array contains 3 records with `countPods` values ​​of [10, 15, 12], the extracted sequence will be [10, 15, 12]. `STDEV()` represents the Standard Deviation function, used to calculate the standard deviation of the extracted sequence to obtain the Pod standard deviation.

[0169] Step A3: Determine whether the noise filtering mechanism based on 2σ standard deviation is satisfied based on the latest number of Pods, the average number of Pods, and the standard deviation of Pods.

[0170] The latest Pod count is detected as a noise point using a 2σ standard deviation noise filtering mechanism. If the 2σ standard deviation noise filtering mechanism is satisfied (i.e., |lastPodNum-averagePods|≤2*stdevPods), the latest Pod count is not a noise point, and step A4 is executed. If the 2σ standard deviation noise filtering mechanism is not satisfied (i.e., |lastPodNum-averagePods|>2*stdevPods), the latest Pod count is a noise point, and step A5 is executed.

[0171] Step A4: Determine the latest Pod count as the current Pod count;

[0172] Step A5: Determine the latest Pod count from the previous runtime as the current Pod count;

[0173] When the latest Pod count is determined to be a noise point, it is necessary to roll back to the Pod count of the previous running time [currentPods=resourceUtilizationRatio2[len(resourceUtilizationRatio2)-2].countPods)]. The above process is the specific process for smoothing the Pod count. Its core purpose is to eliminate the interference caused by the fluctuation of the Pod count through technical means, so as to avoid outliers affecting the stability and reliability of scaling decisions.

[0174] Step A6: Obtain the latest memory utilization rate from the second resource utilization rate sequence, and determine the latest memory utilization rate as the current memory utilization rate;

[0175] Similarly, the latest memory utilization can be obtained from the second resource utilization sequence using the following call statement:

[0176] latestMemory=resourceUtilizationRatio2[len(resourceUtilizationRatio2)-1].MemoryUtilizationRatio.

[0177] Here, len(resourceUtilizationRatio2)-1 represents the index of the last element of the second resource utilization sequence, resourceUtilizationRatio2[...] represents accessing the last element of the second resource utilization sequence by index, and .MemoryUtilizationRatio represents accessing the memory attribute or field of the second resource utilization sequence to obtain the latest memory utilization.

[0178] After obtaining the latest memory utilization rate, perform a numerical conversion on it to get the current memory utilization rate. The numerical conversion process can be implemented using the following statement:

[0179] currentMemory=float64(latestMemory) / 100.0.

[0180] Step A7: Obtain the latest CPU utilization from the CPU utilization sequence and determine the latest CPU utilization as the current CPU utilization.

[0181] Similarly, the latest memory utilization can be obtained from the second resource utilization sequence using the following call statement:

[0182] latestCpu=resourceUtilizationRatio2[len(resourceUtilizationRatio2)-1].CpuUtilizationRatio.

[0183] Here, len(resourceUtilizationRatio2)-1 represents the index of the last element of the second resource utilization sequence, resourceUtilizationRatio2[...] represents accessing the last element of the second resource utilization sequence by index, and .CpuUtilizationRatio represents accessing the CPU attribute or field of the second resource utilization sequence to obtain the latest CPU utilization.

[0184] After obtaining the latest CPU utilization, a numerical conversion is performed on it to obtain the current CPU utilization. The numerical conversion process can be implemented using the following statement:

[0185] currentCpu=float64(latestCpu) / 100.0.

[0186] The process of obtaining the first indicator data of the first weight indicator and the second indicator data of the second weight indicator based on the second data sequence can be achieved through steps B1 to B2:

[0187] Step B1: Obtain the latest first indicator data from the second data sequence, and determine the latest first indicator data as the first indicator data of the first weight indicator;

[0188] Specifically, the latest first indicator data can be obtained from the second data sequence using the following call statement (taking QPS as the first weighted indicator as an example):

[0189] currentQps = Value2[countQps-1].Value; where Value2 represents the second data sequence, which stores multiple QPS data recorded in chronological order. Each element contains a Value attribute (representing the QPS value), countQps-1 is the index of the latest data (array index starts from 0), and .Value indicates that the Value attribute of the element at that index position is extracted. The specific QPS value is determined by the latest QPS obtained as the first indicator data.

[0190] Step B2: Obtain the latest second indicator data from the second data sequence, and determine the latest second indicator data as the second indicator data of the second weight indicator.

[0191] Similarly, the latest second indicator data can be obtained from the second data sequence using the following call statement (taking ACC as the second weighted indicator as an example):

[0192] `currentAcc = Value2[countAcc-1].Value`; where `Value2` represents the second data sequence, which also stores multiple ACC data recorded in chronological order. Each element contains a `Value` attribute (representing the ACC value), `countQps-1` is the index of the latest data (array indices start from 0), and `.Value` indicates that the `Value` attribute of the element at that index is extracted. The specific ACC value is determined by the latest ACC obtained as the second indicator data. It should be noted that the acquisition logic of the first and second indicator data is independent and can be acquired in parallel or sequentially; no limitation is made here.

[0193] In all the embodiments described below, QPS is used as the first weight index to illustrate its complete calculation process, while the calculation process of CPS as the first weight index is the same, and the calculation process of CPS will not be described in detail here.

[0194] like Figure 3 As shown, as an optional implementation, the method described above, step 102, which determines the basic maximum number of replicas based on the first indicator data and resource utilization, includes the following steps:

[0195] Step 301: Obtain the maximum threshold, memory utilization threshold, and CPU utilization threshold of the first weighted index;

[0196] Among them, the maximum metric threshold represents the maximum QPS value set by the instance specification; the CPU utilization threshold (a.cpuRatio) and memory utilization threshold (a.memoryRatio) are set by the corresponding instance specification parameters, generally 0.9 and 0.4 respectively, representing the maximum threshold of CPU and memory utilization that a single Pod can be allowed to run for a long time.

[0197] Step 302: Determine the maximum number of CPU replicas based on the current number of Pods, the maximum metric threshold, the first metric data, the CPU utilization threshold, and the current CPU utilization.

[0198] The maximum number of CPU replicas can be calculated using the following formula:

[0199]

[0200] Where cpuMaxReplicas represents the maximum number of CPU replicas, currentPods represents the current number of Pods, targetQps represents the maximum metric threshold, currentQps represents the current QPS (the first metric), a.cpuRatio represents the CPU utilization threshold, and currentCpu represents the current CPU utilization.

[0201] Step 303: Determine the maximum number of memory replicas based on the current number of Pods, the maximum metric threshold, the first metric data, the memory utilization threshold, and the current memory utilization.

[0202] The maximum number of memory replicas can be calculated using the following formula:

[0203]

[0204] Among them, memoryMaxReplicas represents the maximum number of replicas in memory, currentPods represents the current number of Pods, targetQps represents the maximum metric threshold, currentQps represents the current QPS (the first metric), a.memoryRatio represents the memory utilization threshold, and currentMemory represents the current memory utilization.

[0205] Step 304: Select the maximum value between the maximum number of CPU replicas and the maximum number of memory replicas, and round the maximum value up to obtain the basic maximum number of replicas.

[0206] `maxReplicas = max(cpuMaxReplicas, memoryMaxReplicas)` takes the maximum of the calculated values ​​for the maximum number of replicas on the CPU and the maximum number of replicas in memory. Then, `maxReplicasInt32 = int32(math.Ceil(maxReplicas)` converts the values ​​and rounds them up to get the base maximum number of replicas.

[0207] like Figure 4 As shown, as an optional implementation, as described in the aforementioned method, step 103, which dynamically adjusts the basic maximum replica count based on the second indicator data through a tiered threshold triggering mechanism to determine the final maximum replica count, includes the following steps:

[0208] Step 401: Determine the value per core based on the second metric data and the current number of Pods;

[0209] The total number of cores is determined based on the current number of Pods. Then, the second metric data threshold is divided by the total number of cores to obtain the value per core.

[0210] Step 402: Detect whether the value per core unit exceeds the preset indicator data threshold range;

[0211] The preset metric data threshold range is a data range consisting of pre-defined stress and limit values ​​for each core; the stress and limit values ​​are pre-set values ​​based on the type and specifications of different listeners. The purpose of setting the preset metric data threshold range is to enable tiered threshold triggering to dynamically adjust the base maximum replica count.

[0212] Specifically, if the value per core is detected to be within the preset metric data threshold range, it is necessary to combine the first weight metric and decide whether to increase the number of Pods according to different strategies, and steps 403 to 405 need to be executed; if the value per core is detected to exceed the preset metric data threshold range, the maximum number of replicas should be modified immediately, that is, the number of Pods should be increased, and step 406 needs to be executed; if the value per core is detected to be lower than the preset metric data threshold range, the impact of the second weight metric on the basic maximum number of replicas should be ignored, and step 407 needs to be executed.

[0213] Step 403: Obtain the real-time rate of change of the first weighted indicator;

[0214] The real-time rate of change can intuitively reflect the dynamic fluctuation characteristics of the primary weighted indicator. Especially when the secondary indicator data is above the pressure value but below the limit value, this rate of change can accurately quantify the abnormal fluctuation trend of the primary weighted indicator. This correlation analysis provides data support for deciding whether to increase the number of Pods.

[0215] Step 404: Detect whether the real-time rate of change exceeds the preset rate of change threshold;

[0216] The preset rate of change threshold is a pre-set response boundary value used to determine whether fluctuations in the first weighted indicator trigger the intervention mechanism; the specific preset rate of change threshold can be set according to actual needs and is not limited here.

[0217] If the real-time change rate is detected to exceed the preset change rate threshold, it indicates that the current traffic has exceeded the carrying capacity of the maximum number of basic replicas, and step 405 needs to be executed immediately to adjust the quantity.

[0218] If the real-time change rate is not detected to exceed the preset change rate threshold, it indicates that the current traffic has not exceeded the carrying capacity of the basic maximum number of replicas, and step 407 should be executed immediately without quantity adjustment.

[0219] Step 405: Adjust the basic maximum number of replicas according to the preset adjustment number of Pods, and determine the adjusted basic maximum number of replicas as the final maximum number of replicas;

[0220] Specifically, a preset adjustment number of Pods can be added to the base maximum number of replicas, and the base maximum number of replicas for the added Pods can be determined as the final maximum number of replicas.

[0221] For example, if the base maximum number of replicas is 10 and the preset Pod count is 2 (which can be adjusted), then the final maximum number of replicas will be 12.

[0222] Step 406: Modify the basic maximum replica count based on the second indicator data, and determine the modified basic maximum replica count as the final maximum replica count;

[0223] The specific data modification process can be achieved through steps C1 to C3:

[0224] Step C1: Determine the required number of cores based on the unit value per core and the limit value;

[0225] Divide the per-core unit value by the limit value, and round the result up to determine the required number of cores. For example, if the per-core unit value is 500 and the limit value is 200, then the required number of cores is 3.

[0226] Step C2: Determine the number of replicas to add based on the required number of cores;

[0227] In this embodiment, each Pod is pre-set to have the same number of cores. Therefore, the number of replicas can be determined by the required number of cores and the number of cores included in the Pod. For example, if a Pod is set to have 2 cores, and the required number of cores is 3, then the number of replicas will be 2. If a Pod is set to have 3 or more cores, and the required number of cores is 3, then the number of replicas will be 1.

[0228] Step C3: Modify the base maximum number of replicas based on the number of replicas added.

[0229] The number of replicas added is added to the base maximum number of replicas, and the result is the final maximum number of replicas.

[0230] Step 407: Determine the basic maximum number of replicas as the final maximum number of replicas.

[0231] When the per-core unit value is lower than the pressure value, or when the real-time change rate does not exceed the preset change rate threshold, it indicates that the current traffic has not exceeded the carrying capacity of the basic maximum replica number. There is no need to adjust the number of the baseline maximum replica number or modify the data. Setting the basic maximum replica number as the final maximum replica number can meet the business performance requirements.

[0232] This multi-indicator collaborative decision-making mechanism effectively overcomes the problem of frequent fluctuations in the number of Pods caused by the independent triggering of each indicator in traditional methods, significantly improves the stability of scaling up and down, and further optimizes business performance.

[0233] In practical applications, after determining the final maximum replica count, the HPA object properties are not immediately updated. Instead, they are placed in a recommendation array, which serves as the election window. The recommendation array typically retains historical recommendation values, and the decision to update the HPA object properties is based on all the recommendation values ​​in the array. This effectively avoids frequent fluctuations in the replica count caused by instantaneous metric fluctuations. Therefore, before updating the maximum replica count of the HPA object using the final maximum replica count, further steps are required. Figure 5 The embodiments shown are as follows: Figure 5 The specific steps shown are as follows:

[0234] Step 501: Based on the first metric data, the current number of Pods, the current CPU utilization, the current memory utilization, the maximum metric threshold, the CPU utilization threshold, the memory utilization threshold, and the maximum number of HPA replicas, determine the scaling flag value corresponding to the final maximum number of replicas.

[0235] The scaling flag (isDown) is a key status flag used to indicate the direction of scaling decisions. Specifically, a scaling flag value of 0 indicates scaling up; a scaling flag value greater than 1 indicates scaling down; and a scaling flag value of 1 indicates no change.

[0236] The specific process of determining the value of the stretch mark can be achieved through steps D1 to D4:

[0237] Step D1: Determine the CPU utilization of a single Pod based on the first indicator data, the current number of Pods, and the current CPU utilization; and determine the memory utilization of a single Pod based on the first indicator data, the current number of Pods, and the current memory utilization.

[0238] The maximum QPS that each Pod can currently handle is calculated based on CPU / memory ratio, which is the CPU utilization of a single Pod / the memory utilization of a single Pod.

[0239] The CPU utilization of a single Pod can be calculated using the following formula:

[0240] perPodCpuRate=currentQps / (currentPods×currentCpu);

[0241] PeerPodCpuRateb represents the CPU utilization of a single Pod.

[0242] The memory utilization of a single Pod can be calculated using the following formula:

[0243] perPodMemoryRate=currentQps / (currentPods×currentMemory);

[0244] Among them, perPodMemoryRate represents the memory utilization rate of a single Pod.

[0245] Step D2: Determine the incremental demand value of memory Pods based on single Pod memory utilization, first indicator data, and maximum indicator threshold; and determine the incremental demand value of CPUPods based on single Pod CPU utilization, first indicator data, and maximum indicator threshold.

[0246] The incremental memory / CPU Pod requirement values ​​represent how many more Pods are needed to reach the currently set QPS, based on memory or CPU usage.

[0247] The incremental memory requirement for a Pod can be calculated using the following formula:

[0248] memoryPods=(currentQps-targetQps) / perPodMemoryRate;

[0249] Among them, memoryPods represents the incremental demand value of memory Pods.

[0250] The incremental demand value for CPUPod can be calculated using the following formula:

[0251] cpuPods=(currentQps-targetQps) / perPodCpuRate;

[0252] Here, cpuPods represents the incremental demand value for CPUPods.

[0253] Step D3: Determine the remaining available Pod capacity value for CPU based on the CPU utilization threshold, current CPU utilization, current number of Pods, and maximum number of HPA replicas; and determine the remaining available Pod capacity value for memory based on the memory utilization threshold, current memory utilization, current number of Pods, and maximum number of HPA replicas.

[0254] The CPU remaining available Pod capacity value / memory remaining available Pod capacity value are used to indicate how many Pods are still available in CPU / memory, calculated based on the maximum number of HPA replicas.

[0255] The remaining available Pod capacity using CPU can be calculated using the following formula:

[0256] remainderCpuPods=(a.cpuratio-currentCpu)×currentPods+

[0257] oldMaxReplicas–currentPods;

[0258] Here, `remainderCpuPods` represents the remaining available Pod capacity for CPU, and `oldMaxRepl icas` represents the maximum number of HPA replicas.

[0259] The remaining available Pod capacity can be calculated using the following formula:

[0260] remainderMemoryPods=(a.memoryratio-currentMemory)×currentPods+

[0261] oldMaxReplicas-currentPods;

[0262] Here, `remainderMemoryPods` represents the remaining available Pod capacity in terms of memory.

[0263] Step D4: Determine the scaling flag value corresponding to the final maximum number of replicas based on the final maximum number of replicas, the maximum number of HPA replicas, the incremental demand value of memory Pods, the incremental demand value of CPU Pods, the remaining available Pod capacity value of CPU, the remaining available Pod capacity value of memory, the first indicator data, the current CPU utilization, the current number of Pods, and the latest number of Pods.

[0264] The process of determining the scaling flag value corresponding to the final maximum number of replicas based on the above data can be achieved through steps E1 to E18:

[0265] Step E1: Check if the final maximum number of replicas is the same as the latest number of Pods;

[0266] Step E2: If the final maximum number of replicas is the same as the latest number of Pods, check whether the final maximum number of replicas is selected from the maximum number of CPU replicas.

[0267] If the final maximum number of replicas is the same as the latest number of Pods, it means that the number of Pods has not changed within the [Tx...Ty] time window for calculating the value. In this case, we divide the selected base maximum number of replicas into two categories based on whether it comes from CPU or memory. In each category, we further divide it into two cases; a total of 4 cases, as described in the first to fourth conditions below. In all four cases, the isDown scaling flag is set to 3, and the final maximum number of replicas continues to remain the latest number of Pods.

[0268] Step E3: If the final maximum number of replicas is selected as the maximum number of CPU replicas, determine whether the first condition or the second condition is met based on the CPUPod incremental demand value and the CPU remaining available Pod capacity value.

[0269] The first condition is: cpuPods >= 0 && cpuPods <= 1.0 && residualCpuPods <= 1.0

[0270] If the first condition is met, it means that the current Qps has just exceeded the target maximum threshold, targetQps, the number of Pods exceeding it is less than 1, and the number of remaining available Pods is also less than 1.

[0271] The second condition is: cpuPods<0&&remainderCpuPods>=math.Abs(cpuPods)&&remainderCpuPods<=math.Abs(cpuPods)+2; where math.Abs(cpuPods) represents the absolute value of the incremental demand for CPUPods.

[0272] If the second condition is met, it means that the current Qps has not yet reached the target maximum threshold, targetQps, the remaining available Pods can cover the required number of Pods, and the number of excess Pods does not exceed 2.

[0273] Step E4: If the first condition or the second condition is met, set the value of the scaling flag to the first value;

[0274] If the first or second condition is met, it indicates that shrinkage processing is required. Therefore, the value of the scaling flag can be set to the first data (such as 3), i.e., isDown = 3.

[0275] Step E5: If the final maximum number of replicas is selected as the maximum number of replicas in memory, determine whether the third or fourth condition is met based on the incremental demand value of memory Pods and the remaining available capacity of memory Pods.

[0276] The third condition is:

[0277] memoryPods>=0&&memoryPods<=1.0&&remainderMemoryPods<=1.0

[0278] The explanation is the same as the first condition above; the calculated value comes from memory, so it will not be elaborated here.

[0279] The fourth condition is:

[0280] memoryPods<0&&remainderMemoryPods>=math.Abs(memoryPods)&&remainderMemoryPods<=math.Abs(memoryPods)+2;

[0281] The explanation is the same as the second condition above; the calculated value comes from memory, so it will not be elaborated here.

[0282] If the third or fourth condition is met, in step E6, set the value of the scaling flag to the first value.

[0283] If the third or fourth condition is met, it means that shrinkage processing is also required. Therefore, the value of the scaling flag can also be set to the first value, i.e., isDown=3.

[0284] Step E7: If the final maximum number of replicas is not the same as the latest number of Pods, determine the average data of the first indicator based on the second data sequence.

[0285] That is, obtain the data of each first indicator from the second data sequence, and then perform an average calculation on all the first indicator data to obtain the average data of the first indicator.

[0286] Step E8: Determine whether the fifth condition is met based on the first indicator data and the average data of the first indicator, or determine whether the sixth condition is met based on the current number of Pods and the current number of Pods at the previous running time.

[0287] The fifth condition is: the data for the first indicator is less than the average data for the first indicator calculated previously, and the decrease is greater than 5%;

[0288] The sixth condition is: the current Pods value is less than 2 compared to the current Pods value at the previous runtime.

[0289] Step E9: If the fifth or sixth condition is met, set the value of the scaling flag to the second value.

[0290] If the fifth or sixth condition is met, it means that shrinkage processing is also required. Therefore, the value of the scaling flag can also be set to the second data (such as 4), i.e., isDown=4.

[0291] In the fifth and sixth conditions, there is a special case: when the client request pressure is sufficient and the CPU of all Pods is almost fully utilized, but it is still far from reaching the maximum targetQps, the monitoring will show that the value fluctuates back and forth. Due to the logic setting of isDown=4, it will be impossible to expand the capacity in time for a period of time. Therefore, the following operations are adopted to optimize the handling of this situation.

[0292] During each calculation, the current CPU utilization in the last resource monitoring point ResourceUtilizationRatio2 is taken and it is checked whether it is greater than 80%. If it is, record.highPressure++ is recorded. If it is less than 80% even once, record.highPressure = 0 is set. If record.highPressure > 3, the logic of setting isDown = 4 is skipped, and the subsequent isDowns value setting logic continues.

[0293] Step E10: If the fifth and sixth conditions are not met, check whether the current CPU utilization is less than the preset pressure threshold.

[0294] The preset pressure threshold (e.g., 0.55) can be set according to actual needs, but will not be set here.

[0295] Step E11: If the current CPU utilization is detected to be less than the preset pressure threshold, set the scaling flag value to the third value.

[0296] When the current CPU utilization is detected to be less than the preset pressure threshold, it means that the pressure is not enough. Set the scaling flag value to the third value (such as 5), i.e., isDown=5.

[0297] In practical applications, if the current CPU utilization is less than the preset pressure threshold, or if the fifth or sixth condition is met, a stopScaleOut flag is set. If the final calculated maximum replica count is greater than or equal to the HPA maximum replica count, scaling up stops. This is because when QPS drops significantly in a short period, the native HPA scaling down requires a 5-minute wait (set by the --horizontal-pod-autoscaler-downscale-stabilization-window parameter) before starting, resulting in little change in the current Pods. This causes a significant decrease in perPodCpuRate and perPodMemoryRate, while the corresponding final maximum replica count increases substantially. If the previous QPS was just around the target QPS (e.g., 50k), another surge of traffic within the scaling down time window can easily break the 50k QPS limit, pushing the QPS to 60k-70k. After stabilizing for a period, the calculated final maximum replica count will decrease again, but it will be less than the current number of Pods, making proactive scaling down impossible; it can only maintain its original value. At this point, the monitoring value lossPods = final maximum number of replicas - latest number of Pods, representing the number of Pods lost at this time.

[0298] During the above process, the perPodMemoryRate calculation value decreased most significantly. This is because when QPS pressure decreases, CPU utilization may drop, but the average memory utilization remains relatively constant. Since scaling down requires a certain stabilization time, the currentPods value remains essentially unchanged, leading to a very large calculated memoryMaxReplicas, thus inflating the final maximum replica count. Marking it with the stopScaleOut flag and stopping scaling up are optimization methods employed.

[0299] Step E12: If the current CPU utilization is not less than the preset pressure threshold, check whether the final maximum number of replicas is less than the latest number of Pods.

[0300] Step E13: If the final maximum number of replicas is detected to be less than the latest number of Pods, set the scaling flag value to the fourth value.

[0301] When the final maximum number of replicas is detected to be less than the latest number of Pods, it means that after the Pods are scaled up, the QPS tends to stabilize and the reached value exceeds the target QPS. The calculated perPodCpuRate and perPodMemoryRate will also tend to a large value, causing the final maximum number of replicas to be smaller than the latest number of Pods. Since there will be no active scaling down, the scaling flag value will be set to the fourth value (such as 6), i.e., isDown=6.

[0302] Step E14: If the final maximum number of replicas is detected to be greater than the latest number of Pods, check whether the first metric data is less than the preset initial threshold.

[0303] The preset initial threshold is the initial threshold (1000) mentioned in the above embodiment. This value can be set according to actual needs and is not limited here.

[0304] Step E15: If the first indicator data is detected to be less than the preset initial threshold, set the scaling flag value to the fifth value.

[0305] When the first indicator data is detected to be less than the preset initial threshold, it indicates that a scaling down process is required. Therefore, at this time, the scaling down flag value will be set to the fifth value (such as 2), i.e., isDown = 2.

[0306] Step E16: If the first indicator data is detected to be not less than the preset initial threshold, check whether the final maximum number of replicas is greater than the maximum number of HPA replicas.

[0307] If none of the above situations apply, the final maximum number of replicas and the maximum number of replicas in HPA need to be compared to set the scaling flag data.

[0308] Step E17: If the final maximum number of replicas is not greater than the maximum number of replicas in HPA, set the scaling flag value to the sixth value.

[0309] When the final maximum number of replicas is less than or equal to the maximum number of replicas in HPA, the instruction remains unchanged. Therefore, the scaling flag value needs to be set to the sixth value (such as 1), i.e., isDown = 1.

[0310] Step E18: If the final maximum number of replicas is detected to be greater than the maximum number of replicas of HPA, set the scaling flag value to the seventh value.

[0311] When the final maximum number of replicas exceeds the maximum number of replicas in HPA, it indicates that expansion is required. Therefore, the scaling flag value needs to be set to the seventh value (such as 0), i.e., isDown = 0.

[0312] Step 502: Store the final maximum number of replicas and the corresponding scaling flag value in the recommendation array.

[0313] The recommendation array also includes the final maximum replica count determined at multiple historical moments, as well as the scaling flag values ​​corresponding to each final maximum replica count. In practical applications, changes are only executed if the isDown flag type of all recommendation values ​​within the election window is completely consistent (all <= 1 or all >= 1), to avoid erroneous operations caused by instantaneous index fluctuations.

[0314] Based on the above description, such as Figure 6 As shown, as an optional implementation, the method described above, step 104 of updating the maximum number of HPA replicas in the HPA object using the final maximum number of replicas includes the following steps:

[0315] Step 601: Iterate through the recommended array to detect the scaling flag values ​​corresponding to each final maximum replica count;

[0316] Iterate through all the final maximum replica counts in the recommendation array and check their corresponding scaling flag values. This step is used to collect the scaling status of all replicas and provide a data basis for subsequent decisions.

[0317] Step 602: If it is detected that the scaling flag values ​​corresponding to each final maximum replica number in the recommended array are all greater than the preset value, or there are values ​​that are both greater than and equal to the preset value, select the smallest final maximum replica number from the recommended array as the final target maximum replica number, and update the HPA maximum replica number in the HPA object using the final target maximum replica number.

[0318] When all scaling flag values ​​are greater than the preset value (strictly greater than), or when some scaling flag values ​​are equal to the preset value and the rest are greater than the preset value (mixed state), the smallest final maximum replica number is selected from the recommended array, and the maximum replica number of HPA in the HPA object is updated. The purpose of this selection is to prioritize the smallest replica number to control costs when resource utilization is high.

[0319] Step 603: If it is detected that the scaling flag values ​​corresponding to each final maximum replica number in the recommended array are all less than the preset value, or there are values ​​that are both less than and equal to the preset value, select the largest final maximum replica number from the recommended array as the final target maximum replica number, and update the HPA maximum replica number in the HPA object using the final target maximum replica number.

[0320] When all scaling flag values ​​are less than the preset value (strictly less than), or when some scaling flag values ​​are equal to the preset value and the rest are less than the preset value (mixed state), the largest final maximum replica number is selected from the recommended array, and the maximum replica number of HPA in the HPA object is updated. The purpose of this selection is to select the maximum replica number to ensure service availability when resource utilization is low.

[0321] In practical applications, when the above preset value is 1, the process of selecting the final maximum number of replicas is as follows: In the election window, if all recommended values, i.e., the scaling flag values ​​corresponding to the final maximum number of replicas, areDown, satisfy one of the following two conditions, then the corresponding selection operation is performed; otherwise, the selection update is not performed (all are equal to 1, or there are values ​​greater than 1 and less than 1).

[0322] 1. When isDown <= 1, select the largest final maximum replica count among all values ​​for use in HPA maximum replica count updates;

[0323] 2. When isDown>=1, select the smallest final maximum replica count among all values ​​for use in HPA maximum replica count updates.

[0324] The above-mentioned strategy for selecting the maximum number of replicas is to select the maximum when expanding and the minimum when shrinking, so as to avoid the operation latency of the election window affecting the overall response speed. The strategy can also be flexibly adjusted according to the characteristics of the business itself, and there is no limitation on this.

[0325] The specific process of updating the maximum replica count of the HPA object using the final target maximum replica count in steps 602 and 603 above can be implemented through steps F1 to F10:

[0326] Step F1: Check if the final target maximum number of replicas is the same as the maximum number of replicas in HPA;

[0327] If the final target maximum number of replicas is detected to be the same as the HPA maximum number of replicas, proceed to step F2 immediately; if the final target maximum number of replicas is detected to be different from the HPA maximum number of replicas, proceed to step F3 immediately.

[0328] Step F2: Do not update the maximum number of HPA replicas in the HPA object;

[0329] If the maximum number of replicas of the selected final target is equal to the maximum number of replicas of the HPA, no update will be made, reducing the number of modifications to the HPA object and saving unnecessary interface calls and system overhead.

[0330] Step F3: Check whether the final target maximum number of replicas exceeds the replica number threshold;

[0331] The replica count threshold is a preset multiple of the latest Pod count; this preset multiple can be set according to actual needs and is not limited here. For example, during load testing, there may be a period where the calculated maximum number of replicas in memory or CPU is very high. This is generally because the Pods have just been expanded and have not yet been evenly distributed to handle the traffic. Therefore, the final target maximum replica count for each adjustment is limited to no more than twice the latest Pod count.

[0332] If the maximum number of replicas of the final target is not detected to exceed the replica number threshold, step F4 is executed immediately; if the maximum number of replicas of the final target is detected to exceed the replica number threshold, step F10 is executed immediately.

[0333] Step F4: Check if the final target maximum number of replicas is less than the latest number of Pods;

[0334] If the final target maximum replica count is not less than the latest Pod count, immediately execute step F5; if the final target maximum replica count is less than the latest Pod count, it means that after the Pods are scaled up, the QPS tends to stabilize, and the reached value exceeds the target maximum threshold targetQps. The calculated perPodCpuRate and perPodMemoryRate will also tend to a large value, causing the final target maximum replica count to be smaller than the latest Pod count. In this case, immediately execute step F6.

[0335] Step F5: Replace and update the maximum replica count of HPA in the HPA object based on the final target maximum replica count;

[0336] The update can be completed by writing the final target maximum number of replicas selected by the scaling flag value into the maxReplicas field of the HPA object.

[0337] Step F6: Obtain the tag value for enabling the shrinking tag;

[0338] The "enableActiveScaleDown" label is a flag that controls the automatic scale-down function, and its function is as follows:

[0339] When the tag value is set to true, it allows the number of replicas to be automatically reduced (scaling down) based on resource utilization or preset rules.

[0340] If the tag value is set to false, shrinking operations are prohibited, and only scaling up or maintaining the current number of replicas is allowed.

[0341] The HPA PLUS controller obtains the current tag value of the flag by reading the custom enableActiveScaleDown tag in the HPA object.

[0342] Step F7: Check if the label value is the preset value;

[0343] The default value is true as mentioned above. If the label value is detected as the default value, i.e., enableActiveScaleDown = true, then active scaling down can be triggered, making the actual QPS approach the target maximum threshold targetQps, thereby making lossPods (lossPods = latest number of Pods - final target maximum number of replicas) almost equal to 0, then step F8 needs to be executed; if the label value is detected as not being the default value, i.e., enableActiveScaleDown = false, active scaling down is disabled, then step F9 needs to be executed.

[0344] Step F8: Replace and update the maximum number of HPA replicas in the HPA object based on the final target maximum number of replicas;

[0345] Step F9: Replace and update the maximum number of HPA replicas in the HPA object based on the latest Pod count;

[0346] Step F10: Replace and update the maximum number of HPA replicas in the HPA object based on the replica number threshold.

[0347] When the maximum number of replicas for the final target exceeds the preset multiple limit of the latest Pod number, the replica number threshold is updated to effectively avoid resource exhaustion caused by sudden traffic.

[0348] The above technical solution ensures the safety and reliability of the update operation through multi-level verification (threshold, Pod count comparison, tag status), effectively reducing the risk of update errors.

[0349] Following on from the previous point, the large fluctuations in the final target maximum number of replicas are mainly due to the calculated single Pod CPU utilization and the maximum number of CPU replicas. There are three main reasons for these large fluctuations:

[0350] 1. QPS data fluctuated;

[0351] 2. The current number of Pods is fluctuating;

[0352] 3. The average CPU utilization (including soft interrupts) of the current instance fluctuates;

[0353] The QPS of an instance cannot be accurately estimated using a single model at present. Different types of requests have different QPS capacity per unit of resources.

[0354] Whether there is a long or short connection between the client and the LB (Load Balancer);

[0355] Whether there are long or short connections between the load balancer (LB) and the backend real server (RS);

[0356] The listener type is either HTTP (Hypertext Transfer Protocol) or HTTPS (Hypertext Transfer Protocol Secure).

[0357] The version of the TLS (Transport Layer Security) protocol;

[0358] The specific cryptographic suite used by TLS;

[0359] Whether the TLS session uses a session / tickets session reuse mechanism, etc.

[0360] For all the reasons mentioned above, we can only assume that the distribution of the current request types is stable and then linearly calculate the maximum number of Pods required.

[0361] To address the above issues, it is necessary to smooth the single-Pod CPU utilization and the maximum number of CPU replicas, filtering out outliers with excessively large deviations. The following approach is adopted, using single-Pod CPU utilization and the maximum number of CPU replicas as smoothing targets, and executing steps H1 to H5 for each smoothing target:

[0362] Step H1: Obtain multiple historical data values ​​and the current data value;

[0363] Retain at least five recently calculated historical data values ​​r5, r4, r3, r2, and r1 (sorted from oldest to newest) and the current data value rnow. If the target is a smooth object of single Pod CPU utilization, then the above data values ​​are the historically calculated single Pod CPU utilization and the currently calculated single Pod CPU utilization.

[0364] If the target is a smooth object such as the maximum number of CPU replicas, then the above data values ​​are the historically calculated maximum number of CPU replicas and the currently calculated maximum number of CPU replicas.

[0365] Step H2: Determine the mean and standard deviation based on multiple historical data values ​​and the current data value;

[0366] Continuing from the previous example, the mean can be calculated using the following formula:

[0367]

[0368] The standard deviation can be calculated using the following formula:

[0369]

[0370] Step H3: Determine whether the noise filtering mechanism based on standard deviation is satisfied based on the current data value, mean, and standard deviation.

[0371] A standard deviation-based noise filtering mechanism is used to detect whether a smoothed object is a noise point. If the standard deviation-based noise filtering mechanism is satisfied, then... If the smoothed object is not a noise point, proceed to step H4; if the standard deviation-based noise filtering mechanism is not satisfied, i.e. If the smoothed object is a noise point, proceed to step H5.

[0372] Step H4: Retain the current data value;

[0373] Step H5: Retain the historical data values ​​from the previous moment.

[0374] That is, select the historical data value r1 from the historical data values ​​r5, r4, r3, r2, and r1 for the previous moment and use it for subsequent calculations.

[0375] In practical applications, expansion strategies can also be set to adjust the expansion and contraction speed. Specifically, the target expansion and contraction strategy attribute value corresponding to the first indicator data is retrieved from the expansion and contraction strategy attribute value lookup table; the contraction strategy attribute value in the HPA object is updated using the target expansion and contraction strategy attribute value.

[0376] Among them, the scaling strategy attribute value lookup table pre-stores the correspondence between the index data range of the first weight index and the scaling strategy attribute value, and the scaling strategy attribute value is used to characterize the scaling speed.

[0377] The scaling strategy attribute value lookup table mentioned above may be a database table, an Excel spreadsheet, a configuration file, or other data structure; no specific limitation is made here. The metric data range is based on the targetQps value, as shown in Table 1 for ease of understanding.

[0378] Table 1

[0379] 0-0.3*targetQps 30 0.3*targetQps - 0.5*targetQps 60 0.5*targetQps - 0.7*targetQps 110 0.7*targetQps - 0.85*targetQps 160 0.85*targetQps - 0.95*targetQps 300

[0380] It should be noted that the correspondence between the data range of the first weight indicator and the attribute value of the scaling strategy listed above is only an example. The specific correspondence between the data range of the first weight indicator and the attribute value of the scaling strategy can be set according to actual needs, and is not limited here.

[0381] Figure 7 This diagram illustrates the update process and calculation logic for the maximum replica count and scaling strategy attribute values ​​in this embodiment. The implementation of each module in the diagram is consistent with the aforementioned embodiments, and specific details will not be repeated.

[0382] It should be noted that in practical applications, it can be based on Figure 7 The specific business metrics selected as the first and second weighted metrics are determined based on the business traffic. Figure 7 The maximum quota amount is the maximum metric threshold. When the first weight metric is QPS, only targetQps needs to be set, and when the first weight metric is CPS, only targetCps needs to be set. The specific maximum metric threshold needs to be determined according to the instance specifications. a.cpuRatio and a.memoryRatio need to be set in advance.

[0383] See Figure 8 This is a block diagram illustrating an embodiment of a device for dynamic resource scaling and quota limiting of a cloud load balancer provided in this application. Figure 8 As shown, the device includes:

[0384] The acquisition module 800 is used to periodically poll and obtain business data and resource utilization for the current maximum number of replicas of the computing Pod. The business data includes the first indicator data of the first weight indicator and the second indicator data of the second weight indicator. The first and second weight indicators are business indicators that do not need to be adapted to the HPA controller.

[0385] The first determination module 801 is used to determine the basic maximum number of replicas based on the first indicator data and resource utilization rate;

[0386] The second determination module 802 is used to dynamically adjust the basic maximum number of replicas based on the second indicator data through a hierarchical threshold triggering mechanism to determine the final maximum number of replicas.

[0387] Update module 803 is used to update the maximum number of HPA replicas in the HPA object with the final maximum number of replicas.

[0388] This solution directly incorporates business metrics into HPA replica count decisions, completely bypassing the architectural limitations of traditional solutions that rely on adapters. Its core technological breakthrough lies in calculating the final maximum replica count based on business metrics such as the first and second weighted metrics, and dynamically updating the maximum replica count of HPA objects. The entire process completely skips the intermediate adaptation step of converting business metrics into native HPA metrics, achieving seamless integration between business metrics and the elastic scaling system. This direct-connection design not only eliminates adapter development and maintenance costs but also effectively avoids architectural redundancy, significantly reducing deployment, configuration, and version compatibility maintenance costs.

[0389] Furthermore, the aforementioned technical solution optimizes the calculation of the maximum number of replicas through a hierarchical decision-making mechanism. First, a basic maximum number of replicas is generated based on resource utilization and a primary weighting indicator. Then, this value is dynamically adjusted using a secondary weighting indicator. This multi-indicator collaborative scheduling design effectively overcomes the problem of frequent Pod count fluctuations caused by independent triggering of various indicators in traditional methods, significantly improving scaling stability and further optimizing business performance.

[0390] Figure 9 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 9 The illustrated electronic device 1200 includes at least one processor 1201, a memory 1202, at least one network interface 1204, and other user interfaces 1203. The various components in the electronic device 1200 are coupled together via a bus system 1205. It is understood that the bus system 1205 is used to implement communication between these components. In addition to a data bus, the bus system 1205 also includes a power bus, a control bus, and a status signal bus. However, for clarity, ... Figure 9 The general labeled all buses as Bus System 1205.

[0391] The user interface 1203 may include a display, keyboard, or clicking device (e.g., mouse, trackball, touchpad, or touchscreen).

[0392] It is understood that the memory 1202 in the embodiments of this application can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), Synchronous DRAM (SDRAM), Double Data Rate Synchronous DRAM (DDRSDRAM), Enhanced Synchronous DRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), and Direct Rambus RAM (DRRAM). The memory 1202 described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0393] In some implementations, memory 1202 stores elements, executable units or data structures, or subsets thereof, or extended sets thereof: operating system 12021 and application program 12022.

[0394] The operating system 12021 includes various system programs, such as the framework layer, core library layer, and driver layer, used to implement various basic business functions and handle hardware-based tasks. The application program 12022 includes various applications, such as a media player and a browser, used to implement various application functions. The program implementing the method of this application embodiment can be included in the application program 12022.

[0395] In this embodiment of the application, the processor 1201 executes the method steps provided by each method embodiment by calling the program or instructions stored in the memory 1202, specifically the program or instructions stored in the application program 12022.

[0396] The methods disclosed in the embodiments of this application can be applied to or implemented by the processor 1201. The processor 1201 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the integrated logic circuit of the hardware or by instructions in the form of software in the processor 1201. The processor 1201 may be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software units in the decoding processor. The software unit can reside in a mature storage medium in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. This storage medium is located in memory 1202. Processor 1201 reads the information in memory 1202 and, in conjunction with its hardware, completes the steps of the above method.

[0397] It is understood that the embodiments described herein can be implemented in hardware, software, firmware, middleware, microcode, or a combination thereof. For hardware implementation, the processing unit can be implemented in one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers, microprocessors, other electronic units for performing the functions described herein, or combinations thereof.

[0398] For software implementation, the techniques described herein can be implemented by units that perform the functions described herein. The software code can be stored in memory and executed by a processor. The memory can be implemented in the processor or external to the processor.

[0399] The electronic device provided in this embodiment may be as follows: Figure 9 The electronic device shown can perform the following: Figure 1-7 All steps of the method for dynamic resource scaling and quota limiting in Zhongyun load balancer, thereby achieving... Figure 1-7 For details on the technical effects of the cloud load balancer's method for dynamic resource scaling and quota limiting, please refer to [link / reference needed]. Figure 1-7 The relevant descriptions are presented concisely and will not be elaborated upon here.

[0400] This application also provides a storage medium (computer-readable storage medium). This storage medium stores one or more programs. The storage medium may include volatile memory, such as random access memory; it may also include non-volatile memory, such as read-only memory, flash memory, hard disk, or solid-state drive; and it may also include combinations of the above types of memory.

[0401] When one or more programs in the storage medium can be executed by one or more processors, the above-mentioned method of dynamic scaling and quota limiting of cloud load balancers can be implemented.

[0402] The processor is used to execute programs stored in the memory to implement the steps of a method for dynamic scaling and quota limiting of resources in a cloud load balancer.

[0403] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0404] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented in hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0405] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of this application. It should be understood that the above description is only a specific embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A method for dynamic resource scaling and quota limiting in a cloud load balancer, characterized in that, The method includes: Periodically poll to obtain business data and resource utilization for the current maximum number of replicas of the compute Pod. The business data includes first indicator data of a first weight indicator and second indicator data of a second weight indicator. The first weight indicator and the second weight indicator are business indicators that do not need to be adapted to the HPA controller. The basic maximum number of replicas is determined based on the first indicator data and the resource utilization rate; Based on the second indicator data, the basic maximum number of replicas is dynamically adjusted through a tiered threshold triggering mechanism to determine the final maximum number of replicas. Update the maximum number of HPA replicas in the HPA object using the final maximum number of replicas; The resource utilization rate includes the current number of Pods. The step of dynamically adjusting the basic maximum replica count based on the second metric data using a tiered threshold triggering mechanism to determine the final maximum replica count includes: The value per core is determined based on the second indicator data and the current number of Pods; The system detects whether the value per core exceeds a preset index data threshold range; wherein the preset index data threshold range is a data range consisting of a pre-defined pressure value and a limit value corresponding to each core. If the value per core is detected to be within the preset index data threshold range, the real-time change rate of the first weight index is obtained; Detect whether the real-time rate of change exceeds a preset rate of change threshold; If the real-time change rate is detected to exceed the preset change rate threshold, the basic maximum replica count is adjusted according to the preset adjustment Pod count, and the adjusted basic maximum replica count is determined as the final maximum replica count. If the value per core unit is detected to exceed the preset indicator data threshold range, the basic maximum replica count is modified based on the second indicator data, and the modified basic maximum replica count is determined as the final maximum replica count. If the value per core is detected to be lower than the preset index data threshold range, or if the real-time change rate is detected to be less than the preset change rate threshold, the basic maximum replica count is determined as the final maximum replica count.

2. The method according to claim 1, characterized in that, The periodic polling to obtain business data and resource utilization for the current maximum number of replicas of the compute Pod includes: Obtain the first resource utilization rate sequence collected using the current time window; Extract the start and end timestamps of the time window; Determine whether the time span of the start and end timestamps is less than a preset time span; If the time span of the start and end timestamps is determined to be less than the preset time span, the current first data sequence is obtained by querying the cluster monitoring system based on the preset time span; if the time span of the start and end timestamps is determined to be equal to or greater than the preset time span, the current first data sequence is obtained by querying the cluster monitoring system based on the time span of the start and end timestamps. Time-align the first resource utilization rate sequence and the first data sequence to determine the second resource utilization rate sequence and the second data sequence; The resource utilization rate is obtained based on the second resource utilization rate sequence, and the first indicator data of the first weight indicator and the second indicator data of the second weight indicator are obtained based on the second data sequence.

3. The method according to claim 2, characterized in that, The resource utilization rate includes the current number of Pods, the current memory utilization rate, and the current CPU utilization rate; The step of obtaining the resource utilization rate based on the second resource utilization rate sequence includes: Obtain the latest Pod count from the second resource utilization sequence; The average number of Pods and the standard deviation of Pods are determined based on the second resource utilization sequence; Based on the latest number of Pods, the average number of Pods, and the standard deviation of Pods, determine whether the noise filtering mechanism based on 2σ standard deviation is satisfied; If the noise filtering mechanism based on 2σ standard deviation is satisfied, the latest number of Pods is determined as the current number of Pods; If the noise filtering mechanism based on 2σ standard deviation is not satisfied, the latest Pod count at the previous running time is determined as the current Pod count; Obtain the latest memory utilization rate from the second resource utilization rate sequence, and determine the latest memory utilization rate as the current memory utilization rate; Obtain the latest CPU utilization from the second resource utilization sequence, and determine the latest CPU utilization as the current CPU utilization.

4. The method according to claim 2, characterized in that, The first indicator data of the first weight indicator and the second indicator data of the second weight indicator are obtained based on the second data sequence; Obtain the latest first indicator data from the second data sequence, and determine the latest first indicator data as the first indicator data of the first weight indicator; Obtain the latest second indicator data from the second data sequence, and determine the latest second indicator data as the second indicator data of the second weight indicator.

5. The method according to claim 3, characterized in that, The determination of the basic maximum number of replicas based on the first indicator data and the resource utilization rate includes: Obtain the maximum threshold, memory utilization threshold, and CPU utilization threshold of the first weighted index; The maximum number of CPU replicas is determined based on the current number of Pods, the maximum metric threshold, the first metric data, the CPU utilization threshold, and the current CPU utilization. The maximum number of memory replicas is determined based on the current number of Pods, the maximum metric threshold, the first metric data, the memory utilization threshold, and the current memory utilization. Select the maximum value between the maximum number of CPU replicas and the maximum number of memory replicas, and round the maximum value up to obtain the basic maximum number of replicas.

6. The method according to claim 1, characterized in that, The modification of the basic maximum replica count based on the second indicator data includes: The required number of cores is determined based on the per-core value and the limit value; The number of replicas to be added is determined based on the required number of cores; The base maximum number of replicas is modified based on the number of replicas added.

7. The method according to claim 5, characterized in that, Before updating the maximum number of HPA replicas in the HPA object using the final maximum number of replicas, the method further includes: Based on the first metric data, the current number of Pods, the current CPU utilization, the current memory utilization, the maximum metric threshold, the CPU utilization threshold, the memory utilization threshold, and the maximum number of HPA replicas, determine the scaling flag value corresponding to the final maximum number of replicas; The final maximum number of replicas and the corresponding scaling flag value are stored in the recommendation array; wherein, the recommendation array also includes the final maximum number of replicas determined at multiple historical moments, and the scaling flag value corresponding to each final maximum number of replicas.

8. The method according to claim 7, characterized in that, The step of determining the scaling flag value corresponding to the final maximum replica count based on the first metric data, the current number of Pods, the current CPU utilization, the current memory utilization, the maximum metric threshold, the CPU utilization threshold, the memory utilization threshold, and the maximum HPA replica count includes: The single Pod CPU utilization is determined based on the first indicator data, the current number of Pods, and the current CPU utilization; and the single Pod memory utilization is determined based on the first indicator data, the current number of Pods, and the current memory utilization. The incremental demand value for memory Pods is determined based on the single Pod memory utilization rate, the first indicator data, and the maximum indicator threshold; and the incremental demand value for CPU Pods is determined based on the single Pod CPU utilization rate, the first indicator data, and the maximum indicator threshold. The remaining available Pod capacity value for CPU is determined based on the CPU utilization threshold, the current CPU utilization, the current number of Pods, and the maximum number of HPA replicas; and the remaining available Pod capacity value for memory is determined based on the memory utilization threshold, the current memory utilization, the current number of Pods, and the maximum number of HPA replicas. The scaling flag value corresponding to the final maximum number of replicas is determined based on the final maximum number of replicas, the maximum number of HPA replicas, the incremental demand value of memory Pods, the incremental demand value of CPU Pods, the remaining available Pod capacity of CPU, the remaining available Pod capacity of memory, the first indicator data, the current CPU utilization, the current number of Pods, and the latest number of Pods.

9. The method according to claim 8, characterized in that, The method further includes: The single-pod CPU utilization and the maximum number of CPU replicas are respectively used as smoothing objects, and the following is performed on each of the smoothing objects: Retrieve multiple historical data values ​​and the current data value; The mean and standard deviation are determined based on the multiple historical data values ​​and the current data value; Based on the current data value, the mean, and the standard deviation, determine whether the noise filtering mechanism based on the standard deviation is satisfied. If the current data value is determined to meet the standard deviation-based noise filtering mechanism, it is retained. If the noise filtering mechanism based on standard deviation is not satisfied, the historical data value from the previous moment is retained.

10. The method according to claim 8, characterized in that, The step of determining the scaling flag value corresponding to the final maximum replica count based on the final maximum replica count, the maximum HPA replica count, the incremental memory Pod requirement value, the incremental CPU Pod requirement value, the remaining available CPU Pod capacity value, the remaining available memory Pod capacity value, the first metric data, the current CPU utilization, the current number of Pods, and the latest number of Pods includes: Check whether the final maximum number of replicas is the same as the latest number of Pods; If the final maximum number of replicas is found to be the same as the latest number of Pods, it is determined whether the final maximum number of replicas is selected from the maximum number of CPU replicas. If the final maximum number of replicas is detected to be selected from the maximum number of CPU replicas, it is determined whether the first condition or the second condition is met based on the CPUPod incremental demand value and the CPU remaining available Pod capacity value. If either the first condition or the second condition is met, the value of the telescoping flag is set to the first value; If the final maximum number of replicas is detected to be selected from the maximum number of replicas in memory, it is determined whether the third or fourth condition is met based on the incremental memory Pod requirement value and the remaining available memory Pod capacity value. If the third or fourth condition is met, the value of the telescoping flag is set to the first value; If the final maximum number of replicas is found to be different from the latest number of Pods, the average data of the first indicator is determined based on the second data sequence. Determine whether the fifth condition is met based on the first indicator data and the average data of the first indicator, or determine whether the sixth condition is met based on the current number of Pods and the current number of Pods at the previous running time. If the fifth or sixth condition is met, the value of the telescoping flag is set to the second value; If the fifth and sixth conditions are not met, check whether the current CPU utilization is less than a preset pressure threshold. If the current CPU utilization is detected to be less than the preset pressure threshold, the scaling flag value is set to a third value; If the current CPU utilization is not less than the preset pressure threshold, it is detected whether the final maximum number of replicas is less than the latest number of Pods; If the final maximum number of replicas is detected to be less than the latest number of Pods, the scaling flag value is set to the fourth value; If the final maximum number of replicas is detected to be greater than the latest number of Pods, it is detected whether the first metric data is less than a preset initial threshold. If the first indicator data is detected to be less than the preset initial threshold, the scaling flag value is set to the fifth value; If the first indicator data is detected to be not less than the preset initial threshold, it is detected whether the final maximum number of replicas is greater than the maximum number of HPA replicas; If it is detected that the final maximum number of replicas is not greater than the maximum number of replicas of HPA, the scaling flag value is set to the sixth value; If the final maximum number of replicas is detected to be greater than the maximum number of replicas of the HPA, the scaling flag value is set to the seventh value.

11. The method according to claim 7, characterized in that, The step of updating the maximum number of HPA replicas in the HPA object using the final maximum number of replicas includes: Iterate through and detect the scaling flag values ​​corresponding to each of the final maximum replica counts in the recommendation array; If it is detected that the scaling flag values ​​corresponding to each of the final maximum replica numbers in the recommendation array are all greater than the preset value, or if there are values ​​greater than and equal to the preset value at the same time, the smallest final maximum replica number is selected from the recommendation array as the final target maximum replica number, and the HPA maximum replica number in the HPA object is updated using the final target maximum replica number. If it is detected that the scaling flag values ​​corresponding to each of the final maximum replica counts in the recommendation array are all less than the preset value, or if there are values ​​that are both less than and equal to the preset value, the largest final maximum replica count is selected from the recommendation array as the final target maximum replica count, and the maximum replica count of HPA in the HPA object is updated using the final target maximum replica count.

12. The method according to claim 11, characterized in that, The step of updating the maximum number of HPA replicas in the HPA object using the final target maximum number of replicas includes: Check whether the final target maximum number of replicas is the same as the maximum number of replicas of the HPA; If the maximum number of replicas of the final target is detected to be the same as the maximum number of replicas of the HPA, the maximum number of replicas of the HPA in the HPA object will not be updated; If the maximum number of replicas in the final target is different from the maximum number of replicas in the HPA, it is detected whether the maximum number of replicas in the final target exceeds the replica number threshold; wherein, the replica number threshold is a preset upper limit of the latest number of Pods; If the maximum number of replicas of the final target is not exceeded, it is detected whether the maximum number of replicas of the final target is less than the latest number of Pods; If the maximum number of replicas of the final target is detected to be not less than the latest number of Pods, the maximum number of replicas of HPA in the HPA object is replaced and updated based on the maximum number of replicas of the final target; If the maximum number of replicas of the final target is less than the latest number of Pods, obtain the tag value for enabling the scaling down label; Detect whether the label value is a preset value; If the detected tag value is the preset value, the maximum number of HPA replicas in the HPA object is replaced and updated based on the final target maximum number of replicas; If the detected tag value is not the preset value, the maximum number of HPA replicas in the HPA object is replaced and updated based on the latest Pod count; If the maximum number of replicas of the final target is detected to exceed the replica number threshold, the maximum number of replicas of the HPA in the HPA object is replaced and updated based on the replica number threshold.

13. The method according to claim 1, characterized in that, The method further includes: The target scaling strategy attribute value corresponding to the first indicator data is retrieved from the scaling strategy attribute value lookup table; wherein, the scaling strategy attribute value lookup table pre-stores the correspondence between the indicator data range of the first weight indicator and the scaling strategy attribute value, and the scaling strategy attribute value is used to characterize the scaling speed. Update the scaling-down strategy attribute value in the HPA object using the target scaling-down strategy attribute value.

14. A device for dynamic resource scaling and quota limiting of a cloud load balancer, characterized in that, The device includes: The acquisition module is used to periodically poll and acquire business data and resource utilization for the current maximum number of replicas of the computing Pod. The business data includes first indicator data of a first weight indicator and second indicator data of a second weight indicator. The first weight indicator and the second weight indicator are business indicators that do not need to be adapted to the HPA controller. The first determining module is used to determine the basic maximum number of replicas based on the first indicator data and the resource utilization rate; The second determining module is used to dynamically adjust the basic maximum number of replicas based on the second indicator data through a hierarchical threshold triggering mechanism to determine the final maximum number of replicas. The update module is used to update the maximum number of HPA replicas in the HPA object using the final maximum number of replicas; The resource utilization rate includes the current number of Pods, and the update module is specifically used for: The value per core is determined based on the second indicator data and the current number of Pods; The system detects whether the value per core exceeds a preset index data threshold range; wherein the preset index data threshold range is a data range consisting of a pre-defined pressure value and a limit value corresponding to each core. If the value per core is detected to be within the preset index data threshold range, the real-time change rate of the first weight index is obtained; Detect whether the real-time rate of change exceeds a preset rate of change threshold; If the real-time change rate is detected to exceed the preset change rate threshold, the basic maximum replica count is adjusted according to the preset adjustment Pod count, and the adjusted basic maximum replica count is determined as the final maximum replica count. If the value per core unit is detected to exceed the preset indicator data threshold range, the basic maximum replica count is modified based on the second indicator data, and the modified basic maximum replica count is determined as the final maximum replica count. If the value per core is detected to be lower than the preset index data threshold range, or if the real-time change rate is detected to be less than the preset change rate threshold, the basic maximum replica count is determined as the final maximum replica count.

15. An electronic device, characterized in that, include: A processor and a memory, the processor being configured to execute a program stored in the memory for dynamic resource scaling and quota limiting of a cloud load balancer, to implement the method for dynamic resource scaling and quota limiting of a cloud load balancer as described in any one of claims 1 to 13.

16. A storage medium, characterized in that, The storage medium stores one or more programs, which can be executed by one or more processors to implement the method of dynamic resource scaling and quota limiting of the cloud load balancer according to any one of claims 1 to 13.

Images