Method for scaling container orchestration engine cluster, related device and system

Abstract

This application discloses a scaling method, related equipment, and system for a container orchestration engine cluster, relating to the field of cloud computing container scheduling technology. The container orchestration engine cluster includes at least one node, and each node runs at least one container group (Pod), which is used to implement business functions. The method includes: acquiring resource consumption data, business access data, and other collected data of nodes and / or pods; preprocessing the collected data to obtain normalized data; inputting the normalized data into a hybrid model to obtain load prediction results, wherein the load prediction results are obtained by weighting long-term load prediction results and short-term load prediction results, the long-term load prediction results are used to predict the long-term trend of load changes within a first time period, and the short-term load prediction results are used to predict short-term load fluctuations within the first time period; determining a scaling strategy based on the load prediction results; and performing scaling up or scaling down operations on the business based on the scaling strategy, thereby improving resource utilization.

Description

Technical Field

[0001] This application relates to the field of cloud computing container scheduling technology, and in particular to methods for scaling up and down container orchestration engine clusters, related equipment and systems. Background Technology

[0002] With the widespread adoption of cloud-native technologies, container orchestration engines (Kubernetes, K8s) have become the de facto standard for container orchestration, widely used in the deployment and management of various applications. K8s automatically adjusts application resource quotas through elastic scaling mechanisms such as Horizontal Pod Autoscaler (HPA), Vertical Pod Autoscaler (VPA), and Cluster Autoscaler (CA) to match changes in application workload, thereby achieving high availability and efficient resource utilization for application services. However, this elastic scaling mechanism primarily relies on real-time reactive resource adjustments based on a single resource metric (such as Central Processing Unit (CPU) utilization, memory utilization, etc.) or other metrics (such as Queries Per Second (QPS), etc.). In other words, elastic scaling is only triggered when resource metrics or other metrics exceed or fall below preset target thresholds, resulting in a lag in application resource adjustments and consequently, service latency. Summary of the Invention

[0003] This application discloses a scaling method, related devices, and system for a container orchestration engine cluster. The container orchestration engine can adjust application resources in a timely manner, reducing the response latency of scaling up and down the container orchestration engine, thereby reducing the probability of application service latency. In addition, it improves the accuracy of scaling up and down decisions of the container orchestration engine, further improving resource utilization and reducing the possibility of application service degradation.

[0004] Firstly, this application provides a scaling method for a container orchestration engine cluster. The container orchestration engine cluster includes at least one node, and each node runs at least one container group (pod). A pod is used to implement one or more services. The method includes: acquiring collected data, which includes resource consumption data of nodes and / or pods, and service access data, where the service access data is the accessed data of the services implemented by the pod; preprocessing the collected data to obtain preprocessed normalized data; inputting the normalized data into a hybrid model to obtain a target load prediction result; wherein the hybrid model includes: a long-term load prediction model, a short-term load prediction model, and a weighting module. The long-term load prediction model is used to output a long-term load prediction result based on the normalized data to predict the long-term trend of load change within a first time period. The short-term load prediction model is used to output a short-term load prediction result based on the normalized data to predict short-term load mutations within the first time period. The weighting module is used to weight the long-term load prediction result and the short-term load prediction result to obtain a target load prediction result; determining a first scaling strategy based on the target load prediction result; and performing scaling up or scaling down operations on the services based on the first scaling strategy. The first time period, also known as time period B, is any time period from the current time to the future. The first scaling strategy can be a scaling strategy determined based on the target load prediction results. By using the method provided in the first aspect, Kubernetes can predict the long-term load change trend based on the long-term load prediction model in the hybrid model, predict short-term load mutations based on the short-term load prediction model in the hybrid model, and then combine the load prediction results of the long-term and short-term load prediction models to perform scaling up or scaling down operations on the service, so as to adjust the application resources in a timely manner, reduce the response latency of Kubernetes scaling up and down, and thus reduce the probability of application service latency. In addition, Kubernetes can perform load prediction based on various collected data such as node and / or Pod resource consumption data and business access data, so as to improve the accuracy of load prediction results, thereby improving the accuracy of Kubernetes scaling up and down decisions, further improving resource utilization, and reducing the possibility of application service degradation.

[0005] In one possible implementation, a first scaling-up strategy is determined based on the target load prediction result, specifically including: inputting the target load prediction result into a scheduling model, and the scheduling model outputting the first scaling-up strategy with a first condition as a constraint; wherein, the first condition includes one or more of the following: the type of the new node is a specific type, the cold start latency of the new node does not exceed a first threshold, the new pod is deployed on the first node, the cold start latency of the new pod does not exceed a second threshold; the first node and the new pod have affinity, and / or, the pod on the first node and the new pod have affinity.

[0006] The first scaling strategy is a scaling strategy that balances stability and cost-effectiveness, determined based on the scheduling model. The first condition can be a constraint condition that limits the output of the first scaling strategy by the scheduling model. The first node can also be called node A, which is a node that has affinity with the new Pod and / or whose Pods have affinity with the new Pod. The first threshold can also be called threshold A (e.g., 90 seconds). The second threshold can also be called threshold B (e.g., 80 seconds).

[0007] In this way, the accuracy of Kubernetes' scaling decisions is improved, further enhancing resource utilization and reducing the likelihood of application service degradation.

[0008] In one possible implementation, the target load prediction result is three-dimensional data used to indicate the load prediction result for each time period within a first time period.

[0009] The load prediction results for each time period within the first time period can be measured using resource consumption data (e.g., CPU utilization) and / or service access data (e.g., QPS). It is understood that the content included in this load prediction result can be determined based on the parameters of the hybrid model or expert experience; this embodiment of the application does not impose any limitations on this.

[0010] In one possible implementation, weighting the long-term load forecast results and short-term load forecast results to obtain the target load forecast result specifically includes: determining the value of 'a', where the value of 'a' indicates the weight of the long-term load forecast results in the target load forecast result; and obtaining the target load forecast result z based on the following formula: Where x is the long-term load forecast result of the first service at the first time, y is the short-term load forecast result of the first service at the first time, and the first time is the time within the first time period.

[0011] Here, the first business can also be called business A, which can be any business; the first time can also be called time A, which is any point in time within the first time period.

[0012] In this way, by weighting the long-term load forecast results and the short-term load forecast results, the load characteristics of the business are accurately predicted, further improving the scaling decision accuracy of the container orchestration engine.

[0013] In one possible implementation, the value of 'a' is determined by the relative entropy of the long-term load forecasting model and the short-term load forecasting model.

[0014] The value of 'a' is determined by the load characteristics of the current business, which can be measured by the relative entropy (i.e., KL divergence) between the long-term load forecasting model and the short-term load forecasting model. In other words, the value of 'a' is determined by the relative entropy between the long-term load forecasting model and the short-term load forecasting model.

[0015] It's important to note that the relative entropy (KL divergence) between the two models measures the difference between the probability distributions output by the two models based on the same set of inputs. A larger KL divergence value indicates a greater difference between short-term and long-term load forecasts, meaning the current business load characteristics are more inclined towards short-term bursts of load. In this case, the short-term load forecast model needs higher weighting to ensure accuracy. Conversely, a smaller KL divergence value indicates a smaller difference between short-term and long-term load forecasts, meaning the current business load characteristics are more inclined towards long-term load trends. In this case, the long-term load forecast model needs higher weighting.

[0016] In one possible implementation, when the relative entropy is greater than the third threshold, the value of a is determined by the following formula: Where b is a smoothing coefficient, used to adjust the degree of influence of relative entropy on the value of a. ,and c is the relative entropy. .

[0017] Thus, when the KL divergence of the two models exceeds the third threshold (e.g., 0.3), the weight 'a' of the long-term load prediction model can be adjusted using the above formula to improve the accuracy of predicting the load characteristics of the business and further enhance the scaling decision accuracy of the container orchestration engine.

[0018] In one possible implementation, the method further includes: collecting the actual load results after performing scaling up or scaling down operations on the service; and adjusting the weight parameters of the hybrid model based on the actual load results.

[0019] In this way, the hybrid model can perform adaptive optimization to improve the accuracy of load prediction results and further improve the accuracy of Kubernetes scaling decisions.

[0020] In one possible implementation, after adjusting the weight parameters of the hybrid model based on the actual load results, the method further includes: in the next round of load forecasting, using the adjusted hybrid model to obtain the target load forecast result for the next round; determining the first scaling strategy for the next round based on the target load forecast result for the next round; if the deviation between the actual load result after implementing the first scaling strategy for the next round and the target load forecast result for the next round exceeds a fourth threshold, then rolling back the version of the hybrid model to the version before adjusting the weight parameters.

[0021] The fourth threshold can also be called threshold C (e.g., 50%).

[0022] In this way, Kubernetes can ensure the accuracy of load forecasting results and the stability of systems containing servers that implement scaling methods for container orchestration engine clusters during hybrid model iterations.

[0023] In one possible implementation, the normalized data is three-dimensional data used to indicate the actual load results of the service at each time point within a second time period, which is the time period for acquiring the collected data.

[0024] The second time period, also known as time period A, is the time period for acquiring and collecting data. For example, the normalized data is a three-dimensional tensor structure, with the three dimensions being the time dimension, the node dimension, and the load dimension, respectively. The load can be measured by resource consumption data (such as CPU utilization) and / or business access data (such as QPS).

[0025] In one possible implementation, resource consumption data includes one or more of the following: computing resource consumption data, storage resource consumption data, and network resource consumption data. It should be noted that resource consumption data refers to measurements of the core resource usage of various components (such as nodes and Pods) within a Kubernetes cluster, including but not limited to one or more of the following: computing resource consumption data, storage resource consumption data, and network resource consumption data. For example, computing resources include: CPU resources and Graphics Processing Unit (GPU) resources; computing resource consumption data includes: CPU utilization and GPU utilization. Storage resources include: memory resources and external storage resources; storage resource consumption data includes: memory utilization and disk utilization. Network resources include: bandwidth and network connections; network resource consumption data includes: bandwidth utilization and data transmission / reception volume per unit time.

[0026] In one possible implementation, the business access data includes one or more of the following: the query per second (QPS) of the business, the success rate of business implementation, the success rate of business API call, and the latency of business API call.

[0027] It should be noted that business access data refers to the access data of the business implemented by the Pod, that is, the data related to business functions, performance and user experience generated when external users or internal services access the Pod running on Kubernetes. It can reflect the frequency of business access, including but not limited to one or more of the following: business QPS, business implementation success rate, business interface call success rate, and business interface call latency.

[0028] Secondly, this application provides a scaling device for a container orchestration engine cluster. The container orchestration engine cluster includes at least one node, and each node runs at least one container group pod. A pod is used to implement one or more services. The device includes: a data acquisition module for acquiring collected data, which includes: resource consumption data of nodes and / or pods, and service access data, where the service access data is the accessed data of the service implemented by the pod; a data processing module for preprocessing the collected data to obtain preprocessed normalized data; and a hybrid model for receiving the normalized data and outputting the target load prediction result. The hybrid model includes: a long... The system includes a long-term load forecasting model, a short-term load forecasting model, and a weighting module. The long-term load forecasting model outputs long-term load forecasting results based on normalized data to predict the long-term load trend within a first time period. The short-term load forecasting model outputs short-term load forecasting results based on normalized data to predict short-term load fluctuations within the first time period. The weighting module weights the long-term and short-term load forecasting results to obtain the target load forecasting result. A strategy generation module determines a first scaling-up / scaling strategy based on the target load forecasting result. A strategy execution module performs scaling-up or scaling-down operations on the service based on the first scaling-up / scaling strategy.

[0029] Thirdly, this application provides an electronic device, which includes one or more processors and one or more memories; wherein the one or more memories are coupled to one or more processors, and the one or more memories are used to store a computer program, and when the one or more processors execute the computer program, the electronic device performs the method described in any of the first aspects above.

[0030] Fourthly, this application provides a computer-readable storage medium storing a computer program that is executed by a processor to implement the method described in any of the first aspects above.

[0031] Fifthly, this application provides a computer program product that, when executed by a processor, implements the method described in any one of the first aspects above.

[0032] In a sixth aspect, this application provides a chip including a processor and a memory, wherein the memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory, causing the chip to perform the method described in any of the first aspects above.

[0033] The solutions provided in the second to sixth aspects above are used to implement or cooperate with the methods provided in the first aspect above, and therefore can achieve the same or corresponding beneficial effects as the methods in the first aspect, which will not be elaborated here. Attached Figure Description

[0034] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0035] Figure 1 A schematic diagram of a system 10 provided in an embodiment of this application; Figure 2 A schematic diagram illustrating a method for scaling up or down a container orchestration engine cluster, as provided in an embodiment of this application. Figure 3 A schematic diagram of a system architecture for a container orchestration engine provided in this application embodiment; Figure 4 This is a schematic diagram of the structure of a server 100 provided in an embodiment of this application. Detailed Implementation

[0036] The terminology used in the embodiments of this application is for the purpose of describing particular embodiments only and is not intended to be a limitation of this application.

[0037] In cloud computing scenarios, on the one hand, container orchestration engines (Kubernetes, K8s) can deploy and manage applications by scheduling Pods (container groups) onto nodes.

[0038] A Pod is the smallest scheduling unit in Kubernetes (K8s), consisting of one or more logically related containers. A container is a lightweight, portable, and self-contained software packaging and runtime technology that can package an application and all its necessary dependencies (such as libraries, environment variables, configuration files, etc.) together to form a standardized unit. Generally, a container encapsulates an independent process or service (such as an authentication service); in other words, a container is used to implement a business function. The logical relationship between one or more containers in a Pod means that the containers in a Pod work closely together and share resources to complete a common high-level task or function. For example, container A is used to implement the business function of retrieving product information, container B is used to implement the business function of processing product information, and container C is used to implement the business function of outputting product information. Containers A, B, and C together form a Pod, which is used to implement the high-level function of generating product order information.

[0039] Containers within a Pod are scheduled and run together on the same node, sharing the same lifecycle. A node is the unit of work in a Kubernetes cluster, providing physical or virtual resources (such as compute, storage, and network resources) for the application workload (i.e., Pod). Its primary responsibility is to run applications by running Pods. One or more nodes constitute a Kubernetes cluster. It is understood that the "node" mentioned in this embodiment can also be referred to as a "worker node."

[0040] On the other hand, Kubernetes can adjust application resource quotas through elastic scaling mechanisms.

[0041] Kubernetes offers three elastic scaling mechanisms: Horizontal Pod Autoscaler (HPA), Vertical Pod Autoscaler (VPA), and Cluster Autoscaler (CA).

[0042] HPA adjusts the number of Pod replicas based on resource metrics (such as Central Processing Unit (CPU) utilization, memory utilization, etc.) or other metrics (such as Queries Per Second (QPS)) to achieve resource adjustment. When the application workload increases, Kubernetes can use HPA to expand the number of Pod replicas; when the application workload decreases, Kubernetes can use HPA to reduce the number of Pod replicas. For example, the number of Pods used to generate product order information is 3. During an e-commerce platform's anniversary sale, the workload of the corresponding shopping application will increase, and the demand for generating product order information will increase. Kubernetes can use HPA to expand the number of Pods used to generate product order information from 3 to 10 to cope with the surge in order request pressure, ensure the stability and response speed of the order processing system, and avoid service delays or failures due to Pod overload. After the e-commerce platform's anniversary sale ends, the workload of the corresponding shopping application will decrease, and the demand for generating product order information will also decrease. Kubernetes can use HPA to reduce the number of Pods used to generate product order information from 10 back to 3, saving cluster resources, reducing application operating costs, and avoiding waste caused by idle Pod resources. It should be noted that Pod replicas are identical, interchangeable Pod instances created by the same controller (e.g., Deployment) based on the same Pod template.

[0043] VPA adjusts the resource configuration of individual Pods by analyzing their historical resource usage. For example, VPA adjusts the Requests configuration for CPU and / or memory resources of a single Pod. The Requests configuration refers to the minimum amount of resources a Pod declares to the Kubernetes cluster that must be guaranteed. Another example is the Limits configuration for CPU and / or memory resources of a single Pod. The Limits configuration refers to the maximum resource usage limit set by Kubernetes for a Pod. When Kubernetes detects that a Pod's resource configuration has consistently been lower than its historical actual resource usage over a preset period (e.g., three days), Kubernetes can increase the Pod's resource configuration through VPA; conversely, when Kubernetes detects that a Pod's resource configuration has consistently been higher than its historical actual resource usage over a preset period (e.g., three days), Kubernetes can decrease the Pod's resource configuration through VPA. For example, a Pod might have a Requests configuration of 200m for CPU resources (i.e., 0.2 CPU cores). Over the past week, Kubernetes detected that the actual CPU utilization of this Pod consistently hovered around 500m (i.e., 0.5 CPU cores). This actual CPU utilization was higher than the Pod's CPU resource Requests configuration. Kubernetes can use VPA to increase the Pod's CPU resource Requests configuration from 200m to 500m, providing the Pod with more ample and actual-demand-compliant resource guarantees, thus improving the long-term stability of the service. In the week following the increase in the Pod's CPU resource Requests configuration, Kubernetes detected that the actual CPU utilization of this Pod consistently hovered around 100m (i.e., 0.1 CPU cores). This actual CPU utilization was lower than the Pod's CPU resource Requests configuration. Kubernetes can use VPA to decrease the Pod's CPU resource Requests configuration from 500m to 100m, thereby freeing up a significant amount of reserved but unused CPU resources for the Pod and significantly reducing the overall resource reservation cost of the Kubernetes cluster.

[0044] Specifically, Cache Allocation (CA) adjusts the number of nodes in a Kubernetes cluster by detecting pending Pods that are not scheduled to run on nodes due to insufficient resources. When the node resources in the Kubernetes cluster are insufficient to run newly created Pods, Kubernetes can increase the number of nodes through CA; conversely, when the node resource utilization in the Kubernetes cluster remains below a preset threshold (e.g., 50%), Kubernetes can reduce the number of nodes through CA. For example, consider a cloud-native document writing service with a Kubernetes cluster containing 4 nodes. During the Monday morning rush hour, a large number of users simultaneously edit documents, causing a sharp increase in workload. When Kubernetes attempts to scale up the document collaboration Pods from 10 to 30 using HPA (Hardware Processing), the CPU and memory resources on the existing 4 nodes are already fully utilized, preventing the 20 newly added Pods from being scheduled to run on nodes due to insufficient resources. At this point, Kubernetes detects these unscheduled Pods due to insufficient resources through CA and automatically increases the number of nodes in the cluster from 4 to 8, providing the new Pods with the necessary resources and ensuring smooth and stable service during peak hours. When lunch break arrives, most users stop editing, and the application's workload decreases significantly. Kubernetes uses HPA to reduce the number of Pods from 30 back to 10, resulting in four nodes in the cluster having resource utilization consistently below the preset threshold of 50%. At this point, Kubernetes detects these low-utilization nodes using CA. After confirming that the Pods on these low-utilization nodes can be safely migrated, it automatically reduces the number of nodes in the cluster from eight back to four to save cloud server costs and avoid resource waste caused by idleness.

[0045] Currently, Kubernetes (K8s) performs reactive resource adjustments through its elastic scaling mechanism as follows: First, K8s receives business requests from applications, such as requests from shopping applications to generate product order information. Then, K8s continuously collects resource consumption data (also known as resource metrics, such as CPU utilization and memory utilization) of Pods and / or nodes. Next, K8s compares the collected resource consumption data with a preset target threshold (e.g., 50%) at a preset frequency (e.g., every 15 seconds). If the average resource consumption data over a preset time period (e.g., within 5 minutes) reaches the preset target threshold (e.g., the average resource consumption data is consistently higher or lower than the preset target threshold), K8s is triggered to execute any one of HPA, VPA, or CA elastic scaling actions to achieve resource adjustment. For example, during an e-commerce platform's anniversary sale, a large number of users visit the platform to place orders. A large number of business requests to generate product order information arrive at the K8s service and are load-balanced across three existing Pods used to handle these requests. The processing of these requests by the Pods leads to increased CPU utilization. Kubernetes continuously collects and stores the actual CPU utilization of Pods. Every 15 seconds, it queries the average CPU utilization of a Pod over the past 5 minutes and compares this average CPU utilization with a preset target threshold of 50%. If the average CPU utilization of a Pod consistently exceeds 50%, HPA is used to scale up the number of Pods used to generate product order information from 3 to 10 in order to control the average CPU utilization of the Pods around the preset target threshold of 50%. Ultimately, Kubernetes uses HPA to scale up the number of Pods used to generate product order information from 3 to 10 and schedules the additional 7 Pods to run on the nodes.

[0046] It can be seen that the above-mentioned reactive resource adjustment has the following drawbacks: 1. The elastic scaling action is only triggered when the resource consumption data is higher or lower than the preset target threshold, which will cause the application's resource adjustment to be lagging, thus causing the application's service delay; 2. Only collecting a single resource consumption data and comparing the single resource consumption data with the preset target threshold will result in low accuracy of K8s' scaling decision, which may further lead to resource waste or insufficient resource allocation affecting the application's performance.

[0047] To address the aforementioned issues, this application provides a method for scaling up or down a container orchestration engine cluster. This cluster (i.e., a Kubernetes cluster) includes at least one node, and each node runs at least one Pod. Each Pod implements one or more services. In this method, Kubernetes first acquires resource consumption data, service access data, and other collected data from the nodes and / or Pods in the Kubernetes cluster. Then, the acquired collected data is normalized and input into a hybrid model to obtain load prediction results. Specifically, the load prediction results are obtained by weighting long-term and short-term load prediction results in the hybrid model. The long-term load prediction results are output by the long-term load prediction model within the hybrid model based on the normalized collected data (also called normalized data), and are used to predict the long-term trend of load changes within a preset time period. The short-term load prediction results are output by the short-term load prediction model within the hybrid model based on the normalized collected data (also called normalized data), and are used to predict short-term load fluctuations within a preset time period. After obtaining the load forecast results, Kubernetes determines a scaling strategy (also known as a scaling policy) based on these results and then performs scaling up or down operations on the service based on this strategy. This method allows Kubernetes to adjust application resources in a timely manner, reducing the response latency of Kubernetes scaling up and down, thereby reducing the probability of application service latency. Furthermore, it improves the accuracy of Kubernetes scaling up and down decisions, further enhancing resource utilization and reducing the likelihood of application service degradation.

[0048] To facilitate understanding, the system involved in this application will be explained and described below.

[0049] like Figure 1 As shown, system 10 may include server 100 and device 200. Server 100 is a backend system providing the scaling services for the Kubernetes cluster in this embodiment, and may be a server implementing the scaling method for the Kubernetes cluster. Server 100 may be a single server or a cluster of multiple servers; this embodiment does not limit this. Device 200 may be an electronic device that sends service requests to the service entry point (e.g., Ingress, LoadBalancer, NodePort) of the Kubernetes cluster in server 100, so that the Kubernetes cluster internally routes the service request to the Pod that processes the request. When the workload of the Kubernetes cluster exceeds a preset target threshold, the Kubernetes cluster in server 100 may execute the scaling method for the Kubernetes cluster provided in this embodiment to adjust resources. It is understood that device 200 may be various types of smart terminal devices or a server; this embodiment does not limit this.

[0050] A communication connection can be established between server 100 and device 200 for data exchange. This communication connection can include, but is not limited to: wired connections (e.g., wired connections established via Ethernet), short-range wireless communication connections such as Bluetooth (BT), SparkLink, Wireless Fidelity (Wi-Fi), Near Field Communication (NFC), Infrared (IR), Zigbee, and long-range wireless communication connections. Furthermore, server 100 and device 200 in system 10 can also connect and communicate using any combination of the above methods; this embodiment does not impose any limitations on this. In other words, server 100 and device 200 in system 10 can form a network (i.e., a network) according to a certain communication protocol and networking strategy to achieve mutual communication between server 100 and device 200.

[0051] In some embodiments of this application, server 100 can execute the K8s cluster scaling method provided in the embodiments of this application. That is, server 100 integrates a K8s cluster plugin service that encapsulates the K8s cluster scaling method provided in the embodiments of this application. Server 100 can execute the K8s cluster scaling method provided in the embodiments of this application to first obtain resource consumption data, service access data, and other collected data of nodes and / or Pods in the K8s cluster. Then, the obtained collected data is normalized and input into a hybrid model to obtain a load prediction result. After obtaining the load prediction result, server 100 can also determine a scaling strategy based on the load prediction result and perform scaling up or scaling down operations on the service based on the scaling strategy.

[0052] Figure 2 This example illustrates the process of a scaling method for a container orchestration engine cluster provided in an embodiment of this application. The container orchestration engine cluster (i.e., a Kubernetes cluster) includes at least one node, and each node runs at least one Pod. A Pod is used to implement one or more services. Figure 2 The scaling mechanism of the container orchestration engine cluster shown includes: a data acquisition module, a data processing module, a hybrid model, a policy generation module, and a policy execution module. The hybrid model includes: a long-term load forecasting model, a short-term load forecasting model, and a weighted module. Optionally, the policy generation module includes a scheduling model. Figure 2 The model represented by the dashed box is an optional model, and the step represented by the dashed line is an optional step.

[0053] like Figure 2As shown, Kubernetes first acquires data through its data acquisition module. This acquired data includes: resource consumption data (also known as resource metrics) of nodes and / or Pods, and business access data. Resource consumption data refers to measurements of the core resource usage of various components (such as nodes and Pods) within the Kubernetes cluster, including but not limited to one or more of the following: computing resource consumption data, storage resource consumption data, and network resource consumption data. For example, computing resources include: CPU resources and Graphics Processing Unit (GPU) resources; computing resource consumption data includes: CPU utilization and GPU utilization. Storage resources include: memory resources and external storage resources; storage resource consumption data includes: memory utilization and disk utilization. Network resources include: bandwidth and network connections; network resource consumption data includes: bandwidth utilization and data transmission / reception volume per unit time. Business access data refers to the access data of the business implemented by a Pod. That is, data related to business functionality, performance, and user experience generated when external users or internal services access Pods running on Kubernetes. It can reflect the frequency of business access and includes, but is not limited to, one or more of the following: business QPS, business implementation success rate, business API call success rate, and business API call latency. It should be noted that, see [link to relevant documentation] Figure 2Resource consumption data in the collected data can be obtained through Kubernetes' built-in data collection components (such as the node agent kubelet) and / or external data collection components (such as Prometheus). Business access data in the collected data can be obtained through one or more of the following methods: reporting by the Software Development Kit (SDK) and extraction of log keywords. Obtaining business access data through SDK reporting can be achieved by integrating a dedicated SDK into the business operation logic. This SDK actively collects business access-related metrics (such as QPS and request volume) when the business processes requests and reports these metrics to generate business access data. Obtaining business access data through log keyword extraction can be achieved by outputting logs containing access data (such as request path, status code, and time consumption) when the business processes requests. Log collection tools are used to parse, filter, and statistically analyze the data according to keywords to extract relevant business access data. It is understood that the format of the collected data obtained by Kubernetes is uniform so that Kubernetes can preprocess the collected data to obtain normalized data. For example, the format of the collected data is OpenMetrics. During the data acquisition process, resource consumption data is acquired more frequently than business access data. This is because resource consumption data changes constantly, while business access data only changes when a business request is processed. Controlling the acquisition frequencies of resource consumption data and business access data differently can save power consumption during data acquisition. For example, Kubernetes acquires resource consumption data every 5 seconds and business access data every minute. Besides resource consumption data and business access data, the acquired data may also include other data, such as timestamps, holiday markers, and other business scenario data. This application embodiment does not limit the content of the acquired data.

[0054] After acquiring the collected data, Kubernetes (K8s) preprocesses the data through its data processing module to obtain preprocessed normalized data, providing structured input for subsequent hybrid models used to predict load characteristics. For example, for continuous numerical indicators such as resource consumption data and business access data, K8s can convert data within a sliding window of a preset length (e.g., 5 minutes) into a standard distribution with a mean of 0 and a standard deviation of 1 (i.e., Z-score normalization), eliminating dimensional differences between different data. For instance, if the CPU utilization within a 5-minute sliding window is 20%, 30%, and 40%, the average CPU utilization within this window is 30, and the standard deviation is 10; the Z-score normalized CPU utilization is converted to -1, 0, and 1, respectively. For discrete numerical indicators such as holiday markers in business scenario data, K8s can use One-Hot encoding to convert the data into a binary vector with only one 1 bit and the rest 0 bits, uniquely identifying this discrete numerical indicator. For example, the Spring Festival can be converted into a [1, 0, 0] binary vector, Women's Day into a [0, 1, 0] binary vector, and the Dragon Boat Festival into a [0, 0, 1] binary vector. For timestamps in business scenario data, Kubernetes can obtain the patterns in the time series through sin / cos periodic encoding. It should be noted that the preprocessed normalized data is three-dimensional data, used to indicate the actual load results of the business at various times within time period A, where time period A is the time period for acquiring the collected data. For example, the normalized data is a three-dimensional tensor structure, with the three dimensions being the time dimension, the node dimension, and the load dimension, respectively. This load can be measured through resource consumption data (e.g., CPU utilization) and / or business access data (e.g., QPS).

[0055] After obtaining the preprocessed normalized data, Kubernetes can input the normalized data into the hybrid model to obtain the load prediction result. It should be noted that the hybrid model is used to predict load characteristics and includes: a long-term load prediction model, a short-term load prediction model, and a weighting module. The long-term load prediction model outputs long-term load prediction results based on the normalized data to predict the long-term trend of load changes within time period B. The short-term load prediction model outputs short-term load prediction results based on the normalized data to predict short-term load fluctuations within time period B. The weighting module weights the long-term and short-term load prediction results to obtain the load prediction result. Time period B is any time period from the current time to the future. The above load prediction result is also three-dimensional data, used to indicate the load prediction result (also called load value) of the service at various times within time period B. This load prediction result can be measured by resource consumption data (e.g., CPU utilization) and / or service access data (e.g., QPS). It is understood that the content included in the load prediction result can be determined based on the parameters of the hybrid model or expert experience; this embodiment does not impose such limitations.

[0056] For example, the long-term load prediction model in the hybrid model can be a Long Short-Term Memory (LSTM) network model, and the short-term load prediction model in the hybrid model can be a Transformer model. The LSTM model and the Transformer model mentioned above are actually two modules in the hybrid model. The LSTM model module adopts a bidirectional structure (i.e., it generates output by simultaneously using information from both the past and future directions) to obtain the long-term trend of historical load changes. The Transformer model module identifies short-term load mutations through a multi-head self-attention mechanism (i.e., it splits self-attention into multiple groups, allowing the Transformer model to learn multiple relationships between different positions in the sequence from multiple independent perspectives, and finally fuses the results).

[0057] In the hybrid model, the long-term load forecasting model and the short-term load forecasting model can have different weights to accurately predict load characteristics. For example, 'a' represents the weight of the long-term load forecasting model, indicating the weight of the long-term load forecast in the overall load forecasting result; 1-a represents the weight of the short-term load forecasting model, indicating the weight of the short-term load forecast in the overall load forecasting result; 'x' represents the long-term load forecast result for service A at time A; 'y' represents the short-term load forecast result for service A at time A; and time A is the time within the aforementioned time period B. The load forecast result 'z' for service A at time A can be determined based on the following formula: The value of 'a' is determined by the load characteristics of the current business, which can be measured by the relative entropy (KL divergence) between the long-term and short-term load prediction models. In other words, the value of 'a' is determined by the relative entropy between the long-term and short-term load prediction models. It should be noted that the relative entropy (KL divergence) of the two models measures the difference between the probability distributions output by the two models based on the same set of inputs. A larger KL divergence value indicates a greater difference between the short-term and long-term load prediction results, meaning the current business load characteristics are more inclined towards short-term burst loads, thus requiring a higher weight to the short-term load prediction model to ensure the accuracy of the load prediction results. Conversely, a smaller KL divergence value indicates a smaller difference between the short-term and long-term load prediction results, meaning the current business load characteristics are more inclined towards long-term load trends, thus requiring a higher weight to the long-term load prediction model. In some embodiments, when the KL divergence of the two models exceeds a preset threshold (e.g., 0.3), the value of 'a' can be determined by the following formula: Where b is the smoothing coefficient, which is a factor used to adjust the influence of higher weights on the KL divergence in long-term load forecasting models. (For example It is understandable that the larger the value of b, the greater the influence of the KL divergence on the higher weights in the long-term load forecasting model. c is the KL divergence between the long-term and short-term load forecasting models. As can be seen from the above formula, when the KL divergence between the two models exceeds a preset threshold (e.g., 0.3), the greater the KL divergence between the two models, the smaller the weight of the higher-order long-term load prediction model (i.e., the value of 'a').

[0058] After obtaining the load prediction results, Kubernetes can determine the scaling strategy based on these results through the strategy generation module. It should be noted that this scaling strategy can be determined based on the scheduling model or expert experience; this embodiment does not impose any limitations on this. In fact, the scheduling model is a multi-objective optimization function. A multi-objective optimization function is a mathematical optimization model that includes two or more conflicting objective functions that cannot be simultaneously optimized, seeking a compromise optimal solution under given constraints. For example, the multi-objective optimization function includes two conflicting objective functions: resource cost and service latency penalty, which cannot be simultaneously optimized. These two objective functions can have different weights, which quantify the relative importance of resource cost and service latency penalty. A larger resource cost weight indicates a higher priority in reducing resource overhead, while a larger latency penalty weight indicates a higher priority in ensuring low service latency. Under given constraints, the multi-objective optimization function can be used to find a compromise optimal solution that reduces resource overhead while ensuring low service latency, such as a scaling strategy that balances stability and cost-effectiveness. Currently, the reactive resource scaling described above triggers any one of the HPA, VPA, or CA scaling strategies independently, without comprehensively considering global factors such as node heterogeneity, Pod affinity, and cold start latency. This may lead to scaling strategy conflicts, resulting in low resource utilization after the scaling strategy is executed. For example, without considering the factor of excessively long node cold start latency, when the application load suddenly increases and needs to be scaled up, Kubernetes triggers the CA scaling strategy. However, the cold start latency required for the creation and readiness of a new node exceeds a preset time (e.g., 90 seconds). Within this time window, the current node's resources are insufficient, causing business requests to be pushed back up, response times to slow down, and ultimately service degradation, affecting user experience. To solve the above problems, in some embodiments, the scaling strategy is determined based on a scheduling model. In this way, the accuracy of Kubernetes' scaling decisions is improved, further improving resource utilization and reducing the possibility of application service degradation. Specifically, Kubernetes can input the load prediction results into the scheduling model in the strategy generation module, and the scheduling model outputs the scaling strategy with a first condition as a constraint. The first condition includes, but is not limited to, one or more of the following: the new node is of a specific type (e.g., a GPU node type used for model inference services) (i.e., limiting the heterogeneity of the new node); the cold start latency of the new node does not exceed threshold A (e.g., 90 seconds); the new Pod is deployed on node A; and the cold start latency of the new Pod does not exceed threshold B (e.g., 80 seconds). The new Pod is deployed on node A because: node A and the new Pod have affinity, and / or, the Pod on node A and the new Pod have affinity. As mentioned above, if the load prediction result output by the hybrid model reaches the preset target threshold, it will trigger the scaling strategy of the Kubernetes cluster. There are one or more scaling strategies for the Kubernetes cluster.For example, if the CPU utilization rate in the load prediction results exceeds a preset threshold (e.g., 70%), HPA scaling strategy A is triggered. Simultaneously, if the memory utilization rate in the load prediction results exceeds a preset threshold (e.g., 80%), VPA scaling strategy B is triggered. When multiple scaling strategies are triggered by the load prediction results, the scheduling model outputs a scaling strategy based on a first condition. This specifically includes: first, selecting scaling strategies that meet the first condition; then, simulating the execution of the selected scaling strategies through a pre-scheduling simulation module; finally, evaluating the resource utilization and / or service degradation risk of the simulated execution results of the scaling strategies to obtain a scaling strategy that balances stability and cost-effectiveness. This scaling strategy is then determined as the scaling strategy to be executed, adapting to business needs. Optionally, the strategy generation module includes the aforementioned pre-scheduling simulation module. Figure 2 (Not shown in the diagram). Optionally, if there are multiple first conditions mentioned above, these first conditions can each have their own weight, and Kubernetes can filter scaling strategies based on these weighted first conditions. It should be noted that Pod cold start time refers to the total time from Pod creation to container startup and service readiness. Node cold start time refers to the total time from triggering node creation to the new node joining the cluster and being able to schedule Pods normally. Pod affinity refers to the strategy used in Kubernetes to control the scheduling position of Pods, such as prioritizing scheduling new Pods to nodes close to specific Pods. Node heterogeneity refers to the differences in hardware (such as CPU architecture, GPU, memory size), software (such as operating system, kernel version), and topology (such as availability zone, data center) among nodes in a Kubernetes cluster.

[0059] After determining the scaling strategy, Kubernetes (K8s) can use the strategy execution module to perform scaling up or scaling down operations on services based on the scaling strategy. Specifically, K8s can use the strategy execution module to generate strategy execution instructions for services based on the scaling strategy, and then the K8s cluster (i.e., Figure 2 The container orchestration engine cluster can execute commands based on policies to perform scaling up or scaling down operations on services. It should be noted that scaling up operations include, but are not limited to, one or more of the following: increasing the number of Pod replicas via HPA, increasing the resource configuration of a single Pod via VPA, and increasing the number of nodes in the cluster via CA. Scaling down operations include, but are not limited to, one or more of the following: decreasing the number of Pod replicas via HPA, decreasing the resource configuration of a single Pod via VPA, and decreasing the number of nodes in the cluster via CA.

[0060] Optionally, after performing scaling up or scaling down operations on a service, Kubernetes can collect the actual load results after the scaling up or scaling down operations through the data acquisition module, and adjust the weight parameters of the hybrid model based on these actual load results. In this way, the hybrid model can perform adaptive optimization to improve the accuracy of load prediction results, further enhancing the accuracy of Kubernetes' scaling up and scaling down decisions. It should be noted that the actual load results correspond to the load prediction results and are also three-dimensional data, used to indicate the load results (also called load values) of the service at various times within the current time period C. These can be measured through resource consumption data (e.g., CPU utilization) and / or service access data (e.g., QPS, service interface call latency, service implementation success rate). The aforementioned weight parameters can be used to adjust the weights of the long-term load prediction model and the short-term load prediction model mentioned above. It is understood that, in addition to the weight parameters, Kubernetes can also adjust other parameters of the hybrid model based on the actual load results, such as the window length of the long-term load prediction model (e.g., the LSTM model) and the number of heads of the short-term load prediction model (e.g., the Transformer model).

[0061] In some embodiments, when Kubernetes determines the scaling strategy based on the scheduling model, after collecting the actual load results after performing scaling up or scaling down operations on the service, Kubernetes can also adjust the parameters of the scheduling model based on the actual load results, such as the weights of the multiple first conditions mentioned above. In this way, the scheduling model can perform adaptive optimization to improve the decision accuracy of Kubernetes scaling up and down.

[0062] In some embodiments, after adjusting the weight parameters of the hybrid model based on actual load results, Kubernetes can use the adjusted hybrid model to obtain the load forecast result in the next round of load forecasting. Then, based on the next round of load forecast results, it determines the scaling-down strategy for the next round. If the deviation between the actual load result after implementing the scaling-down strategy and the load forecast result exceeds a threshold C (e.g., 50%), the hybrid model version is rolled back to the version before the weight parameter adjustment to ensure the accuracy of the load forecast results and the system (e.g., ...). Figure 1 The stability of system 10) during hybrid model iteration is shown. It should be noted that K8s can save all versions of the hybrid model and actual load results within a preset time period (e.g., 24 hours).

[0063] See Figure 2A Kubernetes (K8s) cluster consists of a database (etcd), node agents, a scheduler (kube-scheduler), and an Application Programming Interface (API) server (APIserver). The database stores metadata (resource identification information, such as resource names), resource configurations (user- or controller-defined expected operating rules, such as the expected number of Pod replicas, HPA thresholds and scaling policies, Pod affinity / anti-affinity rules, and resource Requests / Limits configurations), and status data (the actual running status of K8s, such as node status, the node where the Pod is running, Pod status, and the actual number of Pod replicas). The node agents receive Pod scheduling instructions from the API server, manage the creation, startup, stopping, and destruction of Pods on their nodes, and report node status, node capacity and available resources, and Pod running status to the API server. The scheduler selects worker nodes for newly created Pods that have not yet been assigned nodes. The API server is the core of the K8s cluster's control plane, providing the K8s API; all queries and modifications to the K8s cluster must be performed through the API server. The API server is used for authentication, authorization, and access control, and saves the final state of resource objects (such as the expected and actual states of Pods, nodes, etc.) to the database.

[0064] The scaling method for a container orchestration engine cluster provided in this application allows Kubernetes (K8s) to predict long-term load trends based on a long-term load prediction model within a hybrid model, and to predict short-term load spikes based on a short-term load prediction model. The combined load prediction results from both models are then used to perform scaling up or down operations on the application, enabling timely resource adjustments and reducing the response latency of K8s scaling, thereby lowering the probability of application service latency. Furthermore, K8s can perform load prediction based on various collected data, such as node and / or Pod resource consumption data and service access data, improving the accuracy of load prediction results and enhancing the precision of K8s scaling decisions. This further improves resource utilization and reduces the likelihood of application service degradation.

[0065] Currently, the reactive resource adjustment described above is based on manually preset static target thresholds to determine whether to trigger scaling strategies. These static thresholds are not suitable for dynamic business scenarios such as e-commerce flash sales, fluctuations in artificial intelligence (AI) training tasks, and trending social media events. To address this issue, in some embodiments, after performing scaling operations on a service, Kubernetes can collect the actual load results after the scaling operation and adjust the target threshold for triggering scaling strategies based on these results. This makes the target threshold applicable to dynamic business scenarios such as e-commerce flash sales, fluctuations in AI training tasks, and trending social media events. This improves the accuracy of Kubernetes' scaling decisions, further enhances resource utilization, and reduces the likelihood of application service degradation.

[0066] The following is a schematic diagram of the software architecture provided in the embodiments of this application.

[0067] Taking the scaling strategy determined based on the scheduling model as an example, Figure 3 An exemplary embodiment of the present application illustrates a system architecture for a container orchestration engine.

[0068] like Figure 3 As shown, the system architecture of a container orchestration engine (i.e., Kubernetes) can include a data acquisition and processing module, a load prediction module, a policy generation module, a policy execution module, and a container orchestration engine cluster (i.e., a Kubernetes cluster). The data acquisition and processing module includes a data acquisition module, a data processing module, and a load database. The load prediction module includes a hybrid model, a load prediction result cache, and a feedback learning module. The policy generation module includes a scheduling model and a policy evaluation module. The policy execution module includes an instruction generation module and an instruction issuance module. The container orchestration engine cluster (i.e., a Kubernetes cluster) includes a database (etcd), node agents, a scheduler (kube-scheduler), and an API server.

[0069] In the data acquisition and processing module, Kubernetes first acquires the collected data through the data acquisition module. The details of this module and the collected data can be found in the previous sections. Figure 2 The relevant implementation examples will not be repeated here. Then, Kubernetes preprocesses the collected data through a data processing module to obtain preprocessed normalized data. This data processing module and the normalized data can be found in the previous sections. Figure 2The relevant embodiments are not described in detail here. Next, Kubernetes stores the normalized data into a load database according to the time sequence relationship, so that the normalized data stored in the load database can be input into the hybrid model to predict the load of the Kubernetes cluster. For example, during a major promotional period on an e-commerce platform, the workload of the corresponding shopping application will surge. During the data acquisition process, Kubernetes (K8s) uses the Prometheus component deployed in the K8s cluster to collect resource consumption data such as the average CPU utilization of 85%, the average memory utilization of 78%, and the peak network input / output (I / O) of 2Gbps, reflecting the infrastructure load. K8s also obtains business access data such as the QPS of business requests exceeding 100,000 times per second, the success rate of order payment implementation of 99.2%, and the API latency P99 value (i.e., the value of 99% of request response timestamps less than 800 milliseconds) of order payment through business SDK reporting and / or log keyword extraction. K8s also obtains business scenario data such as promotional timestamps, holiday markers, and a 40% increase in traffic from competitor activities. During data processing, for continuous numerical indicators such as resource consumption data and business access data, Kubernetes (K8s) converts the data within a 5-minute sliding window into a standard distribution with a mean of 0 and a standard deviation of 1 (i.e., Z-score normalization), eliminating the dimensional differences between different data. For discrete numerical indicators such as holiday markers in business scenario data, K8s uses One-Hot encoding to convert the data into a binary vector with only one 1 bit and the rest 0 bits. For timestamps in business scenario data, K8s uses sin / cos periodic encoding to obtain the patterns of the time series. After normalizing the collected data, K8s constructs the obtained normalized data into a three-dimensional tensor structure. These three dimensions are the time dimension, the node dimension, and the load dimension. The three-dimensional tensor has 30 time steps in the time dimension, 50 nodes in the node dimension, and 12 features in the load dimension (i.e., each node has 12 features at each time step).

[0070] In the load forecasting module, Kubernetes retrieves normalized data from the load database and inputs it into the hybrid model to obtain the load forecasting results. The hybrid model and load forecasting results can be found in the previous sections. Figure 2The relevant embodiments are not described in detail here. Then, Kubernetes stores the load prediction results in the load prediction result cache. It should be noted that the hybrid model can retrieve normalized data from the load database through the first calling module for load prediction. For example, the long-term load prediction model in the hybrid model is an LSTM model, and the short-term load prediction model is a Transformer model. The LSTM model has 128 hidden units and uses a bidirectional structure to obtain the long-term trend of historical load changes within 15 minutes. The Transformer model identifies short-term load mutations through a 4-head self-attention mechanism. The initial weight 'a' of the LSTM model is 0.7; after 3 minutes, it is detected that the KL divergence 'b' between the real-time calculated LSTM model and the Transformer model has changed to 0.45. This KL divergence value exceeds the threshold of 0.3, triggering the adjustment of the LSTM model's weight 'a' based on the following formula: The smoothing coefficient c is set to 0.5, and the weight a of the adjusted LSTM model is set to 0.38, making the current business load characteristics more inclined towards short-term burst loads. The load prediction results are presented in the form of a three-dimensional tensor, with three dimensions: time, nodes, and load. The three-dimensional tensor has 64 time steps in the time dimension, 256 nodes in the node dimension, and 128 features in the load dimension. The load dimension of the load prediction results includes the QPS feature. The LSTM model prediction results indicate that the QPS of the order payment business will increase linearly with a slope of 0.8 in the next 10 minutes. The Transformer model prediction results indicate that the QPS of the order payment business will surge by 120% in three minutes. The load prediction result of the order payment business at this moment is obtained by weighting the 38% LSTM model prediction results and the 62% Transformer model prediction results.

[0071] In the strategy generation module, Kubernetes retrieves load prediction results from the load prediction result cache and inputs these results into the scheduling model. The scheduling model outputs scaling strategies based on a first condition as a constraint. This scheduling model and the first condition can be referenced earlier. Figure 2The relevant embodiments are not described in detail here. It should be noted that the scheduling model can retrieve the load prediction result from the load prediction result cache through the second calling module to generate the scaling strategy. For example, the scheduling model sets the weight of resource cost to 0.6 and the weight of service latency penalty to 0.4. It also sets the constraints of the scaling strategy, including: the type of the new node is a GPU node used for model inference services; the cold start latency of the new node does not exceed 90 seconds; and the new Pods used for order processing and the Pods used for inventory processing are deployed on the same node. K8s filters out scaling strategies that meet the above constraints, then simulates the execution of the selected scaling strategies through the pre-scheduling simulation module, and evaluates the resource utilization and / or service degradation risk of the simulated execution results of the scaling strategies to obtain a scaling strategy that balances stability and cost-effectiveness. The scaling-up strategy includes expanding the number of Pod replicas by 50% through HPA and adding GPU nodes through CA. The simulated execution results of this scaling-up strategy show a resource utilization rate of 91%, a cold start time of 75 seconds for new nodes, a Service Level Agreement (SLA) compliance rate of 98%, and a node overload risk reduced to below 2%, while avoiding resource waste caused by excessive redundancy.

[0072] In the policy execution module, Kubernetes (K8s) generates policy execution instructions for specific business operations based on scaling strategies through the instruction generation module. Then, K8s distributes these instructions to the container orchestration engine cluster (i.e., the K8s cluster) via the instruction distribution module, enabling the K8s cluster to perform scaling operations on the business based on these instructions. For example, K8s creates 15 Pod replicas in batches via the K8s API and coordinates with CA to scale up two GPU nodes. It also uses a resource scheduling algorithm to prioritize deploying newly added Pods on nodes with loads below 60%, effectively avoiding hotspot issues and maintaining load balance. Five minutes after the K8s cluster performs scaling operations on the business, the QPS for the order payment service reaches 95% of the predicted load, CPU utilization remains at 88%, the API latency P99 value for the order payment service decreases to 350 milliseconds, and the success rate of the order payment service remains at 99.5%.

[0073] After performing scaling up or scaling down operations on a service, the Kubernetes cluster can feed back the actual load results to the policy evaluation module in the policy generation module and / or the feedback learning module in the load prediction module. On one hand, Kubernetes can use the feedback learning module to adjust the parameters of the hybrid model (e.g., the weight parameters of the long-term load prediction model) based on the actual load results, thereby updating the hybrid model version; and / or, Kubernetes can use the feedback learning module to adjust the parameters of the scheduling model (e.g., the weights of the multiple first conditions mentioned above) based on the actual load results, thereby updating the scheduling model version. On the other hand, after the K8s cluster feeds back the actual load results to the feedback learning module, the feedback learning module can also obtain the load prediction results in the load prediction result cache, and determine whether the hybrid model version needs to be rolled back based on whether the deviation between the actual load results and the load prediction results exceeds the threshold C (e.g., 50%) mentioned above. If it exceeds the threshold C, the hybrid model version needs to be rolled back to ensure the stability of the system during the hybrid model iteration. And / or, after the K8s cluster feeds back the actual load results to the policy evaluation module, the policy evaluation module can also obtain the simulation execution results of the scaling policy of the scheduling model, and determine whether the scheduling model version needs to be rolled back based on whether the deviation between the actual load results and the simulation execution results of the scaling policy exceeds the threshold D (e.g., 50%). If it exceeds the threshold D, the scheduling model version needs to be rolled back to ensure the stability of the system during the scheduling model iteration. It should be noted that the Kubernetes cluster can directly feed back the actual load results to the policy evaluation module in the policy generation module and / or the feedback learning module in the load prediction module, or it can feed back the actual load results to the policy evaluation module in the policy generation module and / or the feedback learning module in the load prediction module through the data acquisition module. This embodiment of the application does not impose any limitations on this. For example, the long-term load prediction model in the hybrid model is an LSTM model, and the short-term load prediction model in the hybrid model is a Transformer model. The weight parameter of the LSTM model is 0.38, and the window length of the LSTM model is 30 time steps. After performing scaling up or scaling down operations on the business, the Kubernetes cluster adds the actual load results to the training set of the hybrid model and extends the window length of the LSTM model from 30 time steps to 35 time steps through the Proximal Policy Optimization (PPO) reinforcement learning algorithm to enhance the ability to obtain long-term load change trends. If the deviation between the actual load results and the load prediction results exceeds the threshold of 10%, the Kubernetes cluster rolls back the hybrid model version to the version with a weight parameter of 0.5 one hour ago.

[0074] Experiments showed that, when using traditional HPA for reactive resource scaling, the response latency for scaling up a Kubernetes cluster was 3-5 minutes, resource utilization was 65%-75%, and the service degradation rate was 12%. In contrast, when using the scaling up / down method of the container orchestration engine cluster provided in this application, the response latency for scaling up a Kubernetes cluster was 45 seconds, resource utilization was 88%-92%, the service degradation rate was 1.2%, and the load prediction accuracy was as high as 94.3%. Therefore, compared to reactive resource scaling using traditional HPA, the scaling up / down method of the container orchestration engine cluster provided in this application reduces the response latency for scaling up a Kubernetes cluster, improves resource utilization, reduces the service degradation rate, ensures SLA compliance, and enhances the decision-making accuracy for scaling up / down Kubernetes clusters under sudden load scenarios.

[0075] In some embodiments, the scaling method for container orchestration engine clusters provided in this application can be encapsulated as a pluggable K8s cluster plugin service.

[0076] It is understood that this application does not limit the specific models used for the long-term load forecasting model and / or the short-term load forecasting model in the hybrid model. This application also does not limit the type or quantity of data collected.

[0077] Based on the above embodiments, this embodiment provides a hardware structure for a server 100.

[0078] Figure 4 This is a schematic diagram of the structure of a server 100 provided in an embodiment of this application.

[0079] like Figure 4 As shown, server 100 may include processor 110 and memory 120. Components within server 100 are connected and communicate via a bus. Processor 110 is coupled to memory 120, which stores computer programs or instructions and / or data. Processor 110 executes the computer programs or instructions stored in memory 120, or reads the data stored in memory 120, to perform the methods or functions of the modules described in the above embodiments. Optionally, there may be one or more processors 110. Optionally, there may be one or more memories 120. Optionally, the memory 120 may be integrated with the processor 110, or may be separately configured.

[0080] Processor 110 may include one or more processing units, such as application processors (APs), modem processors, GPUs, image signal processors (ISPs), controllers, memory, video codecs, digital signal processors (DSPs), baseband processors, and / or neural network processing units (NPUs). These different processing units may be independent devices or integrated into one or more processors.

[0081] The controller can serve as the nerve center and command center of the server 100. The controller can generate operation control signals based on instruction opcodes and timing signals to control instruction fetching and execution.

[0082] The processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory can store instructions or data that the processor 110 has just used or that are used repeatedly. If the processor 110 needs to use the instruction or data again, it can retrieve it directly from the memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0083] The memory 120 may include one or more random access memory (RAM) and one or more non-volatile memory (NVM). The RAM can be directly read and written by the processor 110. The RAM can be used to store executable programs (e.g., machine instructions) of the operating system or other running programs. The NVM can also store executable programs and user and application data. Executable programs, i.e., user data, stored in the NVM can be pre-loaded into the RAM for direct reading and writing by the processor 110.

[0084] It should be understood that, Figure 4 The server 100 shown is merely an example, and server 100 can have more than... Figure 4 The more or fewer components shown can be combined into two or more components, or they can have different component configurations. Figure 4 The various components shown can be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and / or application-specific integrated circuits.

[0085] In addition, this application also provides a computer-readable storage medium storing computer instructions, which, when executed on a computer, cause the methods in the various method embodiments of this application to be performed.

[0086] This application also provides a computer program product, which includes computer program code or instructions, such that when the computer program code or instructions are run on a computer, the methods in the various method embodiments of this application are executed.

[0087] Furthermore, this application also provides a chip including a processor. A memory for storing a computer program is provided independently of the chip, and the processor is used to execute the computer program stored in the memory to perform the methods in the various method embodiments of this application.

[0088] Furthermore, the chip may also include a communication interface. The communication interface may be an input / output interface or an interface circuit, etc. Furthermore, the chip may also include a memory.

[0089] It should be understood that the processor in the embodiments of this application can be an integrated circuit chip with the ability to process signals. In implementation, each step of the above method embodiments can be completed by the integrated logic circuits in the processor's hardware or by instructions in software form. The processor can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. A general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the embodiments of this application can be directly implemented by a hardware encoding processor, or implemented by a combination of hardware and software modules in the encoding processor. The software modules can be located in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in memory, and the processor reads the information in the memory and, in conjunction with its hardware, completes the steps of the above method.

[0090] The memory in this application embodiment can be volatile memory or non-volatile memory, or it can include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), and synchronous link dynamic memory (SLDRAM).

[0091] It should be noted that when the processor is a general-purpose processor, DSP, ASIC, FPGA, or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component, the memory can be integrated into the processor.

[0092] It should also be noted that the memory described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0093] The above embodiments further illustrate the purpose, technical solution, and advantages of the present invention. It should be understood that the above descriptions are merely embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for scaling up and down a container orchestration engine cluster, characterized in that, The container orchestration engine cluster includes at least one node, and each node runs at least one container group pod. A pod is used to implement one or more business functions. The method includes: Acquire collected data, which includes: resource consumption data of the node and / or the pod, and business access data, wherein the business access data is the accessed data of the business implemented by the pod; The collected data is preprocessed to obtain preprocessed normalized data; The normalized data is input into a hybrid model to obtain the target load prediction result. The hybrid model includes a long-term load prediction model, a short-term load prediction model, and a weighting module. The long-term load prediction model outputs a long-term load prediction result based on the normalized data to predict the long-term load trend within a first time period. The short-term load prediction model outputs a short-term load prediction result based on the normalized data to predict short-term load fluctuations within the first time period. The weighting module weights the long-term load prediction result and the short-term load prediction result to obtain the target load prediction result. A first scaling-up strategy is determined based on the target load prediction results; Based on the first scaling strategy, the service is subjected to scaling up or scaling down operations.

2. The method according to claim 1, characterized in that, The determination of the first scaling strategy based on the target load prediction result specifically includes: The target load prediction result is input into the scheduling model, and the scheduling model outputs the first scaling strategy with a first condition as a constraint; wherein, the first condition includes one or more of the following: the type of the new node is a specific type, the cold start latency of the new node does not exceed a first threshold, the new pod is deployed on the first node, the cold start latency of the new pod does not exceed a second threshold; the first node and the new pod have affinity, and / or, the pod on the first node and the new pod have affinity.

3. The method according to claim 1 or 2, characterized in that, The target load prediction result is three-dimensional data, used to indicate the load prediction result of the service at each time within the first time period.

4. The method according to claim 3, characterized in that, The weighting of the long-term load forecast result and the short-term load forecast result to obtain the target load forecast result specifically includes: Determine the value of 'a', which indicates the weight of the long-term load forecast in the target load forecast result; The target load prediction result z is obtained based on the following formula: ； Where x is the long-term load forecast result of the first service at the first time, y is the short-term load forecast result of the first service at the first time, and the first time is the time within the first time period.

5. The method according to claim 4, characterized in that, The value of 'a' is determined by the relative entropy of the long-term load forecasting model and the short-term load forecasting model.

6. The method according to claim 5, characterized in that, When the relative entropy is greater than the third threshold, the value of 'a' is determined by the following formula: Where b is a smoothing coefficient, used to adjust the degree of influence of the relative entropy on the value of a. ,and c is the relative entropy. .

7. The method according to any one of claims 1-6, characterized in that, The method further includes: Collect the actual load results after performing scaling up or scaling down operations on the service; The weight parameters of the hybrid model are adjusted based on the actual load results.

8. The method according to claim 7, characterized in that, After adjusting the weight parameters of the hybrid model based on the actual load results, the method further includes: In the next round of load forecasting, the adjusted hybrid model is used to obtain the target load forecasting results for the next round; The first scaling-up strategy for the next round will be determined based on the target load prediction results of the next round. If the actual load result after implementing the first scaling strategy in the next round deviates from the target load prediction result in the next round by more than the fourth threshold, then the version of the hybrid model will be rolled back to the version before the weight parameters were adjusted.

9. The method according to any one of claims 1-8, characterized in that, The normalized data is three-dimensional data, used to indicate the actual load results of the service at each time point within the second time period, which is the time period for acquiring the collected data.

10. The method according to any one of claims 1-9, characterized in that, The resource consumption data includes one or more of the following: computing resource consumption data, storage resource consumption data, and network resource consumption data.

11. The method according to any one of claims 1-10, characterized in that, The business access data includes one or more of the following: the query per second (QPS) of the business, the success rate of the business implementation, the success rate of the business interface call, and the latency of the business interface call.

12. A scaling device for a container orchestration engine cluster, characterized in that, The container orchestration engine cluster includes at least one node, and each node runs at least one container group pod. A pod is used to implement one or more services. The device includes: The data acquisition module is used to acquire collected data, which includes: resource consumption data of the node and / or the pod, and business access data, wherein the business access data is the accessed data of the business implemented by the pod; The data processing module is used to preprocess the collected data to obtain preprocessed normalized data; A hybrid model is used to receive the normalized data and output a target load prediction result. The hybrid model includes a long-term load prediction model, a short-term load prediction model, and a weighting module. The long-term load prediction model is used to output a long-term load prediction result based on the normalized data to predict the long-term trend of load changes within a first time period. The short-term load prediction model is used to output a short-term load prediction result based on the normalized data to predict short-term load fluctuations within the first time period. The weighting module is used to weight the long-term load prediction result and the short-term load prediction result to obtain the target load prediction result. The strategy generation module is used to determine a first scaling strategy based on the target load prediction result; The strategy execution module is used to perform expansion or reduction operations on the service based on the first expansion / reduction strategy.

13. An electronic device, characterized in that, The electronic device includes one or more processors and one or more memories; wherein the one or more memories are coupled to the one or more processors, and the one or more memories are used to store a computer program that, when executed by the one or more processors, causes the electronic device to perform the method as described in any one of claims 1-11.

14. A computer storage medium, characterized in that, The computer storage medium stores a computer program that, when executed by a processor, causes the electronic device to perform the method as described in any one of claims 1-11.

15. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it causes the electronic device to perform the method as described in any one of claims 1-11.

Images