Off-line task hybrid deployment method for burst load

Abstract

The application discloses a method for mixed deployment of offline tasks facing burst load, which comprises a preprocessing stage and an updating stage. In the preprocessing stage, an offline mixed deployment control system is constructed in a Kubernetes cluster, so as to dynamically adjust the offline service Pod instances on the server. In the updating stage, at each monitoring interval, the workload prediction model is updated by using the historical workloads of the last K monitoring intervals of the online service through the offline mixed deployment control system, and the number of online service Pods required in the next monitoring interval is quickly predicted through the workload prediction model, the resource matching model and the burst detection method. According to the prediction result and the remaining cluster resources, the available Pod instances of the offline service in the next monitoring interval are actively adjusted in advance, so as to reduce the hysteresis of resource allocation. According to the offline task processing condition of the last monitoring interval, the offline task submission strategy in the current stage is adjusted. The method disclosed by the application accelerates the offline task processing and improves the cluster resource utilization.

Description

Technical Field

[0001] This invention belongs to the fields of cloud computing applications, elastic scaling technology and computer technology, and mainly relates to a hybrid deployment method for online and offline tasks for bursty loads. Background Technology

[0002] In recent years, internet companies have developed rapidly, with their related businesses growing in scale and becoming increasingly diversified. These businesses encompass online services such as payment, search, and recommendation, as well as offline services such as machine learning and ad-hoc query services. Traditional management models typically isolate online and offline services at the resource pool level, reserving excess resources based on the maximum resource requirements of online services to handle sudden or peak loads. This approach results in significant resource waste during off-peak periods for online services. Consequently, we have observed a severe waste of resources in data center production clusters in recent years. A reasonable hybrid deployment of online and offline services can effectively reduce cluster resource waste while ensuring online service performance. Therefore, hybrid deployment of online and offline services plays a crucial role in improving cluster resource utilization.

[0003] Many researchers have proposed different methods for the hybrid deployment of online and offline services, but these methods all have varying degrees of shortcomings. Guo et al. (Guo J, Chang Z, Wang S, et al. Who limits the resource efficiency of my datacenter: An analysis of Alibaba datacenter traces; Proceedings of the International Symposium on Quality of Service. 2019: 1-10.) proposed a semi-containerized hybrid deployment method for online and offline services based on a periodic adaptive garbage collection mechanism for online services. This method reduces cluster resource waste by redesigning a scheduling framework that combines shared state and two-layer scheduling to achieve hybrid deployment and operation of cluster online and offline services. Rzadca et al. (Rzadca K, Findeisen P, Swairski J, et al. Autopilot: workload autoscaling at Google; Proceedings of the Fifteenth European Conference on Computer Systems. 2020: 1-16.) proposed a method based on reinforcement learning and dynamic prediction of resource consumption to reduce resource waste caused by excessive resource requests due to human intervention during the hybrid deployment of online and offline services. For optimizing the hybrid deployment of online and offline services, most existing methods use multi-layer scheduling, which increases the complexity of the hybrid deployment. Moreover, most of them ignore the detection and handling of sudden loads during the adjustment of business resources. When online services encounter sudden loads, serious performance problems will occur, which will also affect the stability of the cluster.

[0004] To reduce cluster resource waste caused by over-allocation of resources, elastic scaling has gradually become a standard configuration for data center production clusters. Elastic scaling is an important function of cloud infrastructure, enabling the cluster to automatically adjust the resources allocated to the services deployed in the cluster according to the dynamically changing workload of business, so as to meet the service quality requirements of business while optimizing resource costs. Summary of the Invention

[0005] This invention addresses the problem of cluster resource waste caused by excessive resource allocation in existing technologies by providing a hybrid deployment method for online and offline tasks accommodating sudden loads. The method includes a preprocessing phase and an update phase. In the preprocessing phase, an online / offline hybrid deployment control system is built in the Kubernetes cluster to dynamically adjust the online and offline service Pod instances on the server. In the update phase, at each monitoring interval, the workload prediction model is updated using the historical workload of the online services over the last K monitoring intervals through the online / offline hybrid deployment control system. The workload prediction model, resource matching model, and burst detection method are used to quickly predict the number of online service Pods required for the next monitoring interval. Based on the prediction results and the remaining cluster resources, the available online / offline service Pod instances in the cluster for the next monitoring interval are proactively adjusted in advance to reduce resource allocation lag. Furthermore, the offline task submission strategy for the current stage is adjusted based on the offline task processing status of the previous monitoring interval. The method of this invention comprehensively considers the hybrid deployment of offline and online services, the detection of sudden loads in online services, and the utilization of cluster resources. While ensuring the performance of online services, it avoids the over-allocation and under-allocation of online service resources, and enables offline services to make full use of the idle resources of the Kubernetes cluster, accelerate offline task processing, and improve the utilization of cluster resources. It has broad application value and prospects in the field of cloud computing.

[0006] To achieve the above objectives, the technical solution adopted by this invention is: a hybrid deployment method for online and offline tasks oriented to handle bursty loads, comprising a preprocessing phase and an update phase:

[0007] A. Preprocessing stage: Stress test the online services deployed in the cluster, collect online service performance index data, form an initial scaling dataset for online services, filter the initial scaling data, use the filtered initial scaling data to train the online service workload prediction model and resource prediction model, and build an online and offline hybrid control system in the cluster based on the trained prediction model, the burst detection method based on discreteness analysis and the cluster resource planning method.

[0008] B. Update Phase: In each monitoring interval, the number of Pods required for online services in the next monitoring interval is predicted by the offline mixed deployment control system; based on the remaining resources of the cluster, the Pod instances for online and offline services in the next monitoring interval are adjusted; finally, based on the offline task processing status of the previous phase and the currently determined number of available Pod instances for offline services in the next monitoring interval, the offline tasks that can be processed in the next monitoring interval are calculated and submitted.

[0009] As an improvement of the present invention, the preprocessing stage specifically includes the following steps:

[0010] A1. Use JMeter to perform stress tests on online services deployed in the cluster, and use Prometheus to record the performance metrics of online services in the Kubernetes cluster. The online service performance metrics include at least workload and 99th percentile latency, forming an initial scaling dataset for online services.

[0011] A2. Filter the initial dataset in step A1. Use a simple linear regression model and a random forest resource model to model the unfiltered initial scaling dataset and the filtered initial scaling dataset respectively to form a workload prediction model and a resource prediction model; the online business workload is the number of requests arriving within a monitoring interval.

[0012] A3. Construct an online service burst detection method using a dispersion analysis-based approach;

[0013] A4. Construct a cluster resource planning method using heuristic search and a minimum-priority stopping strategy;

[0014] A5. Based on the workload prediction model and resource prediction model obtained in step A2, the burst detection method constructed in step A3, and the cluster resource planning method constructed in step A4, an offline mixed deployment control system is built in the Kubernetes cluster.

[0015] As another improvement of the present invention, the updating stage specifically includes the following steps:

[0016] B1. In each monitoring interval, update a sliding window of size k, and using the online and offline mixed control system, update the workload prediction model based on the historical workload of online services within the sliding window, and predict the workload of online services in the next monitoring interval.

[0017] B2. Using the workload predicted in step B1 and the pre-set online service response time constraints as inputs to the resource prediction model, predict the number of online service Pods required in the next stage;

[0018] B3. Combine the predicted online service Pod quantity requirements in step B2 with the historical online service Pod quantity requirements in different monitoring intervals with the most recent length of 1 to k to perform dispersion analysis to detect sudden load. Then, based on the sudden load detection results, adjust the number of online service Pods pre-allocated in the next monitoring interval.

[0019] B4. Based on the remaining resources of the cluster, determine the number of available Pod instances for online and offline services in the next monitoring interval, and adjust the number of Pod instances for online and offline services in the cluster.

[0020] B5. Based on the offline task processing status of the previous stage and the number of available Pod instances for offline services in the next monitoring interval, calculate and submit the offline tasks that can be processed in the next monitoring interval.

[0021] As another improvement of the present invention, the adjustment of the offline business Pod instance in the next monitoring interval cluster specifically includes the following steps:

[0022] S1. Update the workload prediction model using the online business historical workload records within the most recent k monitoring intervals;

[0023] S2. Using the online service workload of the previous monitoring interval as input to the updated workload prediction model, predict the online service workload r of the next monitoring interval. t+1 ;

[0024] S3. Use the predicted online service workload for the next monitoring interval and the pre-set online service latency constraints as inputs to the online service resource prediction model to predict the number of online service Pods required for the next monitoring interval.

[0025] S4. Use the dispersion analysis method based on the online service Pod quantity demand to calculate the different standard deviations between the predicted online service Pod quantity demand for the next monitoring interval and the historical online service Pod quantity demand within the most recent monitoring intervals of length 1 to k to detect whether there is a sudden load in the online service, and output the adjusted online service Pod quantity demand for the next monitoring interval.

[0026] S5. When cluster resources are sufficient, calculate the available idle resources of offline services under the premise of meeting the resource requirements of online service Pods, and adjust the next monitoring interval to the offline service Pod instance. When cluster resources are scarce, continuously stop the offline service Pod instances running in the cluster until the resource requirements of online service Pods are met or there are no offline service Pod instances to stop, and adjust the next monitoring interval of online service Pod instances to the online service Pod instance.

[0027] S6. After the adjustment of online and offline service Pod instances in the cluster is completed for the next monitoring interval, calculate and submit the offline tasks that can be handled in the next monitoring interval, and the method ends.

[0028] Compared with existing technologies, this invention has the following advantages: It proactively adjusts resources for hybrid online and offline services deployed within a Kubernetes cluster through predictive elastic scaling. In the online service Pod demand prediction phase, a simple linear regression model is used to quickly and accurately predict future workloads using limited historical workload data. A random forest resource prediction model improves the accuracy of online service Pod demand prediction. A burst detection method based on dispersion analysis is used to appropriately detect burst loads in online services, adjusting future online service Pod demands to reduce the impact of burst loads. Finally, based on the remaining cluster resources and the priority constraints of online and offline services, the number of online and offline Pod instances in the Kubernetes cluster is dynamically adjusted. Simultaneously, tasks that can be processed by offline services are calculated and submitted. This ensures that the performance of online services is not affected while allowing offline services to fully utilize idle cluster resources, accelerating task processing, achieving resource complementarity between online and offline services, and improving cluster resource utilization. Attached Figure Description

[0029] Figure 1 This is a schematic diagram of the hybrid deployment structure for offline services according to the present invention;

[0030] Figure 2 This is a schematic diagram of the Kubernetes cluster offline service mixed deployment control system of the present invention;

[0031] Figure 3 This is a flowchart illustrating the steps of the present invention to achieve hybrid deployment of online and offline services for bursty loads. Detailed Implementation

[0032] The present invention will be further illustrated below with reference to the accompanying drawings and specific embodiments. It should be understood that the following specific embodiments are for illustrative purposes only and are not intended to limit the scope of the invention.

[0033] Example 1

[0034] A hybrid deployment method for online and offline tasks to handle sudden load surges is proposed. This method utilizes a built-in hybrid deployment control system to dynamically adjust the online and offline service Pod instances deployed in the cluster, optimizing the hybrid deployment of online and offline services. This ensures the performance of online services while enabling offline services to fully utilize idle cluster resources, accelerating task processing and improving cluster resource utilization. The method includes the following stages:

[0035] A. Preprocessing stage: Deploy the on- and off-line hybrid deployment control system in the Kubernetes cluster to dynamically adjust the on- and off-line business Pod instances deployed in the cluster.

[0036] Stress testing was conducted on online services deployed in the Kubernetes cluster using the JMeter stress testing tool. Performance metrics data for these online services, including workload and 99th percentile latency, were collected by the Prometheus monitoring system. Kubernetes HPA was used for elastic scaling of the online services. Initial scaling data, including workload, 99th percentile latency, and the required number of online service Pods, was filtered. The filtered initial scaling data was used to train online service workload and resource prediction models. The online service workload was defined as the number of requests arriving within a monitoring interval. Based on the initially trained prediction models, a burst detection method based on discrete analysis, and a cluster resource planning method, an online / offline hybrid deployment control system was built in the Kubernetes cluster. During offline service operation, the number of offline service Pod instances was dynamically adjusted according to the online service Pod demand prediction results and cluster resource planning to adapt to changes in online service workload and improve cluster resource utilization.

[0037] The specific steps of the preprocessing stage are as follows:

[0038] A1. Use JMeter to perform stress tests on the online services deployed in the cluster, and use Prometheus to record the CPU utilization, workload, 99th percentile latency, and number of Pods of the online services in the Kubernetes cluster, forming an initial scaling dataset for the online services.

[0039] A2. Filter the initial dataset from step A1. Model the unfiltered initial scaling dataset and the filtered initial scaling dataset using a simple linear regression model and a random forest model respectively, forming a workload prediction model and a resource prediction model. The filtering rules are as follows:

[0040]

[0041] The filterate function outputs true, indicating that the data in the initial scaling dataset is filtered out; the filterate function outputs false, indicating that the data in the initial scaling dataset is retained. 99th responsetime represents the 99th percentile delay, and responseTimeSLO represents the pre-set online business delay constraint.

[0042] The formula for the simple linear regression workload prediction model is as follows:

[0043] r t+1 =c t *r t +et

[0044]

[0045] Where, r t+1 c represents the predicted workload for the (t+1)th monitoring interval. t Let r represent the model regression coefficients for the t-th monitoring interval. t e represents the actual workload in the t-th monitoring interval. t Let represent the model error of the t-th monitoring interval, and k represent the k most recent monitoring intervals.

[0046] The formula for the random forest resource prediction model is as follows:

[0047]

[0048] Pod t+1 r represents the predicted number of online service Pods required for the next monitoring interval. t+1 This represents the predicted online workload for the next monitoring interval, responseTimeSLO represents the response time constraint, and N represents the number of trees in the random forest.

[0049] A3. Construct an online business burst detection method using a method based on discreteness analysis.

[0050] A4. Construct a cluster resource planning method using heuristic search and a minimum priority stopping strategy.

[0051] A5. Based on the initial workload prediction model and resource prediction model, as well as the established burst detection method and cluster resource planning method, an offline hybrid control system is built on the Kubernetes cluster.

[0052] B. Update Phase: During each heartbeat period, the task sorting is updated based on the task execution status, and the cluster environment is updated in real time based on the server resource utilization and the fuzzy logic control system.

[0053] During each heartbeat period, the task ranking is updated in real time based on the current execution status of the tasks: such as the number of completed tasks, the number of tasks currently being processed, the number of tasks yet to be assigned, and the current job execution speed. With a fuzzy control system configured on the server, the number of work slots on each server is dynamically adjusted based on its current CPU utilization, memory utilization, and network bandwidth utilization. By changing the number of work slots, server utilization is improved, all jobs are completed as early as possible, and the energy consumption of the entire cluster is reduced.

[0054] The update phase includes the following steps:

[0055] B1. In each monitoring interval, update a sliding window of size k, utilize the online / offline mixed control system, update the workload prediction model based on the historical workload of online services within the sliding window, and predict the workload of online services in the next monitoring interval.

[0056] B2. Using the workload predicted in step B1 and the pre-set online service response time constraints as inputs to the resource prediction model, predict the number of online service Pods required in the next stage.

[0057] B3. Combining the predicted online service Pod quantity requirement from step B2 with the historical online service Pod quantity requirement within the most recent set of different monitoring intervals with lengths from 1 to K, perform dispersion analysis to detect burst loads. Then, based on the burst detection results, adjust the number of online service Pods pre-allocated for the next monitoring interval. The standard deviation calculation formula based on dispersion analysis is shown below:

[0058]

[0059] Where n is the size of the selected monitoring interval set, k represents the maximum length of the selected monitoring interval set, and vi represents the standard deviation calculated by performing dispersion analysis on the predicted online service Pod quantity demand and the historical online service Pod quantity within the most recent monitoring interval set of length n. i This represents the required number of pods for the i-th online business. avg This represents the average value calculated from the predicted demand for online service Pods and the historical number of online service Pods within the most recent monitoring interval set of length n.

[0060] B4. Based on the remaining resources of the cluster, determine the number of available Pod instances for online and offline services in the next monitoring interval, and adjust the number of Pod instances for online and offline services.

[0061] B5. Based on the offline task processing status of the previous stage and the number of available Pod instances for offline services in the next monitoring interval, calculate and submit the offline tasks that can be processed in the next monitoring interval. The calculation rules for the tasks that can be processed in the next monitoring interval are as follows:

[0062] nextTasks(beforeTasks, offPods)=offPods*offCapcity-beforeTasks

[0063] Where nextTasks represents the number of offline tasks that can be processed in the next monitoring interval, beforeTasks represents the number of offline tasks that have not been completed in the current monitoring interval, offPods represents the number of offline business Pod instances in the next monitoring interval, and offCapcity represents the number of offline tasks that a single offline business Pod instance can process within a monitoring interval.

[0064] The next monitoring interval cluster adjustment for offline business Pod instances includes the following steps:

[0065] S1. Update the workload prediction model using the online business historical workload records within the most recent k monitoring intervals.

[0066] S2. Using the online service workload of the previous monitoring interval as input to the updated workload prediction model, predict the online service workload r of the next monitoring interval. t+1 .

[0067] S3. Using the predicted online service workload for the next monitoring interval and the pre-set online service latency constraints as inputs to the online service resource prediction model, the required number of online service Pods for the next monitoring interval is predicted.

[0068] S4. Use the dispersion analysis method based on the online service Pod quantity demand to calculate the different standard deviations between the predicted online service Pod quantity demand for the next monitoring interval and the historical online service Pod quantity demand within the most recent monitoring intervals of length 1 to k to detect whether there is a sudden load in the online service, and output the adjusted online service Pod quantity demand for the next monitoring interval.

[0069] The online service burst detection method in step S4 includes the following steps:

[0070] S401. Calculate the standard deviation of the set consisting of the predicted online service Pod quantity demand and the online service Pod quantity demand in different monitoring intervals of length 1 to k using the dispersion analysis method, and record the current standard deviation.

[0071] S402. If the current standard deviation is greater than or equal to the maximum standard deviation of the current record during the loop, then jump to S403. Otherwise, continue to the next loop.

[0072] S403. Update the maximum standard deviation to the current standard deviation, and adjust the maximum number of online service Pods requirement to the maximum historical number of online service Pods requirement within the monitoring interval set corresponding to the current cycle length.

[0073] S404. Loop ends. If the current maximum standard deviation is greater than or equal to the pre-set online business burst threshold, then a burst load is considered to exist, and proceed to S405. Otherwise, proceed to S406.

[0074] S405. Set the current online service status to a burst state, and adjust the online service Pod quantity requirement for the next monitoring interval to the maximum value between the maximum online service Pod quantity requirement recorded during the loop and the predicted online Pod quantity requirement for the next monitoring interval.

[0075] S406. If the current online service status is a burst state and the predicted number of online service Pods required for the next monitoring interval is less than the current number of online service Pod instances, then proceed to S07. Otherwise, proceed to S408.

[0076] S407. Adjust the online service status to non-emergency status, and adjust the online service Pod quantity requirement for the next monitoring interval to the current online service Pod instance quantity.

[0077] S408. Maintain the online service Pod count requirement for the next monitoring interval as the predicted online service Pod count requirement.

[0078] S409. Output the required number of online service Pods for the next monitoring interval. The method ends here.

[0079] S5. When cluster resources are sufficient, calculate the available idle resources for offline services while meeting the resource requirements of online service Pods, and adjust the next monitoring interval for offline service Pod instances. When cluster resources are scarce, continuously stop offline service Pod instances running in the cluster until the resource requirements of online service Pods are met or there are no offline service Pod instances to stop, and then adjust the next monitoring interval for online service Pod instances.

[0080] The cluster resource planning method in step S5 includes the following steps:

[0081] S501. Sort the remaining resources of each node in the cluster in ascending order of remaining CPU resources, remaining memory resources, and remaining storage resources.

[0082] S502. If the predicted number of online service Pods required for the next monitoring interval is less than or equal to the number of online service Pod instances currently running in the cluster, proceed to S503. Otherwise, proceed to S504.

[0083] S503. Starting from the sorted first node of the cluster, stop the online business Pod instances running on each node one by one until the number of online business Pods required for the next monitoring interval is met. Then, reorder each node of the cluster according to the set rules, and start from the reordered first node of the cluster to gradually deploy offline business Pod instances on the nodes until each node can no longer deploy more offline business instances.

[0084] S504. Starting from the sorted first node of the cluster, stop the offline business Pod instances running on each node one by one until the online business Pod quantity requirement for the next monitoring interval is met or each node has no more offline business Pod instances to stop. Then, reorder each node of the cluster according to the set rules. Based on the predicted online business Pod quantity requirement for the next monitoring interval, start from the reordered first node of the cluster and gradually deploy online business Pod instances on the nodes until each node cannot deploy more online business instances or meets the predicted online business Pod quantity requirement.

[0085] S505. The adjustment of hybrid deployment of online and offline services in the Kubernetes cluster is complete. This concludes the method.

[0086] S6. After the adjustment of online and offline service Pod instances in the cluster is completed for the next monitoring interval, calculate and submit the offline tasks that can be handled in the next monitoring interval, and the method ends.

[0087] This invention provides a predictive elastic scaling method for proactively adjusting the online and offline service Pod instances in a hybrid cluster deployment. Specifically, it includes the following three steps: 1) Collecting initial scaling data based on stress testing and Kubernetes reactive elastic scaling. This initial scaling data includes workload, 99th percentile latency, and online service Pod requirements. Utilizing the initially trained online service workload prediction model, resource prediction model, and burst detection and cluster resource planning methods, it then performs the following: 2) Updating the workload prediction model based on the workload of the last k monitoring intervals. Predicting the online service Pod quantity requirement for the next monitoring interval using the workload prediction model, resource prediction model, and burst detection method, and outputting the result; 3) Adjusting the online and offline service Pod instances in the cluster based on the predicted online service Pod quantity requirement for the next monitoring interval and the cluster resource planning function. This maximizes the fulfillment of online service Pod resource requirements while allowing offline services to fully utilize idle cluster resources, accelerating offline task processing and improving cluster resource utilization. This method has broad application value and promising prospects in the cloud computing field.

[0088] like Figure 1The diagram illustrates a specific example of the present invention, including a Kubernetes cluster 1, cluster nodes 2, online service Pod instances 3, offline service Pod instances 4, a mixed online / offline service deployment control system 9, online service workloads 10, and an offline service task pool 11. First, based on the collected workload data from the k most recent monitoring intervals, the mixed online / offline service deployment control system predicts the required number of online service Pods for the next monitoring interval and adjusts the number of online and offline service Pod instances in the cluster based on the prediction results. The cluster contains multiple nodes, and each node can simultaneously deploy multiple online and offline service Pod instances. A schematic diagram of the mixed online / offline deployment control system for the Kubernetes cluster built according to the present invention is shown below. Figure 2 As shown, it includes online business workload prediction 5, online business resource demand prediction 6, burst detection 7, and cluster resource planning 8.

[0089] Assuming a monitoring interval of 1 minute and the number of recent monitoring intervals (k) is set to 5, the set of online business workloads collected from the last 5 monitoring intervals is R = {R1, R2, R3, R4, R5}. The offline task pool set is T = {T1, T2, T3, T4, T5, T6, T7, T8, T9, T10}. The Kubernetes cluster node set is S = {S1, S2}. Each node in the cluster can deploy 3 online business Pod instances and 6 offline business Pod instances. Node S1 has already deployed one online business Pod instance and one offline business Pod instance. The completed offline tasks are T1 and T2.

[0090] Figure 3 This is a flowchart illustrating the hybrid deployment of online and offline services to handle sudden load surges, as described in this embodiment of the invention. Figure 3 As shown, the steps for adjusting the mixed deployment of online and offline services are as follows:

[0091] S1. Update the online business workload prediction model based on the online business workload set R, predict the online business workload for the next monitoring interval, update the prediction model, and obtain the regression coefficient c5 and error e5.

[0092] S2. Use the online service workload R5 of the previous monitoring interval as the input of the online service workload prediction model to predict the online service workload R6 of the next monitoring interval = c5*R5+e5.

[0093] S3. Using the predicted online service workload R6 and latency constraint responseTimeSLO for the next monitoring interval as input to the online service resource prediction model, the required number of online service Pods (Pod6) for the next monitoring interval is predicted.

[0094] S4. Based on the online service Pod quantity requirement set PodList = {PodIns1, PodIns2, PodIns3, PodIns4, PodIns5} from the last 5 monitoring intervals and the predicted online service Pod quantity requirement Pod6 for the next monitoring interval, use the dispersion analysis method to iteratively calculate Pod6 and PodList subsets {PodIns5}, {PodIns4, PodIns5}, {PodIns3, PodIns4, PodIns5}, {PodIns3, PodIns4, P The standard deviations v1, v2, v3, v4, and v5 among the elements in {podIns5}, {PodIns2, PodIns3, PodIns4, PodIns5}, and {PodIns1, PodIns2, PodIns3, PodIns4, PodIns5} are checked sequentially. It is found that the standard deviations v1 and v5 are greater than or equal to the pre-set online service burst threshold, and v5 > v1. Therefore, the maximum number of online service Pods required is adjusted to the maximum historical number of online service Pods required within the most recent monitoring interval set of length 5, which is 2. Finally, since v5 ≥ the pre-set online service burst threshold, and the maximum number of online service Pods required is greater than the predicted number of online service Pods required in the next monitoring interval, the number of online service Pods required in the next monitoring interval is adjusted to the maximum number of online service Pods required.

[0095] S5. Output the number of Pods required for the online services in the next monitoring interval.

[0096] S6. The next monitoring interval requires the deployment of two online service Pod instances. Since one online service Pod instance has already been deployed on the current node S1, only one more online service Pod instance needs to be added. Furthermore, since there are currently no online service Pod instances on S2, the new online service Pod instance will be deployed on S2 for load balancing. At this point, the cluster still has sufficient remaining resources, and S1 has already deployed one offline task Pod instance. Therefore, S1 can deploy three more offline service Pod instances, and S2 can deploy four more offline service Pod instances. The offline tasks T1 and T2 have already been completed. According to the formula, the set of offline tasks that can be submitted in the next monitoring interval is {T3, T4, T5, T6, T7, T8, T9, T10}.

[0097] Through the above process, this invention realizes a hybrid deployment method for online and offline services on a Kubernetes cluster. By predicting online service workload, predicting online service resource requirements, detecting online service bursts, and planning cluster resources, the invention proactively adjusts the Pod instances of online and offline services deployed on the Kubernetes cluster in advance. This ensures the performance of online services while enabling offline services to make full use of idle cluster resources, accelerate offline task processing, and improve cluster resource utilization.

[0098] It should be noted that the above content merely illustrates the technical concept of the present invention and should not be construed as limiting the scope of protection of the present invention. For those skilled in the art, various improvements and modifications can be made without departing from the principle of the present invention, and all such improvements and modifications fall within the scope of protection of the claims of the present invention.

Claims

1. A hybrid deployment method for online and offline tasks to handle sudden load surges, characterized in that, Includes a preprocessing stage and an update stage: A. Preprocessing stage: Stress test the online services deployed in the cluster, collect online service performance index data, form the initial scaling dataset of online services, filter the initial scaling data, use the filtered initial scaling data to train the online service workload prediction model and resource prediction model, and build the online and offline mixed deployment control system in the cluster based on the trained prediction models, the burst detection method based on discreteness analysis and the cluster resource planning method. B. Update Phase: In each monitoring interval, the number of Pods required for online services in the next monitoring interval is predicted by the offline mixed deployment control system; the Pod instances for online and offline services in the next monitoring interval are adjusted according to the remaining resources of the cluster; finally, the number of offline tasks that can be processed in the next monitoring interval is calculated and submitted based on the offline task processing status of the previous phase and the number of available Pod instances for offline services in the next monitoring interval that is currently determined. B1. In each monitoring interval, update a sliding window of size k, and using the online and offline mixed control system, update the workload prediction model based on the historical workload of online services within the sliding window, and predict the workload of online services in the next monitoring interval. B2. Using the workload predicted in step B1 and the pre-set online service response time constraints as inputs to the resource prediction model, predict the number of online service Pods required in the next stage; B3. Combine the predicted online service Pod quantity requirements in step B2 with the historical online service Pod quantity requirements in different monitoring intervals with the most recent length of 1 to k to perform dispersion analysis to detect sudden load. Then, based on the sudden load detection results, adjust the number of online service Pods pre-allocated in the next monitoring interval. B4. Based on the remaining resources of the cluster, determine the number of available Pod instances for online and offline services in the next monitoring interval, and adjust the number of Pod instances for online and offline services in the cluster. B5. Based on the offline task processing status of the previous stage and the currently determined number of available Pod instances for offline services in the next monitoring interval, calculate and submit the offline tasks that can be processed in the next monitoring interval; the specific steps for adjusting the offline service Pod instances in the cluster for the next monitoring interval include: S1. Update the workload prediction model using the online business historical workload records within the most recent k monitoring intervals; S2. Using the online service workload from the previous monitoring interval as input to the updated workload prediction model, predict the online service workload for the next monitoring interval. ; S3. Use the predicted online service workload for the next monitoring interval and the pre-set online service latency constraints as inputs to the online service resource prediction model to predict the number of online service Pods required for the next monitoring interval. S4. Use the dispersion analysis method based on the online service Pod quantity demand to calculate the different standard deviations between the predicted online service Pod quantity demand for the next monitoring interval and the historical online service Pod quantity demand within the most recent monitoring intervals of length 1 to k to detect whether there is a sudden load in the online service, and output the adjusted online service Pod quantity demand for the next monitoring interval. S5. When cluster resources are sufficient, calculate the available idle resources of offline services under the premise of meeting the resource requirements of online service Pods, and adjust the next monitoring interval to the offline service Pod instance. When cluster resources are scarce, continuously stop the offline service Pod instances running in the cluster until the resource requirements of online service Pods are met or there are no offline service Pod instances to stop, and adjust the next monitoring interval of online service Pod instances to the online service Pod instance. S6. After the adjustment of online and offline service Pod instances in the cluster is completed for the next monitoring interval, calculate and submit the offline tasks that can be handled in the next monitoring interval, and the method ends.

2. The hybrid deployment method for online and offline tasks oriented to bursty loads as described in claim 1, characterized in that: The preprocessing stage specifically includes the following steps: A1. Use JMeter to perform stress tests on online services deployed in the cluster, and use Prometheus to record the performance metrics of online services in the Kubernetes cluster. The online service performance metrics include at least workload and 99th percentile latency, forming an initial scaling dataset for online services. A2. Filter the initial dataset in step A1. Use a simple linear regression model and a random forest resource model to model the unfiltered initial scaling dataset and the filtered initial scaling dataset respectively to form a workload prediction model and a resource prediction model; the online business workload is the number of requests arriving within a monitoring interval. A3. Construct an online service burst detection method using a dispersion analysis-based approach; A4. Construct a cluster resource planning method using heuristic search and a minimum-priority stopping strategy; A5. Based on the workload prediction model and resource prediction model obtained in step A2, the burst detection method constructed in step A3, and the cluster resource planning method constructed in step A4, an offline mixed deployment control system is built in the Kubernetes cluster.

3. The hybrid deployment method for online and offline tasks oriented to bursty loads as described in claim 2, characterized in that: In step A2, the modeling formula for the simple linear regression workload prediction model is as follows: ； ； in, This represents the predicted workload for the (t+1)th monitoring interval. This represents the model regression coefficient for the t-th monitoring interval. This represents the actual workload at the t-th monitoring interval. Let represent the model error of the t-th monitoring interval, and k represent the k most recent monitoring intervals.

4. The hybrid deployment method for online and offline tasks oriented to bursty loads as described in claim 2, characterized in that: In step A2, the modeling formula for the random forest resource prediction model is as follows: ； in This indicates the predicted number of online service Pods required for the next monitoring interval. This indicates the predicted online business workload for the next monitoring interval. This represents the response time constraint, and N represents the number of trees in the random forest.

5. The hybrid deployment method for online and offline tasks oriented to bursty loads as described in claim 4, characterized in that, The online service burst detection method in step S4 specifically includes the following steps: S401. Calculate the standard deviation of the set consisting of the predicted online service Pod quantity demand and the online service Pod quantity demand in the most recent monitoring interval set with a length of 1 to k using the dispersion analysis method, and record the current standard deviation; S402. If the current standard deviation is greater than or equal to the maximum standard deviation of the current record during the loop, then jump to S403; otherwise, continue to the next loop. S403. Update the maximum standard deviation to the current standard deviation, and adjust the maximum number of online service Pods requirement to the maximum historical number of online service Pods requirement within the monitoring interval set corresponding to the current cycle length; S404. Loop ends. If the current maximum standard deviation is greater than or equal to the pre-set online business burst threshold, then a burst load is considered to exist, and proceed to S405; otherwise, proceed to S406. S405. Set the current online service status to a burst state, and adjust the online service Pod quantity requirement for the next monitoring interval to the maximum value between the maximum online service Pod quantity requirement recorded during the loop and the predicted online Pod quantity requirement for the next monitoring interval. S406. If the current online service status is a burst state and the predicted number of online service Pods required for the next monitoring interval is less than the current number of online service Pod instances, then proceed to S407; otherwise, proceed to S408. S407. Adjust the online service status to non-emergency status, and adjust the online service Pod quantity requirement for the next monitoring interval to the current online service Pod instance quantity; S408. Maintain the online service Pod count requirement for the next monitoring interval as the predicted online service Pod count requirement; S409. Output the required number of online service Pods for the next monitoring interval. The method ends here.

6. The hybrid deployment method for online and offline tasks oriented to bursty loads as described in claim 5, characterized in that: The cluster resource planning method in step S5 specifically includes the following steps: S501. Sort the remaining resources of each node in the cluster in ascending order of remaining CPU resources, remaining memory resources, and remaining storage resources. S502. If the predicted number of online service Pods required for the next monitoring interval is less than or equal to the number of online service Pod instances currently running in the cluster, proceed to S503; otherwise, proceed to S504. S503. Starting from the sorted first node of the cluster, stop the online business Pod instances running on each node one by one until the number of online business Pods required for the next monitoring interval is met. Then, reorder each node of the cluster according to the set rules, and start from the reordered first node of the cluster to gradually deploy offline business Pod instances on the nodes until each node can no longer deploy more offline business instances. S504. Starting from the first node of the sorted cluster, stop the offline business Pod instances running on each node one by one until the online business Pod quantity requirement for the next monitoring interval is met or each node has no offline business Pod instances to stop. Then, reorder each node of the cluster according to the set rules. Based on the predicted online business Pod quantity requirement for the next monitoring interval, start from the reordered first node of the cluster and gradually deploy online business Pod instances on the nodes until each node cannot deploy more online business instances or meets the predicted online business Pod quantity requirement. S505. The adjustment of hybrid deployment of online and offline services in the Kubernetes cluster is complete. This concludes the method.

Images