Hybrid scheduling system and method for application performance perception

Abstract

The invention discloses a hybrid scheduling system and method for application performance perception, and the system comprises an interference injection and monitoring module which is used for managing and regulating the whole data collection process, so as to obtain key performance indexes of applications on all nodes in a cluster; the performance model module is used for modeling and analyzing the performance of the application by using the collected monitoring data so as to predict the performance of the application under different resource configurations; the reinforcement learning scheduling module is used for automatically adjusting and optimizing a scheduling strategy according to an analysis result of the performance model module by utilizing a reinforcement learning method; the scheduling simulator module is used for simulating various scheduling scenes and application loads in a real environment and verifying and optimizing a scheduling strategy. According to the method, the optimal scheduling strategy can be automatically selected for scheduling under the hybrid scene, the performance and the resource utilization rate of the application can be improved, the QoS of the application is effectively guaranteed, and the overall operation efficiency and stability of the cluster are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of colocation scheduling, and in particular to an application performance-aware colocation scheduling system and method. Background Art

[0002] With the rapid advancement of Internet technology and the growing demand for computer hardware and software in Internet applications, cloud computing technology is gradually replacing traditional IT infrastructure. The rapid development of cloud computing technology has greatly promoted the expansion of large-scale data centers, which have become the core of modern information technology architecture. Cloud computing has also gradually become the main mode for enterprise deployment and operation and maintenance of enterprise services. With the support of cloud computing technology, enterprises have greatly improved IT operation and maintenance efficiency. However, despite the huge hardware investment, many studies have shown that the average resource utilization rate of data center servers is very low, only about 10-20%. This inefficient resource utilization not only leads to high operating costs, but also causes significant resource waste.

[0003] Since the concept of colocation was proposed, this strategy of deploying multiple applications on the same physical machine using container technology has quickly attracted widespread attention. Colocation is a strategy that allocates idle resources of online clusters to offline jobs while ensuring the performance of online jobs. This strategy aims to reduce the operating costs of data centers by optimizing the utilization of hardware resources. The colocation mode allows online and offline jobs to run on the same server cluster, and effectively improves the overall resource utilization efficiency by dynamically allocating idle resources to offline jobs. This not only reduces idle resources, but also ensures the performance requirements of online services.

[0004] With the emergence of large-scale computing applications such as artificial intelligence and scientific computing, the demand for computing resources for offline tasks continues to increase. Application performance in colocation scenarios has begun to attract attention, and ensuring good service application performance can meet all user quality of service (QoS) requirements for cloud service providers. The present invention focuses on processing in offline colocation scenarios, where online services mainly include Web hosting platforms, cloud storage and database services, and offline services mainly include 3D video rendering, deep learning training and data mining algorithms.

[0005] With the continuous development of cloud-native technology, most of the Internet uses the open source CNCF project Kubernetes open source orchestration system to implement a hybrid platform. However, as a highly versatile framework, Kubernetes' native scheduling strategy has been unable to meet the application deployment requirements in the actual production environment, exposing obvious deficiencies: ① Insufficient resource perception: Kubernetes can only perceive basic resources such as CPU, memory capacity, and hard disk space, but in a hybrid scenario, application performance is not only affected by these resources, but also by other applications and system pressures; this single resource perception capability cannot meet the complex and changing requirements of hybrid scenarios. ② Fixed scheduling strategy: Kubernetes' scheduling strategy is fixed and lacks flexibility. It cannot automatically adjust the scheduling strategy according to changes in the cluster environment; for example, when the resource usage or application load pattern in the cluster changes, Kubernetes cannot adaptively optimize the scheduling strategy to ensure a balance between resource utilization and application performance.

[0006] In order to cope with the complex scheduling requirements in the colocation scenario, the scheduling simulation platform is also used to verify and optimize the scheduling strategy. However, the current scheduling simulation platform also has some obvious defects: ① Insufficient performance simulation: Most of the existing scheduling simulation platforms focus on the simulation of resource allocation and usage, and lack accurate simulation of application performance; since application performance is affected by many factors, simple resource allocation simulation cannot accurately reflect the application behavior and performance in the actual production environment. ② Lack of support for actual application scenarios: Many scheduling simulation platforms are too idealistic in design and lack support for various complex application scenarios in the actual production environment; for example, the simulation platform may not fully consider factors such as the mutual influence between applications, the fluctuation of system load, and network delay, resulting in a large deviation between the simulation results and the actual situation. Research shows that it is not feasible to model all elements of the system with high precision. A cloud simulator can be designed to focus on specific aspects, functions, or architectures of the cloud. Therefore, for scenarios where application performance needs to be simulated, performance models can be used to simulate application performance to achieve relatively realistic simulation results.

[0007] Therefore, in order to improve the utilization of system resources while ensuring the QoS of applications, an application performance-aware scheduling strategy and a scheduling simulator platform for application performance simulation are proposed, which have good practical significance and application value. This can not only make up for the defects of existing scheduling strategies and simulation platforms, but also provide strong support for the deployment and optimization of colocation platforms in actual production environments. Summary of the invention

[0008] The purpose of the present invention is to address the deficiencies of Kubernetes scheduling strategies in the prior art in colocation scenarios and propose an application performance-aware colocation scheduling system and method. The present invention can provide a more comprehensive and efficient scheduling optimization solution, which can optimize system resource utilization while ensuring application QoS.

[0009] The objective of the present invention is achieved through the following technical solutions: In a first aspect, an embodiment of the present invention provides an application performance-aware colocation scheduling system, including:

[0010] The interference injection and monitoring module is used to manage and regulate the entire data collection process to obtain the key performance indicators of the application on each node in the cluster; the interference injection and monitoring module includes a collector and a pressure generator, the pressure generator is used to inject load on the application deployment node, and the collector is used to obtain the monitoring data of the application in real time;

[0011] The performance model module is used to model and analyze the performance of the application using the monitoring data collected by the interference injection and monitoring module, and to predict the performance of the application under different resource configurations through the constructed performance regression model;

[0012] A reinforcement learning scheduling module is used to automatically adjust and optimize the scheduling strategy based on the analysis results of the performance model module using the reinforcement learning method. The reinforcement learning scheduling module automatically selects the optimal scheduling strategy in different colocation scenarios through continuous learning and optimization. It also adjusts the scheduling parameters to adapt to different colocation scenarios.

[0013] The scheduling simulator module is used to use the simulation capabilities of the simulator to simulate various scheduling scenarios and application loads in the real environment, verify the feasibility and effectiveness of the scheduling strategy, and optimize it according to the verification results to obtain the optimal scheduling strategy.

[0014] Furthermore, the interference injection and monitoring module specifically includes:

[0015] (a1) During the initialization phase of the monitoring task, the application performance indicator collection capability is activated, including starting the collector and stress generator on the application deployment node to ensure monitoring coverage;

[0016] (a2) Injecting loads on application deployment nodes through stress generators to simulate different application loads and system stress scenarios;

[0017] (a3) During the monitoring data collection and processing phase, the key performance indicators of the application are monitored in real time through the collector on the application deployment node, and the monitoring data is transmitted to the remote collector on the indicator collection node for centralized management; after receiving the monitoring data, the remote collector performs indicator analysis to obtain the key performance indicators, and continuously monitors and evaluates the key performance indicators;

[0018] (a4) Display the indicator analysis results through visualization tools to generate a real-time monitoring dashboard that intuitively displays the current status and historical trends of key performance indicators; adjust the monitoring frequency and range by setting different monitoring thresholds.

[0019] Furthermore, the performance model module specifically includes:

[0020] (b1) The monitoring data collected by the interference injection and monitoring module enters the performance model module for preprocessing to obtain preprocessed monitoring data;

[0021] (b2) performing feature analysis on the preprocessed monitoring data to extract key features from the original monitoring data;

[0022] (b3) using the key features as known inputs and the performance of the application as the prediction results, adopting multiple regression algorithms to perform regression prediction modeling, and obtaining multiple performance regression models; wherein the regression algorithms include linear regression, decision tree regression, support vector machine regression and neural network algorithm;

[0023] (b4) training multiple performance regression models using the monitoring data after preprocessing and feature analysis, and adjusting the parameters of the performance regression models with the goal of optimizing performance during the training process;

[0024] (b5) constructing a test set through cross-validation, using the test set to evaluate the performance of each performance regression model, and selecting the top N performance regression models with the largest prediction accuracy; and optimizing the selected top N performance regression models according to the evaluation results, adjusting their parameters and structures;

[0025] (b6) According to the evaluation results, the performance regression model with the smallest prediction error and the strongest generalization ability is selected from the optimized performance regression models as the final performance regression model.

[0026] Furthermore, in the step (b1), the pretreatment specifically includes:

[0027] (b1.1) Data cleaning: remove outliers and missing values ​​in monitoring data;

[0028] (b1.2) De-noising: Use the moving average filtering method of smoothing filtering to remove noise and calculate the sliding average of data points to smooth the data;

[0029] (b1.3) Format conversion: converting monitoring data into the format required for performance model input;

[0030] (b1.4) Standardization: The monitoring data is standardized to eliminate the dimensional differences between different features.

[0031] Furthermore, in the step (b2), extracting key features from the original monitoring data specifically includes:

[0032] Statistical analysis and machine learning algorithms are used to select features from the feature analysis results; the selected features are further extracted and transformed, specifically by calculating the correlation of the selected features, sorting the selected features in descending order according to the size of the correlation, and selecting the selected features corresponding to the top K maximum correlation values ​​as the extracted key features.

[0033] Furthermore, the reinforcement learning scheduling module specifically includes:

[0034] (c1) using the prediction results of the performance regression model constructed in real time by the performance model module as the state input in reinforcement learning;

[0035] (c2) The actions in reinforcement learning are: the application to be scheduled at the current moment is scheduled to a certain node;

[0036] (c3) Design the reward in reinforcement learning. The reward function is set to the cluster resource imbalance and application performance degradation, and its expression is:

[0037] Reward t =R(State t ,Action t )=αImBalance t +βAvgPerf t

[0038] Among them, Reward t represents the reward value at time t, R(·) represents the reward function, and ImBalance t Indicates the resource imbalance at time t, AvgPerf t represents the application performance degradation rate at time t, α and β are the weight coefficients of resource imbalance and application performance degradation rate respectively;

[0039] (c4) the evaluation network outputs actions to the scheduler for scheduling decisions based on the node status and the status of the application to be scheduled output by the interference injection and monitoring module;

[0040] (c5) Every other target network parameter update cycle, the parameters of the evaluation network are copied to the target network to reduce the fluctuation of the target reward value during training.

[0041] Furthermore, the scheduling simulator module specifically includes:

[0042] (d1) Create fake nodes and fake applications to simulate the actual situation in the cluster environment. The fake nodes are simulated computing nodes used to test the performance of different scheduling strategies under various resource configurations and load conditions; the fake applications are simulated application performance data used to evaluate the impact of scheduling strategies on application performance;

[0043] (d2) Simulating the performance of fake applications based on the performance model, that is, applying the performance model to fake nodes and fake applications to simulate the performance of applications in the real environment. The simulator compares the generated fake performance data with the real performance data, evaluates the performance of the scheduling strategy of the reinforcement learning scheduling module under different conditions, and analyzes the impact of the scheduling strategy on application performance;

[0044] (d3) Based on the performance data and analysis results generated by the simulator, the scheduling strategy is further optimized. The optimization process specifically includes: evaluating the performance of the current scheduling strategy; improving the current scheduling strategy according to the strategy evaluation results; reapplying the improved scheduling strategy to the simulator for verification, evaluating its performance in the virtual environment, and optimizing the scheduling strategy through multiple iterations.

[0045] A second aspect of an embodiment of the present invention provides a scheduling method for the above-mentioned application performance-aware colocation scheduling system, comprising the following steps:

[0046] (1) The user submits an application request, including the application name, CPU request, and memory request;

[0047] (2) The scheduling simulator module receives the application's request and remotely calls the reinforcement learning scheduling module to obtain the scheduling decision;

[0048] (3) The reinforcement learning scheduling module first requests the interference injection and monitoring module to obtain the current resource status of each node. At the same time, the performance model module predicts the performance of each node and scheduling application to obtain the predicted value of the performance degradation rate of the application at each node.

[0049] (4) The evaluation network of the reinforcement learning scheduling module selects the optimal node for placing the application based on the status data of the performance model module to determine the scheduling strategy;

[0050] (5) Scheduling is performed according to the scheduling strategy. After scheduling is completed, the resource imbalance and application performance degradation are calculated by monitoring data, the reward value is calculated, and the evaluation network is updated to the target network. At this point, the scheduling process of reinforcement learning is completed.

[0051] Compared with the prior art, the present invention has the following beneficial effects:

[0052] (1) The present invention provides a comprehensive and efficient hybrid scheduling system through innovative modules, which greatly improves the reliability, performance and resource utilization of the Kubernetes cluster.

[0053] (2) The present invention innovatively proposes a dynamic interference injection and monitoring framework. The interference injection in the interference injection and monitoring modules generates a controllable stress environment by simulating different application loads and system stress scenarios, thereby testing the performance of the application under various conditions. At the same time, the monitoring module collects key performance indicators in real time, such as latency, throughput, and resource utilization, and provides high-precision monitoring data. It effectively solves the interference injection and monitoring problems of the observable system, and is conducive to the collection of a more comprehensive monitoring data set.

[0054] (3) The performance-aware reinforcement learning scheduling module proposed in the present invention is specifically designed for application performance prediction and scheduling optimization problems in hybrid scenarios. The reinforcement learning scheduling module has superior performance and self-learning capabilities for solving serialized decision-making problems. Its scheduling strategy is not fixed, and it can dynamically adjust the strategy to adapt to changes in the cluster environment. Specifically, through real-time monitoring and data collection, a detailed performance model is established to predict the performance of the application under different resource configurations, and the scheduling strategy is dynamically adjusted according to the application QoS and cluster resource balance. By optimizing the reinforcement learning algorithm design and reasonable resource allocation, the resource consumption of large-scale real scheduling experiments is effectively reduced, ensuring the accuracy and effectiveness of scheduling decisions.

[0055] (4) The present invention proposes a scheduling simulator based on a performance model, which solves the current problem of lack of application performance simulation scheduling tools for Kubernetes. The scheduling simulator module can simulate various scheduling scenarios and application loads in the real production environment through the prediction data provided by the performance model module, and verify and optimize the scheduling strategy. The simulator can not only simulate complex scenarios from a single machine to multiple nodes, but also perform performance testing and functional testing, compare with the original solution, and demonstrate the optimization effect of the new solution on various indicators. Through this simulation platform, users can fully verify the effect of the scheduling strategy before actual deployment, reduce the risk in the actual production environment, and ensure the stability and efficiency of the system.

[0056] (5) The present invention significantly improves the resource utilization and application performance guarantee capabilities in the hybrid deployment scenario; the present invention breaks through the limitations of the traditional Kubernetes scheduling strategy and realizes the comprehensive perception and prediction of application performance; the present invention can dynamically adjust the scheduling strategy by introducing the reinforcement learning method, ensure the accuracy and effectiveness of the scheduling decision, and optimize resource allocation; it greatly improves the intelligence level of the scheduling system, effectively guarantees the QoS of the application, and improves the overall operation efficiency and stability of the cluster. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 It is a structural block diagram of the application performance-aware co-location scheduling system of the present invention;

[0058] Figure 2 It is a structural block diagram of the interference injection and monitoring module of the present invention;

[0059] Figure 3 A flowchart of the application performance-aware scheduling strategy based on reinforcement learning of the present invention;

[0060] Figure 4 It is an architectural block diagram of the scheduling simulator module of the present invention. DETAILED DESCRIPTION

[0061] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the attached claims. It should be understood that the above general description and the detailed description that follows are merely exemplary and explanatory and cannot limit the present application.

[0062] The terms used in this application are for the purpose of describing specific embodiments only and are not intended to limit this application. The singular forms of "a", "said" and "the" used in this application and the appended claims are also intended to include plural forms unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0063] It should be understood that, although the term first, second, third, etc. may be used in the present application to describe various information, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the present application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of..." or "when..." or "in response to determination". Moreover, the term "include", "comprise" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process and method including a series of elements include not only those elements, but also include other elements not clearly listed, or also include elements inherent to such process and method. In the absence of more restrictions, the elements limited by the statement "including one..." do not exclude the existence of other identical elements in the process, method, article or equipment including the elements.

[0064] The present invention is described in detail below in conjunction with the accompanying drawings. In the absence of conflict, the features of the following embodiments and implementations can be combined with each other.

[0065] The application performance-aware colocation scheduling system of the present invention includes an interference injection and monitoring module, a performance model module, a reinforcement learning scheduling module and a scheduling simulator module. Figure 1 shown.

[0066] In this embodiment, the interference injection and monitoring module is used to manage and control the entire data collection process to obtain key performance indicators of applications on each node in the cluster; the interference injection and monitoring module includes a collector and a pressure generator, such as Figure 2 As shown, the stress generator is used to inject load on the application deployment node, and the collector is used to obtain the monitoring data of the application in real time.

[0067] It should be understood that the interference injection and monitoring module plays a vital role, which is responsible for the fine management and regulation of the entire data collection process. The interference injection and monitoring module penetrates the cluster environment and monitors the resource usage and application performance data on each node in real time. By accurately capturing the usage of key resources such as CPU, memory, and network, the interference injection and monitoring module provides comprehensive and accurate basic data support for scheduling strategies. This in-depth monitoring provides unprecedented transparency for cluster administrators, making the analysis of resource utilization and application performance more accurate and comprehensive.

[0068] It should be noted that the embodiment of the present invention proposes a framework for collecting application performance indicators in a colocation scenario, which allows users to flexibly configure according to actual needs to ensure the efficiency and accuracy of data collection.

[0069] Furthermore, if Figure 2 As shown, the interference injection and monitoring module specifically includes:

[0070] (a1) During the monitoring task initialization phase, the application performance indicator collection capability is activated, including starting the collector and stress generator on the application deployment node to ensure monitoring coverage.

[0071] (a2) Inject load on application deployment nodes through the stress generator to simulate different application load and system stress scenarios.

[0072] It should be noted that the stress generator can generate controllable stress. The generated controllable stress can be used to test the performance of the application under different load conditions, helping to identify potential performance bottlenecks and optimization opportunities.

[0073] (a3) During the monitoring data collection and processing stage, the key performance indicators of the application are monitored in real time through the collector on the application deployment node, and the monitoring data is transmitted to the remote collector on the indicator collection node for centralized management; after receiving the monitoring data, the remote collector performs indicator analysis to obtain key performance indicators such as latency, throughput, resource utilization, etc., and continuously monitors and evaluates the key performance indicators.

[0074] It should be noted that key performance indicators are some performance indicator parameters required for scheduling, which can be obtained according to actual needs, including but not limited to the usage of key resources such as CPU, memory, network, as well as latency, throughput, etc.

[0075] (a4) Display the indicator analysis results through visualization tools (such as Grafana) to generate a real-time monitoring dashboard that intuitively displays the current status and historical trends of key performance indicators; users can set different monitoring thresholds according to specific needs, adjust the monitoring frequency and range, and ensure the adaptability and scalability of the monitoring system.

[0076] It should be understood that through these detailed operation steps, the interference injection and monitoring module not only improves the efficiency and accuracy of data collection, but also ensures the scalability and maintainability of the system through its modular architecture. In summary, the interference injection and monitoring module provides strong support for application performance optimization in colocation scenarios.

[0077] In this embodiment, the performance model module is used to model and analyze the performance of the application using the monitoring data collected by the interference injection and monitoring module, and to predict the performance of the application under different resource configurations through the constructed performance regression model.

[0078] It should be noted that the performance model module plays a vital role in the present invention. It is responsible for in-depth analysis and processing of the monitoring data collected by the interference injection and monitoring modules, modeling and analyzing the performance of the application, and predicting the performance of the application under different resource configurations through a detailed performance regression model, providing a scientific basis for subsequent scheduling strategy optimization. The performance model module uses machine learning methods and monitoring data collected by the interference injection and monitoring modules to model and predict the performance of the application, expand the native scheduling strategy, and verify its feasibility in Kubernetes scheduling; this includes monitoring and analysis of key performance indicators, such as latency, throughput, and resource utilization, so as to ensure that the scheduling strategy can adapt to changes in the cluster environment and optimize resource utilization.

[0079] Furthermore, the performance model module specifically includes:

[0080] (b1) The monitoring data collected by the interference injection and monitoring module enters the performance model module for preprocessing to obtain preprocessed monitoring data.

[0081] Furthermore, the preprocessing specifically includes:

[0082] (b1.1) Data cleaning: Remove outliers and missing values ​​from monitoring data to ensure the accuracy and consistency of monitoring data.

[0083] (b1.2) De-noising: Use the moving average filtering method of smoothing filtering to remove noise, and calculate the sliding average of data points to smooth the data, which helps to remove short-term fluctuations.

[0084] (b1.3) Format conversion: Converting monitoring data into the format required by the performance model input can ensure the availability and compatibility of the monitoring data.

[0085] (b1.4) Standardization: Standardizing the monitoring data to eliminate the dimensional differences between different features helps improve the training effect of the performance model.

[0086] Specifically, for an application, the monitoring data collected by the interference injection and monitoring module will first enter the performance model module for preprocessing, which is a key step to ensure the accuracy of model training and prediction, including data cleaning, denoising, format conversion and standardization.

[0087] (b2) Perform feature analysis on the preprocessed monitoring data to extract key features that have a greater impact on the prediction results from the original monitoring data.

[0088] Furthermore, key features that have a greater impact on the prediction results are extracted from the original monitoring data, specifically including: using statistical analysis and machine learning algorithms to perform feature selection on the feature analysis results, and selecting features that have an important impact on the application performance prediction; further extracting and transforming the selected features, specifically by calculating the correlation of the selected features, sorting the selected features in descending order according to the size of the correlation, selecting the selected features corresponding to the top K maximum correlation values ​​as the extracted key features, and using the key features for subsequent analysis, which can effectively enhance the representativeness and prediction ability of the features.

[0089] (b3) Using key features as known inputs and application performance as prediction results, multiple regression algorithms are used to perform regression prediction modeling to obtain multiple performance regression models. Among them, regression algorithms include linear regression, decision tree regression, support vector machine regression, neural network and other algorithms.

[0090] (b4) Using the monitoring data after preprocessing and feature analysis to train multiple performance regression models, during the training process, the parameters of the performance regression models are adjusted with the goal of optimizing performance.

[0091] (b5) A test set is constructed through cross-validation, and the performance of each performance regression model is evaluated using the test set. The top N performance regression models with the highest prediction accuracy are selected; and based on the evaluation results, the selected top N performance regression models are optimized and their parameters and structures are adjusted to further improve the prediction accuracy.

[0092] It should be understood that by constructing a test set through cross-validation, such as five-fold cross-validation, and then using the test set to test the performance regression model, the generalization ability of the performance regression model can be ensured.

[0093] (b6) According to the evaluation results, the performance regression model with the smallest prediction error and the strongest generalization ability is selected from the optimized performance regression models as the final performance regression model.

[0094] It should be understood that through the above detailed operation steps, the performance model module not only improves the accuracy and reliability of application performance prediction, but also provides strong support for resource scheduling optimization in colocation scenarios. Overall, the performance model module is of great significance in ensuring application QoS and system resource utilization.

[0095] In this embodiment, the reinforcement learning scheduling module is used to automatically adjust and optimize the scheduling strategy according to the analysis results of the performance model module using the reinforcement learning method; the reinforcement learning scheduling module automatically selects the optimal scheduling strategy in different colocation scenarios through continuous learning and optimization, which can effectively improve system resource utilization and application performance; users can adjust scheduling parameters according to specific needs, such as the type and frequency of scheduling strategies, to be suitable for different colocation scenarios, so that the present invention is not only suitable for a wide range of colocation environments, but also can meet the specific needs of specific users, and is conducive to flexible customization of scheduling strategies.

[0096] It should be understood that the reinforcement learning scheduling module is one of the core innovations of the present invention. By deeply studying the existing cloud computing job scheduling strategies and identifying their limitations in the colocation scenario, this in-depth analysis provides unprecedented transparency for the improvement of scheduling strategies, making the optimization of scheduling strategies more accurate and comprehensive.

[0097] Furthermore, based on the effectiveness of reinforcement learning in sequential decision-making problems, the present invention proposes an application performance-aware scheduling strategy based on reinforcement learning, such as Figure 3 This strategy innovatively combines the performance model with the reinforcement learning scheduling module and uses the prediction results of the performance model as part of the state, thereby enhancing the scheduling strategy's perception and prediction capabilities of application performance. Figure 3 As shown in Figure 1, the reinforcement learning scheduling module specifically includes:

[0098] (c1) The prediction results of the performance regression model constructed in real time by the performance model module are used as the state input in reinforcement learning.

[0099] Specifically, the state s includes: the resource inventory matrix state of each node (CPU, memory, network bandwidth and disk bandwidth) and the resource request matrix (CPU, memory, network bandwidth and disk bandwidth) of the current scheduled application, and its formula is expressed as:

[0100]

[0101] Among them, State t represents the state at time t, Indicates the resource inventory of node n at time t, where n is the total number of nodes at time t, and Pod t A matrix representing the applications (Pods) to be scheduled at time t.

[0102] It should be noted that the performance model module establishes a detailed application performance regression model through real-time monitoring and data collection, and predicts the performance of the application under different resource configurations. These prediction results serve as the state input of the reinforcement learning method, providing more accurate and comprehensive environmental information, so that the reinforcement learning method can make more optimized scheduling decisions.

[0103] (c2) The action in reinforcement learning is: the application to be scheduled at the current moment is scheduled to a certain node, expressed as:

[0104] Action t ∈{Node i |i=1,2,…,n}

[0105] Among them, Action t Represents the action at time t.

[0106] (c3) Design the reward in reinforcement learning. The reward function is set to the cluster resource imbalance and application performance degradation, and its expression is:

[0107] Reward t =R(State t ,Action t )=αImBalance t +βAvgPerf t

[0108] Among them, Reward t represents the reward value at time t, R(·) represents the reward function, and ImBalance t Indicates the resource imbalance at time t, AvgPerf t It represents the application performance degradation rate at time t, α and β are the weight coefficients of resource imbalance and application performance degradation rate respectively. Resource imbalance ImBalance is the standard deviation of all node resources. The more unbalanced the resources are, the greater the ImBalance is. AvgPerf is the degradation rate of the performance of all applications compared to the optimal performance.

[0109] It should be noted that the cluster resource balance is obtained by measuring the distribution of resources in the cluster. The reward function encourages the scheduling strategy to achieve a more balanced resource allocation, thereby improving resource utilization. The application performance degradation is obtained by measuring the degree of degradation of application performance. The reward function encourages the scheduling strategy to minimize performance degradation, thereby ensuring the QoS of the application.

[0110] (c4) The evaluation network outputs actions to the scheduler to make scheduling decisions based on the node status and the status of the application to be scheduled output by the interference injection and monitoring module.

[0111] It should be noted that the optimization goal of the reinforcement learning scheduling module is to improve resource balance and utilization while ensuring application QoS; the performance model module inputs the application name and selects the scheduling node status, and outputs the predicted performance degradation rate; the interference injection and monitoring module will output the status of all nodes in real time, and the node status can be obtained through the interference injection and monitoring module.

[0112] (c5) The parameters of the evaluation network are copied to the target network every other target network parameter update cycle to reduce the fluctuation of the target reward value during training. This can solve the strong correlation problem in the deep Q network reinforcement learning (DQN) algorithm and help improve the stability of the reinforcement learning method.

[0113] In this embodiment, the scheduling simulator module is used to utilize the simulation capability of the simulator to simulate various scheduling scenarios and application loads in a real environment, verify the feasibility and effectiveness of the scheduling strategy, and optimize it according to the verification results to obtain the optimal scheduling strategy.

[0114] It should be understood that in the field of cluster scheduling, the use of simulators for verification has become a key evaluation tool due to its advantages of being fast and low-cost. The present invention proposes an innovative scheduling simulator architecture, the architecture of which is shown in FIG. Figure 4 As shown in Figure 1, the scheduling simulator module verifies and optimizes the scheduling strategy through the simulation capability of large-scale scenarios. This simulation mechanism is crucial to ensure the stability and efficiency of the scheduling strategy in the actual production environment. The key contents of the scheduling simulator are described in detail below.

[0115] Furthermore, the scheduling simulator module specifically includes:

[0116] (d1) Create fake nodes and fake applications to simulate the actual situation in the cluster environment. The fake nodes are simulated computing nodes used to test the performance of different scheduling strategies under various resource configurations and load conditions; the fake applications are simulated application performance data used to evaluate the impact of scheduling strategies on application performance.

[0117] In this process, the collection of control flow events mainly focuses on the life cycle management of the device, including events such as device initialization, connection establishment, data transmission, and connection termination. By simulating the life cycle management of the device in detail, the simulator can capture key control flow events and provide accurate reference data for the optimization of scheduling strategies.

[0118] (d2) Simulate the performance of fake applications based on the performance model, that is, apply the performance model to fake nodes and fake applications to simulate the performance of applications in the real environment. The simulator compares the generated fake performance data with the real performance data, evaluates the performance of the scheduling strategy of the reinforcement learning scheduling module under different conditions, and analyzes the impact of the scheduling strategy on application performance, specifically including evaluating the key performance indicators such as resource utilization, application latency and throughput of the scheduling strategy under different load conditions.

[0119] (d3) Based on the performance data and analysis results generated by the simulator, further optimize the scheduling strategy. The optimization process specifically includes: evaluating the performance of the current scheduling strategy and identifying its shortcomings in resource utilization and application performance; improving the current scheduling strategy based on the strategy evaluation results and proposing improvement suggestions. For example, you can adjust the resource allocation strategy, add a load balancing mechanism, or adopt a more intelligent scheduling algorithm; reapply the improved scheduling strategy to the simulator for verification and evaluate its performance in the virtual environment. By optimizing the scheduling strategy through multiple iterations, the effectiveness and stability of the scheduling strategy can be gradually improved.

[0120] In summary, the scheduling simulator module of the present invention provides an efficient and accurate scheduling evaluation tool by creating fake nodes and fake applications, simulating application performance based on performance models, optimizing scheduling strategies, and conducting experimental verification, thereby significantly improving the development and optimization efficiency of scheduling strategies.

[0121] It is worth mentioning that an embodiment of the present invention further provides a scheduling method, which is implemented based on the application performance-aware co-location scheduling system in the above embodiment.

[0122] The scheduling method of the present invention specifically comprises the following steps:

[0123] (1) The user submits an application request, including the application name, CPU request, and memory request.

[0124] (2) The scheduling simulator module receives the application's request and remotely calls the reinforcement learning scheduling module to obtain the scheduling decision.

[0125] (3) The reinforcement learning scheduling module first requests the interference injection and monitoring module to obtain the current resource status of each node. At the same time, it uses the performance model module to predict the performance of each node and scheduling application to obtain the predicted value of the performance degradation rate of the application at each node.

[0126] (4) The evaluation network of the reinforcement learning scheduling module selects the optimal node for placing the application based on the status data of the performance model module to determine the scheduling strategy.

[0127] (5) Perform scheduling according to the scheduling strategy. After the scheduling is completed, calculate the resource imbalance and application performance degradation through monitoring data, calculate the reward value and update the evaluation network to the target network. At this point, the scheduling process of reinforcement learning is completed.

[0128] For example, in a certain Internet company, facing the strict demand scenario of ensuring QoS, the traditional scheduling system failed to provide sufficient support. After adopting the application performance-aware colocation scheduling system and method described in the present invention, the results show that after the simulation application of this scheduling system, resource allocation is more efficient and stable, which significantly improves the overall performance of the system, thereby effectively supporting the company's core business needs. Through the introduction of reinforcement learning methods, the system can adaptively adjust the scheduling strategy, ensure the QoS of the application, and optimize the cluster resource utilization.

[0129] The present invention significantly improves the resource utilization and application performance guarantee capability in the co-location scenario; the present invention breaks through the limitations of the traditional Kubernetes scheduling strategy and realizes the comprehensive perception and prediction of application performance; the present invention can dynamically adjust the scheduling strategy by introducing the reinforcement learning method, and the specific implementation includes data preprocessing, feature analysis and model training, so as to ensure the accuracy and effectiveness of scheduling decisions and optimize resource allocation; it greatly improves the intelligence level of the scheduling system, effectively guarantees the QoS of the application, and improves the overall operation efficiency and stability of the cluster.

[0130] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An application performance-aware colocation scheduling system, characterized in that: include: Interference injection and monitoring module, used to manage and control the entire data collection process to obtain key performance indicators of applications on each node in the cluster; The interference injection and monitoring module includes a collector and a pressure generator, wherein the pressure generator is used to inject load on the application deployment node, and the collector is used to obtain monitoring data of the application in real time; The performance model module is used to model and analyze the performance of the application using the monitoring data collected by the interference injection and monitoring module, and to predict the performance of the application under different resource configurations through the constructed performance regression model; The reinforcement learning scheduling module is used to automatically adjust and optimize the scheduling strategy based on the analysis results of the performance model module using the reinforcement learning method; The reinforcement learning scheduling module automatically selects the optimal scheduling strategy in different colocation scenarios through continuous learning and optimization. It also adjusts scheduling parameters to suit different colocation scenarios. and The scheduling simulator module is used to use the simulation capabilities of the simulator to simulate various scheduling scenarios and application loads in the real environment, verify the feasibility and effectiveness of the scheduling strategy, and optimize it according to the verification results to obtain the optimal scheduling strategy.

2. The application performance-aware colocation scheduling system according to claim 1, characterized in that: The interference injection and monitoring module specifically includes: (a1) During the initialization phase of the monitoring task, the application performance indicator collection capability is activated, including starting the collector and stress generator on the application deployment node to ensure monitoring coverage; (a2) Injecting loads on application deployment nodes through stress generators to simulate different application loads and system stress scenarios; (a3) During the monitoring data collection and processing phase, the key performance indicators of the application are monitored in real time through the collector on the application deployment node, and the monitoring data is transmitted to the remote collector on the indicator collection node for centralized management; after receiving the monitoring data, the remote collector performs indicator analysis to obtain the key performance indicators, and continuously monitors and evaluates the key performance indicators; (a4) Display the indicator analysis results through visualization tools to generate a real-time monitoring dashboard that intuitively displays the current status and historical trends of key performance indicators; adjust the monitoring frequency and range by setting different monitoring thresholds.

3. The application performance-aware colocation scheduling system according to claim 1, characterized in that: The performance model module specifically includes: (b1) The monitoring data collected by the interference injection and monitoring module enters the performance model module for preprocessing to obtain preprocessed monitoring data; (b2) performing feature analysis on the preprocessed monitoring data to extract key features from the original monitoring data; (b3) using the key features as known inputs and the performance of the application as the prediction results, adopting multiple regression algorithms to perform regression prediction modeling, and obtaining multiple performance regression models; wherein the regression algorithms include linear regression, decision tree regression, support vector machine regression and neural network algorithm; (b4) training multiple performance regression models using the monitoring data after preprocessing and feature analysis, and adjusting the parameters of the performance regression models with the goal of optimizing performance during the training process; (b5) constructing a test set through cross-validation, using the test set to evaluate the performance of each performance regression model, and selecting the top N performance regression models with the largest prediction accuracy; and optimizing the selected top N performance regression models according to the evaluation results, adjusting their parameters and structures; (b6) According to the evaluation results, the performance regression model with the smallest prediction error and the strongest generalization ability is selected from the optimized performance regression models as the final performance regression model.

4. The application performance-aware colocation scheduling system according to claim 3, characterized in that: In the step (b1), the pretreatment specifically includes: (b1.1) Data cleaning: remove outliers and missing values ​​in monitoring data; (b1.2) De-noising: Use the moving average filtering method of smoothing filtering to remove noise and calculate the sliding average of data points to smooth the data; (b1.3) Format conversion: converting monitoring data into the format required for performance model input; (b1.4) Standardization: The monitoring data is standardized to eliminate the dimensional differences between different features.

5. The application performance-aware colocation scheduling system according to claim 1, characterized in that: In the step (b2), extracting key features from the original monitoring data specifically includes: Statistical analysis and machine learning algorithms are used to select features from the feature analysis results; the selected features are further extracted and transformed, specifically by calculating the correlation of the selected features, sorting the selected features in descending order according to the size of the correlation, and selecting the selected features corresponding to the top K maximum correlation values ​​as the extracted key features.

6. The application performance-aware colocation scheduling system according to claim 1, characterized in that: The reinforcement learning scheduling module specifically includes: (c1) using the prediction results of the performance regression model constructed in real time by the performance model module as the state input in reinforcement learning; (c2) The actions in reinforcement learning are: the application to be scheduled at the current moment is scheduled to a certain node; (c3) Design the reward in reinforcement learning. The reward function is set to the cluster resource imbalance and application performance degradation, and its expression is: Reward t =R(State t ,Action t )=αImBalance t +βAvgPerf t Among them, Reward t represents the reward value at time t, R(·) represents the reward function, and ImBalance t Indicates the resource imbalance at time t, AvgPerf t represents the application performance degradation rate at time t, α and β are the weight coefficients of resource imbalance and application performance degradation rate respectively; (c4) the evaluation network outputs actions to the scheduler for scheduling decisions based on the node status and the status of the application to be scheduled output by the interference injection and monitoring module; (c5) Every other target network parameter update cycle, the parameters of the evaluation network are copied to the target network to reduce the fluctuation of the target reward value during training.

7. The application performance-aware colocation scheduling system according to claim 1, characterized in that: The scheduling simulator module specifically includes: (d1) Create fake nodes and fake applications to simulate the actual situation in the cluster environment. The fake nodes are simulated computing nodes used to test the performance of different scheduling strategies under various resource configurations and load conditions; the fake applications are simulated application performance data used to evaluate the impact of scheduling strategies on application performance; (d2) Simulating the performance of fake applications based on the performance model, that is, applying the performance model to fake nodes and fake applications to simulate the performance of applications in the real environment. The simulator compares the generated fake performance data with the real performance data, evaluates the performance of the scheduling strategy of the reinforcement learning scheduling module under different conditions, and analyzes the impact of the scheduling strategy on application performance; (d3) Based on the performance data and analysis results generated by the simulator, the scheduling strategy is further optimized. The optimization process specifically includes: evaluating the performance of the current scheduling strategy; improving the current scheduling strategy according to the strategy evaluation results; reapplying the improved scheduling strategy to the simulator for verification, evaluating its performance in the virtual environment, and optimizing the scheduling strategy through multiple iterations.

8. A scheduling method for an application performance-aware colocation scheduling system based on any one of claims 1 to 7, characterized in that: The following steps are involved: (1) The user submits an application request, including the application name, CPU request, and memory request; (2) The scheduling simulator module receives the application's request and remotely calls the reinforcement learning scheduling module to obtain the scheduling decision; (3) The reinforcement learning scheduling module first requests the interference injection and monitoring module to obtain the current resource status of each node. At the same time, the performance model module predicts the performance of each node and scheduling application to obtain the predicted value of the performance degradation rate of the application at each node. (4) The evaluation network of the reinforcement learning scheduling module selects the optimal node for placing the application based on the status data of the performance model module to determine the scheduling strategy; (5) Perform scheduling according to the scheduling strategy. After the scheduling is completed, calculate the resource imbalance and application performance degradation through monitoring data, calculate the reward value and update the evaluation network to the target network; At this point, the scheduling process of reinforcement learning is completed.

Images