A performance-aware hybrid scheduling system and method

Abstract

The application discloses a kind of application performance perception mixed part scheduling system and method, the system includes: interference injection and monitoring module is used to manage and regulate the whole data acquisition process, to obtain the key performance indicators of application on each node in cluster;Performance model module is used to utilize the monitoring data collected to model and analyze the performance of application, to predict the performance of application under different resource configurations;Reinforcement learning scheduling module is used to utilize reinforcement learning method, according to the analysis result of performance model module, automatically adjust and optimize scheduling strategy;Scheduling simulator module is used to simulate various scheduling scenarios and application load in real environment, verify and optimize scheduling strategy.The application can automatically select optimal scheduling strategy for scheduling under mixed deployment scenario, which is beneficial to improve the performance of application and resource utilization, effectively guarantee the QoS of application, improve the overall operation efficiency and stability of cluster.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of mixed deployment scheduling, and particularly relates to a mixed deployment scheduling system and method with application performance sensing. BACKGROUND

[0002] With the rapid progress of Internet technology and the growing demand of Internet applications for computer hardware and software, cloud computing technology is gradually replacing traditional IT infrastructure. The rapid development of cloud computing technology has greatly promoted the expansion of large-scale data centers, which have become the core of modern information technology architecture, and cloud computing has gradually become the main mode of enterprise deployment and operation of enterprise services. Under the support of cloud computing technology, enterprises have greatly improved the efficiency of IT operation. However, despite huge hardware investment, a number of studies have shown that the average resource utilization rate of data center servers is very low, about 10-20%. This inefficient resource utilization not only leads to high operating costs, but also causes significant resource waste.

[0003] Since the concept of mixed deployment was proposed, this strategy of deploying multiple applications on the same physical machine using container technology has quickly attracted widespread attention. Colocation is a strategy that allocates idle resources of online clusters to offline jobs under the premise of ensuring online job performance. This strategy aims to reduce the operating cost of data centers by optimizing the utilization of hardware resources. The colocation mode allows online and offline jobs to run on the same server cluster, and effectively improves the overall resource utilization by dynamically allocating idle resources to offline jobs. This not only reduces resource idling, but also guarantees the performance requirements of online services.

[0004] The emergence of large-scale computing applications such as artificial intelligence and scientific computing has increased the demand for computing resources by offline tasks. Application performance in the mixed deployment scenario has begun to attract attention, and ensuring good service application performance can meet all quality of service (QoS) requirements of users for cloud service providers. The present application focuses on handling offline mixed deployment scenarios, in which online business mainly includes web hosting platforms, cloud storage and database services, etc., and offline business mainly includes 3D video rendering, deep learning training and data mining algorithms, etc.

[0005] With the continuous development of cloud-native technology, most of the Internet uses the open source CNCF project Kubernetes open source orchestration system to implement hybrid deployment platforms. However, as a framework designed with great versatility, the native scheduling strategy of Kubernetes has been difficult to meet the application deployment requirements in actual production environments, exposing obvious shortcomings: ① Insufficient resource awareness: Kubernetes can only perceive CPU, memory capacity, hard disk space, and other basic resources, but in the hybrid deployment scenario, application performance is not only affected by these resources, but also by the pressure of other applications and systems; this single resource awareness capability cannot meet the needs of complex and variable hybrid deployment scenarios. ② Fixed scheduling strategy: Kubernetes' scheduling strategy is fixed and lacks flexibility, and it cannot automatically adjust the scheduling strategy according to changes in the cluster environment; for example, when the resource usage or application load pattern in the cluster changes, Kubernetes cannot adaptively optimize the scheduling strategy to ensure the balance between resource utilization and application performance.

[0006] In order to cope with the complex scheduling requirements in the hybrid deployment scenario, scheduling simulation platforms are also used to verify and optimize scheduling strategies. However, the current scheduling simulation platforms also have some obvious defects: ① Insufficient performance simulation: existing scheduling simulation platforms mostly focus on simulating resource allocation and usage, lacking accurate simulation of application performance; as application performance is affected by multiple factors, simple resource allocation simulation cannot accurately reflect the application behavior and performance in actual production environments. ② Lack of support for actual application scenarios: many scheduling simulation platforms are overly idealistic in design, lacking support for various complex application scenarios in actual production environments; for example, the simulation platform may not fully consider the mutual influence between applications, fluctuations in system load, and network delays, resulting in a large deviation between simulation results and actual situations. Research shows that it is not feasible to model all elements of a system with high precision. A cloud simulator can be designed to focus on specific aspects, functions, or architectures of the cloud. Therefore, for scenarios that require simulation of application performance, a performance model can be used to simulate application performance to achieve relatively realistic simulation results.

[0007] Therefore, in order to improve system resource utilization while ensuring application QoS, an application performance-aware scheduling strategy and an application performance simulation scheduling simulator platform are proposed, which have good practical significance and application value. This not only makes up for the shortcomings of existing scheduling strategies and simulation platforms, but also provides strong support for deploying and optimizing hybrid deployment platforms in actual production environments. SUMMARY

[0008] The application aims at the deficiency of Kubernetes scheduling strategy in mixed deployment scenarios in the prior art, and provides an application performance-aware mixed deployment scheduling system and method.

[0009] The application aims at the deficiency of Kubernetes scheduling strategy in mixed deployment scenarios in the prior art, and provides an application performance-aware mixed deployment scheduling system and method.

[0010] The interference injection and monitoring module is used for managing and regulating the whole data acquisition process to obtain the key performance indicators of the application on each node in the cluster; the interference injection and monitoring module comprises an acquisition device and a stress generator; the stress generator is used for injecting load on the application deployment node; and the acquisition device is used for acquiring monitoring data of the application in real time;

[0011] The performance model module is used for modeling and analyzing the performance of the application by using the monitoring data collected by the interference injection and monitoring module, and predicting the performance of the application under different resource configurations through the constructed performance regression model;

[0012] The reinforcement learning scheduling module is used for automatically adjusting and optimizing the scheduling strategy according to the analysis result of the performance model module by using the reinforcement learning method; the reinforcement learning scheduling module is continuously learned and optimized to automatically select the optimal scheduling strategy under different mixed deployment scenarios; the scheduling parameters are adjusted to adapt to different mixed deployment scenarios; and

[0013] The scheduling simulator module is used for simulating various scheduling scenarios and application loads in the real environment by using the simulation capability of the simulator, verifying the feasibility and effectiveness of the scheduling strategy, and optimizing the scheduling strategy according to the verification result to obtain the optimal scheduling strategy.

[0014] Further, the interference injection and monitoring module specifically comprises:

[0015] (a1) in the monitoring task initialization stage, the application performance indicator acquisition capability is activated, specifically comprising starting the acquisition device and the stress generator on the application deployment node to ensure the coverage of monitoring;

[0016] (a2) the stress generator is used for injecting load on the application deployment node to simulate different application loads and system stress scenarios;

[0017] (a3) In the monitoring data collection and processing stage, the key performance indicators of the application are monitored in real time by the collector on the application deployment node, and the monitoring data is transmitted to the remote collector on the indicator collection node for centralized management; after receiving the monitoring data, the remote collector performs indicator analysis to obtain the key performance indicators, and continuously monitors and evaluates the key performance indicators;

[0018] (a4) The indicator analysis results are displayed through a visualization tool to generate a real-time monitoring dashboard, which intuitively displays the current state and historical trend of the key performance indicators; the monitoring frequency and range are adjusted by setting different monitoring thresholds.

[0019] Further, the performance model module specifically includes:

[0020] (b1) The monitoring data collected by the interference injection and monitoring module is preprocessed in the performance model module to obtain preprocessed monitoring data;

[0021] (b2) Feature analysis is performed on the preprocessed monitoring data to extract key features from the original monitoring data;

[0022] (b3) A plurality of performance regression models are obtained by using a plurality of regression algorithms to perform regression prediction modeling with the key features as known inputs and the performance of the application as prediction results; wherein the regression algorithms include linear regression, decision tree regression, support vector machine regression, and neural network algorithm;

[0023] (b4) The plurality of performance regression models are trained using the preprocessed and feature-analyzed monitoring data, and the parameters of the performance regression models are adjusted to optimize performance during the training process;

[0024] (b5) A test set is constructed by cross-validation, and the performance of each performance regression model is evaluated using the test set, and the top N performance regression models with the highest prediction accuracy are selected; and the selected top N performance regression models are optimized according to the evaluation results to adjust their parameters and structures;

[0025] (b6) According to the evaluation results, the performance regression model with the smallest prediction error and the strongest generalization ability is selected as the final performance regression model from the optimized performance regression models.

[0026] Further, in step (b1), the preprocessing specifically includes:

[0027] (b1.1) Data cleaning: removing outliers and missing values in the monitoring data;

[0028] (b1.2) Denoising: using a moving average filtering method of smoothing filtering to remove noise, and calculating the sliding average of data points to smooth the data;

[0029] (b1.3) format conversion: converting the monitoring data into a format required by the performance model input;

[0030] (b1.4) standardization processing: standardizing the monitoring data to eliminate dimensional differences between different features.

[0031] Further, in step (b2), the key features are extracted from the original monitoring data, specifically including:

[0032] statistical analysis and machine learning algorithms are used to select features from the feature analysis results; the selected features are further extracted and converted, specifically by calculating the correlation of the selected features, sorting the selected features in descending order according to the correlation, and selecting the top K correlation maximum value corresponding to the selected features as the extracted key features.

[0033] Further, the reinforcement learning scheduling module specifically includes:

[0034] (c1) the prediction result of the performance regression model constructed by the performance model module in real time is taken as the state input in reinforcement learning;

[0035] (c2) the action in reinforcement learning is to schedule the application to be scheduled to a certain node at the current time;

[0036] (c3) the reward in reinforcement learning is designed, and the reward function is set as the cluster resource imbalance degree and the application performance degradation degree, and its expression is:

[0037] Reward t =R(State t ,Action t )=αImBalance t +βAvgPerf t

[0038] wherein, Reward t represents the reward value at time t, R(·) represents the reward function, ImBalance t represents the resource imbalance degree at time t, AvgPerf t represents the application performance degradation rate at time t, and α and β are weight coefficients of the resource imbalance degree and the application performance degradation rate, respectively;

[0039] (c4) the evaluation network outputs the action according to the node state and the application state to be scheduled output by the interference injection and monitoring module to make scheduling decision for the scheduler;

[0040] (c5) every target network parameter update period, the parameters of the evaluation network are copied to the target network to reduce the fluctuation of the target reward value in the training process.

[0041] Further, the scheduling simulator module specifically comprises:

[0042] (d1) creating fake nodes and fake applications to simulate the actual situation in the cluster environment, wherein the fake nodes are simulated computing nodes for testing the performance of different scheduling strategies under various resource configurations and load conditions, and the fake applications are simulated application performance data for evaluating the impact of the scheduling strategy on application performance;

[0043] (d2) simulating the performance of the fake applications based on the performance model, i.e. applying the performance model to the fake nodes and fake applications to simulate the performance of the applications in the real environment, the simulator compares the generated fake performance data with the real performance data to evaluate the performance of the scheduling strategy of the reinforcement learning scheduling module under different conditions and analyze the impact of the scheduling strategy on application performance;

[0044] (d3) based on the performance data and analysis results generated by the simulator, further optimizing the scheduling strategy, which specifically includes: evaluating the performance of the current scheduling strategy; improving the current scheduling strategy according to the strategy evaluation results; applying the improved scheduling strategy to the simulator for verification to evaluate its performance in the virtual environment, and optimizing the scheduling strategy through multiple iterations.

[0045] The second aspect of the embodiment of the application provides a scheduling method of the application performance-aware mixed-deployment scheduling system, which comprises the following steps:

[0046] (1) the user submits a request for an application, including the name of the application, the CPU request and the memory request;

[0047] (2) the scheduling simulator module receives the request for the application, remotely calls the reinforcement learning scheduling module, and obtains a scheduling decision;

[0048] (3) the reinforcement learning scheduling module first requests the interference injection and monitoring module to obtain the resource state of each node, and simultaneously performs performance prediction on each node and the scheduled application through the performance model module to obtain the predicted value of the performance degradation rate of the application on each node;

[0049] (4) the evaluation network of the reinforcement learning scheduling module selects the optimal node for placing the application according to the state data of the performance model module to determine the scheduling strategy;

[0050] (5) scheduling is performed according to the scheduling strategy, after the scheduling is completed, the resource imbalance degree and the application performance degradation are calculated through the monitoring data, the reward value is calculated and the evaluation network is updated to the target network; thus, the scheduling process of one reinforcement learning is completed.

[0051] Compared with the prior art, the application has the beneficial effects that:

[0052] (1) The application provides a comprehensive and efficient mixed mode scheduling system through innovative individual module composition, greatly improving the reliability, performance and resource utilization of the Kubernetes cluster.

[0053] (2) The application innovatively proposes a dynamic interference injection and monitoring framework, and the interference injection in the interference injection and monitoring module generates a controllable pressure environment by simulating different application loads and system pressure scenarios, so as to test the performance of the application under various conditions; at the same time, the monitoring module collects key performance indicators such as delay, throughput and resource utilization in real time, and provides high-precision monitoring data; effectively solve the interference injection and monitoring problem of the observable system, and it is beneficial to collect more comprehensive monitoring data sets.

[0054] (3) The performance-aware reinforcement learning scheduling module proposed by the application is specially designed for application performance prediction and scheduling optimization in mixed deployment scenarios, and the reinforcement learning scheduling module has superior performance and self-learning ability to solve the problem of serialized decision-making, and the scheduling strategy is not fixed, which can dynamically adjust the strategy to adapt to the changes of the cluster environment; through real-time monitoring and data collection, a detailed performance model is established to predict the performance of the application under different resource configurations, and the scheduling strategy is dynamically adjusted according to the application QoS and cluster resource balance; through the optimization of the reinforcement learning algorithm design and reasonable resource configuration, the resource consumption of large-scale real scheduling experiments is effectively reduced, and the accuracy and effectiveness of the scheduling decision are ensured.

[0055] (4) The application proposes a scheduling simulator based on a performance model, which solves the problem that there is currently a lack of application performance simulation scheduling tools for Kubernetes, and the scheduling simulator module can simulate various scheduling scenarios and application loads in the real production environment through the prediction data provided by the performance model module, verify and optimize the scheduling strategy; the simulator can not only simulate complex scenarios from single machine to multi-node, but also can perform performance testing and function testing, compared with the original scheme, the optimization effect of the new scheme in various indicators is displayed; through this simulation platform, users can fully verify the effect of the scheduling strategy before actual deployment, reduce the risk in the actual production environment, and ensure the stability and efficiency of the system.

[0056] (5) The application significantly improves the resource utilization rate and application performance guarantee capability in the mixed deployment scene; the application breaks through the limitation of the traditional Kubernetes scheduling strategy, realizes comprehensive perception and prediction of application performance; the application can dynamically adjust the scheduling strategy by introducing the reinforcement learning method, ensures the accuracy and effectiveness of the scheduling decision, optimizes the resource allocation; greatly improves the intelligent level of the scheduling system, effectively guarantees the QoS of the application, and improves the overall operation efficiency and stability of the cluster. BRIEF DESCRIPTION OF DRAWINGS

[0057] Figure 1 is a structural block diagram of the application performance perception mixed deployment scheduling system;

[0058] Figure 2 is a structural block diagram of the interference injection and monitoring module of the application;

[0059] Figure 3 is a flowchart of the application performance perception scheduling strategy based on reinforcement learning of the application;

[0060] Figure 4 is an architectural block diagram of the scheduling simulator module of the application. DETAILED DESCRIPTION

[0061] The exemplary embodiments will be described in detail herein with reference to the attached drawings. The following description is only exemplary and is not intended to limit the present application. Rather, the following description is intended to explain the exemplary embodiments consistent with the present application, and to enable a person skilled in the art to make and use the application. It will be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application.

[0062] The terms used in the present application are only for the purpose of describing specific embodiments, and are not intended to limit the present application. The singular forms "a", "an" and "the" as used in the present application and the appended claims are also intended to include the plural forms, unless the context clearly indicates otherwise. It should also be understood that the term "and / or" as used herein means and includes any or all possible combinations of one or more associated listed items.

[0063] It should be understood that, although the terms first, second, third, etc. can be employed in this application to describe various information, these information should not be limited to these terms. These terms are only used to distinguish one type of information from another type of information. For example, without departing from the scope of the present application, the first information can also be referred to as the second information, and similarly, the second information can also be referred to as the first information. Depending on the context, the word "if' as used herein can be interpreted to mean "when" or "upon" or "in response to determining". Also, the term "include" or "contain" or any other variant thereof is intended to cover a non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed, or further includes elements inherent to such a process, method, article or device. Without more limitations, the element defined by the statement "includes a" does not exclude the presence of additional identical elements in the process, method, article or device including the element.

[0064] The present application will be described in detail below with reference to the accompanying drawings. The features in the following examples and embodiments can be combined with each other without conflict.

[0065] The application performance-aware mixed deployment scheduling system of the present application includes an interference injection and monitoring module, a performance model module, a reinforcement learning scheduling module and a scheduling simulator module, as shown in Figure 1 .

[0066] In this embodiment, the interference injection and monitoring module is used to manage and regulate the entire data collection process to obtain the key performance indicators of the applications on each node in the cluster; the interference injection and monitoring module includes a collector and a stress generator, as shown in Figure 2 , the stress generator is used to inject load on the application deployment node, and the collector is used to obtain real-time monitoring data of the application.

[0067] It should be understood that the interference injection and monitoring module plays a crucial role, which is responsible for fine management and regulation of the entire data collection process. The interference injection and monitoring module deeply penetrates the cluster environment, real-time monitors the resource usage and application performance data on each node, and provides comprehensive and accurate basic data support for the scheduling strategy by accurately capturing the usage of CPU, memory, network and other key resources. This deep monitoring provides unprecedented transparency for cluster administrators, making the analysis of resource utilization and application performance more accurate and comprehensive.

[0068] It should be noted that the present application embodiment proposes an application performance indicator collection framework for mixed deployment scenarios, which allows users to flexibly configure according to actual needs to ensure the efficiency and accuracy of data collection.

[0069] Further, as shown in Figure 2 The interference injection and monitoring module specifically includes:

[0070] (a1) In the monitoring task initialization phase, the application performance index collection capability is activated, specifically including starting the collector and stress generator on the application deployment node to ensure the coverage of the monitoring.

[0071] (a2) Inject load on the application deployment node through the stress generator to simulate different application load and system stress scenarios.

[0072] It should be noted that the stress generator can generate controllable stress, and through the generated controllable stress, the performance of the application under different load conditions can be tested, helping to identify potential performance bottlenecks and optimization opportunities.

[0073] (a3) In the monitoring data collection and processing phase, the collector on the application deployment node monitors the key performance indicators of the application in real time, and transmits the monitoring data to the remote collector on the index collection node for centralized management; after the remote collector receives the monitoring data, it performs index analysis to obtain key performance indicators such as delay, throughput, resource utilization, etc., and continuously monitors and evaluates the key performance indicators.

[0074] It should be noted that the key performance indicators are some performance indicator parameters required for scheduling, which can be obtained according to actual needs, including but not limited to the use of CPU, memory, network and other key resources, as well as delay, throughput, etc.

[0075] (a4) The index analysis results are displayed through a visualization tool (such as Grafana) to generate a real-time monitoring dashboard that visually displays the current state and historical trends of key performance indicators; users can set different monitoring thresholds, adjust monitoring frequency and range according to specific needs, to ensure the adaptability and scalability of the monitoring system.

[0076] It should be understood that through these detailed operation steps, the interference injection and monitoring module not only improves the efficiency and accuracy of data collection, but also through its modular architecture, ensures the scalability and maintainability of the system. In summary, the interference injection and monitoring module provides strong support for application performance optimization in mixed deployment scenarios.

[0077] In this embodiment, the performance model module is used to model and analyze the performance of the application using the monitoring data collected by the interference injection and monitoring module, and to predict the performance of the application under different resource configurations through the constructed performance regression model.

[0078] It should be noted that the performance model module plays a crucial role in the present application, which is responsible for in-depth analysis and processing of the interference injection and monitoring data collected by the monitoring module, modeling and analyzing the performance of the application, predicting the performance of the application under different resource configurations through detailed performance regression models, and providing scientific basis for subsequent scheduling strategy optimization. The performance model module uses machine learning methods to model and predict the performance of the application using the monitoring data collected by the interference injection and monitoring module, expands the original scheduling strategy, and verifies its feasibility in Kubernetes scheduling; this includes monitoring and analyzing key performance indicators such as latency, throughput and resource utilization, etc., so as to ensure that the scheduling strategy can adapt to changes in the cluster environment and optimize resource utilization.

[0079] Further, the performance model module specifically includes:

[0080] (b1) The monitoring data collected by the interference injection and monitoring module enters the performance model module for preprocessing to obtain preprocessed monitoring data.

[0081] Further, the preprocessing specifically includes:

[0082] (b1.1) Data cleaning: removing outliers and missing values in the monitoring data to ensure the accuracy and consistency of the monitoring data.

[0083] (b1.2) Denoising: using a moving average filter method to remove noise, calculating the sliding average of data points to smooth the data, which helps to remove short-term fluctuations.

[0084] (b1.3) Format conversion: converting the monitoring data into the format required by the performance model input, which can ensure the availability and compatibility of the monitoring data.

[0085] (b1.4) Standardization: standardizing the monitoring data to eliminate the dimensional differences between different features, which helps to improve the training effect of the performance model.

[0086] Specifically, for an application, the monitoring data collected by the interference injection and monitoring module will first enter the performance model module for preprocessing, which is a key step to ensure the accuracy of model training and prediction, including data cleaning, denoising, format conversion and standardization.

[0087] (b2) Feature analysis of preprocessed monitoring data to extract key features that have a greater impact on the prediction results from the original monitoring data.

[0088] Further, key features that have a greater impact on the prediction result are extracted from the original monitoring data, specifically including: using statistical analysis and machine learning algorithms to perform feature selection on the feature analysis result, selecting features that have an important impact on application performance prediction; further extracting and converting the selected features, specifically by calculating the correlation of the selected features, sorting the selected features in descending order according to the correlation, and selecting the top K correlation maximum value corresponding to the selected features as the extracted key features. Using key features for subsequent analysis can effectively enhance the representativeness and prediction ability of the features.

[0089] (b3) Using the key features as known inputs and the performance of the application as the prediction result, a plurality of performance regression models are obtained by using a plurality of regression algorithms for regression prediction modeling. The regression algorithms include linear regression, decision tree regression, support vector machine regression, and neural network algorithms.

[0090] (b4) The monitoring data after preprocessing and feature analysis is used to train the plurality of performance regression models, and the parameters of the performance regression models are adjusted to optimize the performance during the training process.

[0091] (b5) A test set is constructed by cross-validation, and the performance of each performance regression model is evaluated using the test set. The top N performance regression models with the highest prediction accuracy are selected. The selected top N performance regression models are optimized according to the evaluation results to adjust their parameters and structures to further improve the prediction accuracy.

[0092] It should be understood that by constructing a test set through cross-validation, for example, five-fold cross-validation, and then testing the performance regression model using the test set, the generalization ability of the performance regression model can be ensured.

[0093] (b6) According to the evaluation results, the performance regression model with the smallest prediction error and the strongest generalization ability is selected from the optimized performance regression models as the final performance regression model.

[0094] It should be understood that through the above detailed operation steps, the performance model module not only improves the accuracy and reliability of application performance prediction, but also provides strong support for resource scheduling optimization in mixed deployment scenarios. In summary, the performance model module has important significance in ensuring application QoS and system resource utilization.

[0095] In this embodiment, the reinforcement learning scheduling module is used to automatically adjust and optimize the scheduling strategy according to the analysis result of the performance model module by using the reinforcement learning method; the reinforcement learning scheduling module is continuously learned and optimized to automatically select the optimal scheduling strategy in different mixed deployment scenarios, which can effectively improve the system resource utilization and application performance; the user can adjust the scheduling parameters such as the type and frequency of the scheduling strategy according to the specific needs to adapt to different mixed deployment scenarios, so that the application not only adapts to a wide range of mixed deployment environments, but also meets the specific needs of specific users, which helps to customize the scheduling strategy flexibly.

[0096] It should be understood that the reinforcement learning scheduling module is one of the core innovations of the application, which identifies the limitations of the existing cloud computing job scheduling strategy in the mixed deployment scenario through deep research, and this in-depth analysis provides unprecedented transparency for the improvement of the scheduling strategy, making the optimization of the scheduling strategy more accurate and comprehensive.

[0097] Further, based on the effectiveness of reinforcement learning in sequential decision problems, the application proposes an application performance-aware scheduling strategy based on reinforcement learning, as shown in Figure 3 The strategy innovatively combines the performance model with the reinforcement learning scheduling module, and takes the prediction result of the performance model as part of the state, thereby enhancing the scheduling strategy's awareness and prediction ability of the application performance. As shown in Figure 3 The reinforcement learning scheduling module specifically includes:

[0098] (c1) taking the prediction result of the performance regression model built by the performance model module in real time as the state input in reinforcement learning.

[0099] Specifically, the state s includes: the resource inventory matrix state (CPU, memory, network bandwidth and disk bandwidth) of each node and the resource request matrix (CPU, memory, network bandwidth and disk bandwidth) of the current scheduled application, which is expressed as:

[0100]

[0101] wherein State t represents the state at time t, represents the resource inventory of node n at time t, n is the total number of nodes at time t, Pod t represents the matrix of the application (Pod) to be scheduled at time t.

[0102] It should be noted that the performance model module establishes a detailed application performance regression model through real-time monitoring and data collection, predicts the performance of the application under different resource configurations, and uses the prediction results as the state input of the reinforcement learning method to provide more accurate and comprehensive environment information, so that the reinforcement learning method can make more optimized scheduling decisions.

[0103] (c2) the action in reinforcement learning is to schedule the application to be scheduled to a certain node at the current time, denoted as:

[0104] Action t ∈{Node i ∣i=1,2,…,n}

[0105] wherein Action t represents the action at time t.

[0106] (c3) the reward in reinforcement learning is designed, and the reward function is set as the cluster resource imbalance degree and the application performance degradation degree, and the expression is:

[0107] Reward t =R(State t ,Action t )=αImBalance t +βAvgPerf t

[0108] wherein Reward t represents the reward value at time t, R(·) represents the reward function, ImBalance t represents the resource imbalance degree at time t, AvgPerf t represents the application performance degradation rate at time t, and α and β are weight coefficients of the resource imbalance degree and the application performance degradation rate. The resource imbalance degree ImBalance is the standard deviation of all node resources, and the more uneven the resources are, the larger the ImBalance is. AvgPerf is the performance degradation rate of all applications compared with the optimal performance.

[0109] It should be noted that the cluster resource balance degree is obtained by measuring the distribution of resources in the cluster, and the reward function encourages the scheduling strategy to achieve more balanced resource allocation, thereby improving the resource utilization. The application performance degradation degree is obtained by measuring the degradation degree of the application performance, and the reward function encourages the scheduling strategy to minimize the performance degradation, thereby ensuring the QoS of the application.

[0110] (c4) the network outputs the action according to the node state and the application state to be scheduled output by the interference injection and monitoring module, and makes scheduling decisions for the scheduler.

[0111] It should be noted that, under the premise of ensuring the application QoS, improving the resource balance degree and utilization is the optimization goal of the reinforcement learning scheduling module; the performance model module inputs the application name and selects the scheduling node state, and outputs the predicted performance degradation rate; the interference injection and monitoring module can output the state of all nodes in real time, and the node state can be obtained through the interference injection and monitoring module.

[0112] (c5) every other target network parameter update period, the parameters of the evaluation network are copied to the target network to reduce the fluctuation of the target reward value in the training process, which can solve the strong correlation problem in the deep Q network reinforcement learning (DQN) algorithm, and help to improve the stability of the reinforcement learning method.

[0113] In this embodiment, the scheduling simulator module is used to simulate various scheduling scenarios and application loads in the real environment by using the simulation capability of the simulator, verify the feasibility and effectiveness of the scheduling strategy, and optimize it according to the verification result to obtain the optimal scheduling strategy.

[0114] It should be understood that in the field of cluster scheduling, verification using a simulator has become a key evaluation tool due to its fast and low-cost advantages. The present application proposes an innovative scheduling simulator architecture, as shown in the architectural diagram Figure 4 The scheduling simulator module verifies and optimizes the scheduling strategy through the simulation capability of large-scale scenarios. This simulation mechanism is crucial for ensuring the stability and efficiency of the scheduling strategy in the actual production environment. The key contents of the scheduling simulator are described in detail below.

[0115] Further, the scheduling simulator module specifically includes:

[0116] (d1) creating fake nodes and fake applications to simulate the actual situation in the cluster environment, wherein the fake nodes are simulated computing nodes for testing the performance of different scheduling strategies under various resource configurations and load conditions; the fake applications are simulated application performance data for evaluating the impact of the scheduling strategy on the application performance.

[0117] In this process, the collection of control flow events mainly focuses on the life cycle management of the device, including events such as initialization, connection establishment, data transmission and connection termination of the device. By simulating the life cycle management of the device in detail, the simulator can capture key control flow events and provide accurate reference data for the optimization of the scheduling strategy.

[0118] (d2) Simulate the performance of the fake application based on the performance model, that is, apply the performance model to the fake nodes and fake applications to simulate the performance of the application in the real environment. The simulator compares the generated fake performance data with the real performance data, evaluates the performance of the scheduling strategy of the reinforcement learning scheduling module under different conditions, analyzes the influence of the scheduling strategy on the application performance, and specifically includes evaluating the resource utilization, application delay and throughput and other key performance indicators of the scheduling strategy under different load conditions.

[0119] (d3) Based on the performance data and analysis results generated by the simulator, further optimize the scheduling strategy, and the optimization process specifically includes: evaluating the performance of the current scheduling strategy, identifying its shortcomings in resource utilization and application performance; according to the evaluation results of the strategy, improve the current scheduling strategy and propose improvement suggestions, for example, the resource allocation strategy can be adjusted, the load balancing mechanism can be increased, or a more intelligent scheduling algorithm can be used; the improved scheduling strategy is re-applied to the simulator for verification, and the performance in the virtual environment is evaluated, and the scheduling strategy is optimized through multiple iterations to gradually improve the effectiveness and stability of the scheduling strategy.

[0120] In summary, the scheduling simulator module of the present application creates fake nodes and fake applications, simulates application performance based on a performance model, optimizes the scheduling strategy, and performs experimental verification, providing an efficient and accurate scheduling evaluation tool that significantly improves the development and optimization efficiency of the scheduling strategy.

[0121] It is worth mentioning that the embodiment of the present application also provides a scheduling method based on the application performance-aware mixed deployment scheduling system in the above embodiment.

[0122] The scheduling method of the present application specifically includes the following steps:

[0123] (1) The user submits a request for the application, including the name of the application, the CPU request and the memory request, etc.

[0124] (2) The scheduling simulator module receives the application request, remotely calls the reinforcement learning scheduling module, and obtains the scheduling decision.

[0125] (3) The reinforcement learning scheduling module first requests the interference injection and monitoring module to obtain the resource state of each node, and simultaneously performs performance prediction on each node and the scheduled application through the performance model module to obtain the predicted value of the performance degradation rate of the application on each node.

[0126] (4) The evaluation network of the reinforcement learning scheduling module selects the optimal node for placing the application according to the state data of the performance model module to determine the scheduling strategy.

[0127] (5) According to the scheduling strategy, after scheduling is completed, the resource imbalance degree and the application performance decline are calculated through monitoring data, the reward value is calculated and the evaluation network is updated to the target network. Thus, the scheduling process of one reinforcement learning is completed.

[0128] Exemplarily, in a certain Internet company, the traditional scheduling system cannot provide sufficient support in the face of strict demand scenarios for guaranteeing QoS. After using the application performance aware mixed deployment scheduling system and method described in the application, the results show that after simulating this scheduling system, the resource allocation is more efficient and stable, significantly improving the overall performance of the system, thereby effectively supporting the core business needs of the company. Through the introduction of the reinforcement learning method, the system can adaptively adjust the scheduling strategy to ensure the QoS of the application and optimize the utilization rate of cluster resources.

[0129] The application significantly improves the resource utilization rate and application performance guarantee capability in the mixed deployment scenario; the application breaks through the limitations of the traditional Kubernetes scheduling strategy, realizes comprehensive perception and prediction of application performance; the application can dynamically adjust the scheduling strategy by introducing the reinforcement learning method, and the specific implementation includes data preprocessing, feature analysis and model training, ensuring the accuracy and effectiveness of scheduling decisions and optimizing resource allocation; greatly improving the intelligent level of the scheduling system, effectively guaranteeing the QoS of the application, and improving the overall operation efficiency and stability of the cluster.

[0130] The above embodiments are only used to illustrate the technical solutions of the application, but not limit it; although the application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the application.

Claims

1. An application performance-aware hybrid scheduling system, the system comprising: Comprise: An interference injection and monitoring module for managing and regulating the entire data collection process to obtain key performance indicators of applications on each node in the cluster; The interference injection and monitoring module comprises a collector and a stress generator, and the stress generator is used to inject load on the application deployment node, and the collector is used to obtain monitoring data of the application in real time; A performance model module for modeling and analyzing the performance of the application using the monitoring data collected by the interference injection and monitoring module, and predicting the performance of the application under different resource configurations through the constructed performance regression model; A reinforcement learning scheduling module for automatically adjusting and optimizing the scheduling strategy according to the analysis results of the performance model module using the reinforcement learning method; The reinforcement learning scheduling module continuously learns and optimizes to automatically select the optimal scheduling strategy in different mixed deployment scenarios; adjust the scheduling parameters to adapt to different mixed deployment scenarios; And A scheduling simulator module for simulating various scheduling scenarios and application loads in a real environment using the simulation capabilities of the simulator to verify the feasibility and effectiveness of the scheduling strategy, and optimize it according to the verification results to obtain the optimal scheduling strategy.

2. The performance-aware hybrid block scheduling system of claim 1, wherein, The interference injection and monitoring module specifically comprises: (a1) In the monitoring task initialization stage, activate the application performance indicator collection capability, specifically including starting the collector and stress generator on the application deployment node to ensure the coverage of monitoring; (a2) Inject load on the application deployment node through the stress generator to simulate different application loads and system pressure scenarios; (a3) In the monitoring data collection and processing stage, the collector on the application deployment node monitors the key performance indicators of the application in real time, and transmits the monitoring data to the remote collector on the indicator collection node for centralized management; after the remote collector receives the monitoring data, it analyzes the indicators to obtain the key performance indicators, and continuously monitors and evaluates the key performance indicators; (a4) The indicator analysis results are displayed through a visualization tool to generate a real-time monitoring dashboard to visually display the current state and historical trend of the key performance indicators; adjust the monitoring frequency and range by setting different monitoring thresholds.

3. The performance-aware hybrid block scheduling system of claim 1, wherein, The performance model module specifically comprises: (b1) The monitoring data collected by the interference injection and monitoring module enters the performance model module for preprocessing to obtain preprocessed monitoring data; (b2) Perform feature analysis on the preprocessed monitoring data to extract key features from the original monitoring data; (b3) Use key features as known input and application performance as prediction results to perform regression prediction modeling using multiple regression algorithms to obtain multiple performance regression models; wherein the regression algorithms include linear regression, decision tree regression, support vector machine regression, and neural network algorithm; (b4) Train multiple performance regression models using preprocessed and feature-analyzed monitoring data, and adjust the parameters of the performance regression models during the training process to optimize performance. (b5) constructing a test set by cross-validation, evaluating the performance of each performance regression model by using the test set, selecting the top N performance regression models with the highest prediction accuracy, and optimizing the selected top N performance regression models according to the evaluation results to adjust their parameters and structures; (b6) selecting the performance regression model with the smallest prediction error and the strongest generalization ability from the optimized performance regression models as the final performance regression model according to the evaluation results.

4. The application performance aware hybrid dispatch system of claim 3, wherein, In the step (b1), the preprocessing specifically includes: (b1.1) data cleaning: removing outliers and missing values in the monitoring data; (b1.2) noise removal processing: removing noise by using a moving average filtering method of smoothing filtering, and calculating the sliding average of data points to smooth the data; (b1.3) format conversion: converting the monitoring data into the format required by the performance model input; (b1.4) standardization processing: standardizing the monitoring data to eliminate the dimensional differences between different features.

5. The application performance aware hybrid component scheduling system of claim 3, wherein, In the step (b2), the key features extracted from the original monitoring data specifically include: performing feature selection on the feature analysis results by using statistical analysis and machine learning algorithms; further extracting and converting the selected features, specifically by calculating the correlation of the selected features, sorting the selected features in descending order according to the correlation, and selecting the top K selected features corresponding to the maximum correlation values as the extracted key features.

6. The performance-aware hybrid block scheduling system of claim 1, wherein, The reinforcement learning scheduling module specifically includes: (c1) taking the prediction results of the performance regression model constructed by the performance model module in real time as the state input in reinforcement learning; (c2) the action in reinforcement learning is to schedule the application to be scheduled to a certain node at the current time; (c3) designing the reward in reinforcement learning, and setting the reward function as the cluster resource imbalance degree and the application performance degradation degree, and the expression is: ； wherein, represents a reward value at time t, represents a reward function, represents a resource imbalance degree at time t, represents an application performance degradation rate at time t, and are weight coefficients of the resource imbalance degree and the application performance degradation rate, respectively. (c4) the evaluation network outputs the action to the scheduler for scheduling decision-making according to the node state and the application state output by the interference injection and monitoring module; (c5) every other target network parameter update period, the parameters of the evaluation network are copied to the target network to reduce the fluctuation of the target reward value in the training process.

7. The performance-aware hybrid component scheduling system of claim 1, wherein, The scheduling simulator module specifically includes: (d1) creating fake nodes and fake applications to simulate the actual situation in the cluster environment, wherein the fake nodes are simulated computing nodes for testing the performance of different scheduling strategies under various resource configurations and load conditions; the fake applications are simulated application performance data for evaluating the influence of the scheduling strategy on the application performance; (d2) simulating the performance of the fake applications based on the performance model, that is, applying the performance model to the fake nodes and fake applications to simulate the performance of the applications in the real environment, and comparing the generated fake performance data with the real performance data to evaluate the performance of the scheduling strategy of the reinforcement learning scheduling module under different conditions and analyze the influence of the scheduling strategy on the application performance. (d3) Based on the performance data and analysis results generated by the simulator, further optimize the scheduling strategy, and the optimization process specifically includes: evaluating the performance of the current scheduling strategy; According to the strategy evaluation results, improve the current scheduling strategy; The improved scheduling strategy is applied to the simulator for verification, and its performance in the virtual environment is evaluated, and the scheduling strategy is optimized through multiple iterations.

8. A dispatching method based on the application performance-aware hybrid dispatching system according to any one of claims 1-7, characterized in that, Comprise the following steps: (1) The user submits an application request, including the name of the application, CPU request and memory request; (2) The scheduling simulator module receives the application request, remotely calls the reinforcement learning scheduling module, and obtains the scheduling decision; (3) The reinforcement learning scheduling module first requests the interference injection and monitoring module to obtain the current resource state of each node, and simultaneously performs performance prediction on each node and the scheduled application through the performance model module to obtain the predicted value of the application performance degradation rate under the performance of each node; (4) The evaluation network of the reinforcement learning scheduling module selects the optimal node for placing the application according to the state data of the performance model module to determine the scheduling strategy; (5) According to the scheduling strategy, scheduling is performed, and after the scheduling is completed, the resource imbalance degree and the application performance degradation are calculated through the monitoring data, the reward value is calculated, and the evaluation network is updated to the target network; At this point, the scheduling process of a reinforcement learning is completed.

Images