A cloud native computing resource dynamic scheduling control method based on reinforcement learning

Abstract

This invention discloses a dynamic scheduling and control method for cloud-native computing resources based on reinforcement learning, relating to the field of cloud-native computing resource scheduling technology. The method includes the following steps: establishing a traditional scheduling method and a reinforcement learning scheduling method; collecting task volume and computing resource requirements for different time periods on different dates to obtain task-related data for each time period; calculating task volume indicators and fluctuation indicators for each time period to obtain task indicator data for each time period; classifying each time period and obtaining corresponding indicator thresholds; and controlling the scheduling method based on the corresponding indicator thresholds. This invention addresses the problem that existing cloud-native computing resource scheduling technologies, when scheduling computing resources in batch processing task scheduling scenarios, cannot dynamically set thresholds for switching scheduling methods based on historical task volume and scheduling conditions for different time periods, thus failing to dynamically control the scheduling method.

Description

Technical Field

[0001] This invention relates to the field of cloud-native computing resource scheduling technology, specifically to a dynamic scheduling and control method for cloud-native computing resources based on reinforcement learning. Background Technology

[0002] Cloud-native computing resource scheduling technology is a core underlying control technology that follows cloud-native technology concepts and architectural specifications, takes containerization, declarative APIs, and immutable infrastructure as its core foundation, and Kubernetes as the industrial-grade de facto standard carrier. It is designed for heterogeneous computing resources and all types of workloads in cloud-native distributed environments, and enables precise matching of resource supply and demand and dynamic management of the entire lifecycle.

[0003] Existing cloud-native computing resource scheduling technologies, when scheduling computing resources in batch processing task scenarios, such as the statistical processing of business reports for chain retail enterprises, often only use traditional scheduling methods based on the native Kubernetes scheduling framework; or rely on manual subjective control, using reinforcement learning-based scheduling methods when necessary. Traditional scheduling methods are suitable for situations with low task volume or high task volume but stable load, with the core advantage of definite and controllable decision-making and no additional maintenance costs; they are the basic scheduling solution for batch processing scenarios. Reinforcement learning-based scheduling methods are suitable for situations with high task volume and large load fluctuations in batch processing tasks, with the core advantage of achieving globally optimal decision-making, dynamically adapting to load fluctuations, and having a much stronger ability to cope with sudden loads than traditional scheduling. However, using only traditional scheduling methods is insufficient for high task volume and high fluctuation scenarios. During peak periods, allocable computing resources are already in a tight balance, and resource demand fluctuates greatly. Traditional static scheduling rules cannot consider subsequent resource demands and the overall resource situation, nor can they adapt to dynamically changing task urgency. This can easily lead to queue congestion, causing the core task guarantee capability to fail, and making it impossible to use resources rationally and schedule them appropriately. On the other hand, manual subjective control relies entirely on the personal experience of operations and maintenance personnel. Different personnel have different judgments, and the switching of scheduling strategies is completely uncontrollable, which can easily lead to inappropriate switching. Therefore, existing cloud-native computing resource scheduling technologies, when scheduling computing resources in batch processing task scheduling scenarios, cannot dynamically set the threshold for switching scheduling methods based on the task volume and scheduling status of different historical periods, and thus cannot dynamically control the scheduling methods. Summary of the Invention

[0004] This invention aims to at least partially address one of the technical problems in the prior art. It establishes both traditional and reinforcement learning scheduling methods, collects task volume and computing resource requirements for different time periods on different dates to obtain task-related data for each time period, calculates task volume and fluctuation indicators for each time period to obtain task indicator data for each time period, classifies each time period, and obtains corresponding indicator thresholds. Based on these thresholds, it controls the scheduling methods. This addresses the problem that existing cloud-native computing resource scheduling technologies, when scheduling computing resources in batch processing task scenarios, cannot dynamically set thresholds for switching scheduling methods based on historical task volume and scheduling conditions for different time periods, thus failing to dynamically control the scheduling methods.

[0005] To achieve the above objectives, this application provides a dynamic scheduling and control method for cloud-native computing resources based on reinforcement learning, comprising the following steps:

[0006] We established traditional scheduling methods and reinforcement learning scheduling methods, and collected task volume and computing resource requirements for different time periods on different dates to obtain task-related data for each time period.

[0007] Calculate the task volume index and fluctuation index for each time period based on the task-related data for each time period, and obtain the task index data for each time period.

[0008] The task indicator data for each time period are categorized, and the corresponding indicator thresholds are obtained.

[0009] The scheduling method is controlled based on the current task volume and fluctuation indicators, and the corresponding indicator thresholds.

[0010] Furthermore, traditional scheduling methods and reinforcement learning scheduling methods are established, and task volume and computational resource requirements for different time periods on different dates are collected to obtain task-related data for each time period, including the following sub-steps:

[0011] In batch task scheduling scenarios, for the scheduling of cloud-native computing resources, there are two scheduling methods: one based on manually predefined static rules, heuristic algorithms, and fixed thresholds, which are called traditional scheduling methods; and another based on reinforcement learning, which are called reinforcement learning scheduling methods.

[0012] Based on the daily cloud-native computing resources required, the date type is divided into Category I dates and Category II dates; Category I dates require fewer cloud-native computing resources than Category II dates.

[0013] Each day is divided into multiple uniform time periods by a first time interval, where the first time interval is t1; any date is designated as the first date, and any time period of the first date is designated as the first time period.

[0014] Furthermore, establishing traditional scheduling methods and reinforcement learning scheduling methods, and collecting task volume and computational resource requirements for different time periods on different dates to obtain task-related data for each time period also includes the following sub-steps:

[0015] Within the first time period, tasks currently in the queue and running states are synchronously retrieved every t2 seconds. The CPU, memory, and GPU requests values ​​of all corresponding Pods are accumulated and recorded in order as CPU usage, memory usage, and GPU usage, respectively. After completion, the CPU usage sequence, memory usage sequence, and GPU usage sequence of the first time period are obtained and recorded as the resource demand information of the first time period; where t2 is the set duration.

[0016] Within the first time period, every t2 time interval, the system synchronously retrieves all nodes in the Ready and schedulable states at the current moment with the resource demand information. It then accumulates the Allocatable values ​​of CPU, memory, and GPU for all nodes, and records them in order as the total CPU, total memory, and total GPU. After completion, the system obtains the total CPU, total memory, and total GPU sequences for the first time period, which are recorded as the total resource information for the first time period.

[0017] Furthermore, establishing traditional scheduling methods and reinforcement learning scheduling methods, and collecting task volume and computational resource requirements for different time periods on different dates to obtain task-related data for each time period also includes the following sub-steps:

[0018] Within the first time period, the sum of the number of tasks in the queue and running state at the current moment is collected every t2 time interval and recorded as the instantaneous total number of tasks. After completion, the instantaneous task quantity sequence is obtained.

[0019] The resource demand information, total resource information, and instantaneous task volume sequence for the first time period are recorded as the task-related data for the first time period; the task-related data for each date and time period are collected repeatedly.

[0020] Within the first time period, obtain the scheduling method used in the first time period, and obtain the proportion of tasks completed on time within the first time period, which is recorded as the task completion rate of the first time period.

[0021] Within the first time period, the sum of the actual usage of CPU, memory, and GPU of all schedulable nodes is collected every t2 time interval as a percentage of the corresponding total allocable resources, and recorded as CPU utilization, memory utilization, and GPU utilization, respectively; after completion, CPU utilization sequence, memory utilization sequence, and GPU utilization sequence are obtained.

[0022] Calculate the mean values ​​of the CPU utilization sequence, memory utilization sequence, and GPU utilization sequence respectively, and take the largest mean value as the resource utilization rate of the first time period; repeatedly obtain the task completion rate and resource utilization rate for each time period of each date.

[0023] Furthermore, based on the task-related data for each time period, the task volume indicators and fluctuation indicators for each time period are calculated, and the task indicator data for each time period are obtained through the following sub-steps:

[0024] For the first time period, outliers in the CPU usage sequence are filtered out and removed, and the average value of the remaining CPU usage sequence is calculated and recorded as the effective mean of the CPU usage sequence, denoted as YC.

[0025] Repeatedly obtain the effective mean values ​​of the memory usage sequence, GPU usage sequence, CPU total usage sequence, memory total usage sequence, and GPU total usage sequence, and denote them as YN, YG, ZC, ZN, and ZG respectively in order;

[0026] Calculate YC / ZC, YN / ZN, and YG / ZG separately, and record them as the resource usage ratio. Record the largest resource usage ratio as the resource demand saturation AE of the first period, and mark it as the task volume indicator.

[0027] Furthermore, the calculation of task volume and fluctuation indicators for each time period based on task-related data for each time period, and the resulting task indicator data for each time period, also includes the following sub-steps:

[0028] Filter out outliers in the instantaneous task volume sequence and remove them. Calculate the average value RP, maximum value RA, and minimum value RB of the remaining instantaneous task volume sequence. Calculate (RA-RB) / RP, and record it as the intra-segment volatility V1 of the first time period.

[0029] Dates within k1 days before the first date that have the same date type as the first date are recorded as adjacent identical dates of the first date; time periods that are adjacent identical dates and are the same as the first time period are recorded as adjacent identical time periods of the first time period, where k1 is the number of days set;

[0030] Obtain the resource demand saturation of each adjacent time period, denoted as the adjacent demand saturation; calculate the mean XP and standard deviation XB of all adjacent demand saturations; calculate |AE-XP| / XB, denoted as the period deviation V2 of the first time period;

[0031] Calculate q1×V1+q2×V2 to mark the fluctuation index of the first period; repeatedly obtain the task volume index and fluctuation index for each period to obtain the task index data for each period, where q1 and q2 are the set weights.

[0032] Furthermore, classifying each time period based on its task indicator data and obtaining the corresponding indicator thresholds includes the following sub-steps:

[0033] Obtain the task volume indicators for all time periods of all first-category dates within the most recent k2 dates, and record them as the historical task volume set; calculate the mean WP and standard deviation WB of the historical task volume set; record WP+2×WB as the task volume threshold, where k2 is the set number of days;

[0034] For any given time period, if the corresponding task volume indicator is greater than the corresponding task volume threshold, it is marked as a high-volume time period; otherwise, it is marked as a low-volume time period.

[0035] Furthermore, classifying each time period based on its task indicator data and obtaining the corresponding indicator thresholds includes the following sub-steps:

[0036] Get all high-volume periods within the last k2 dates and denote them as recent high-volume periods. Denote any one of the recent high-volume periods as the first high-volume period. Get the periods in all recent high-volume periods that use the traditional scheduling method and denote them as traditional periods.

[0037] The task completion rate and resource utilization rate of the first high-volume period are recorded as AR and AU respectively in order; the average resource utilization rate of all traditional periods is calculated and recorded as the utilization baseline UB; if the task completion rate of the first high-volume period is less than e1, the scheduling adaptation score of the first high-volume period is recorded as 0; otherwise, AR×0.6+(AU-UB)×0.4 is calculated and recorded as the scheduling adaptation score FA of the first high-volume period.

[0038] Repeatedly obtain the scheduling adaptation score for all recent high-volume periods, and mark the recent high-volume periods with a scheduling adaptation score greater than 0.6 as reference high-volume periods.

[0039] Furthermore, classifying each time period based on its task indicator data and obtaining the corresponding indicator thresholds includes the following sub-steps:

[0040] If the first high volume period is a reference high volume period, calculate (FA-0.6)×10+1, and record it as the sample weight of the first high volume period; repeat to obtain the sample weight of all reference high volume periods;

[0041] Arrange all volatility indicators from the reference high-volume periods in ascending order to obtain a volatility indicator sequence. Label any volatility indicator in this sequence as BV. i , where i represents the position number;

[0042] Take BV i The two sides have k3 volatility indicators before and after, denoted as BV. i The nearest neighbor indicator, calculate all nearest neighbor indicators and BV i The sum of the sample weights, denoted as BH, is calculated as BH / (BV). i+k3 -BV i-k3 ), denoted as BV i The weighted density; where k3 is the number of settings;

[0043] The weighted density of all volatility indicators in the repeating volatility indicator sequence is arranged in the order of the volatility indicator sequence and denoted as the weighted density sequence. The two largest peaks are obtained and denoted as DM1 and DM2 respectively according to their order in the volatility indicator sequence.

[0044] The volatility indicators corresponding to DM1 and DM2 are denoted as VF1 and VF2 respectively in sequence; in the volatility indicator sequence, the volatility indicator with the lowest weighted density between VF1 and VF is taken and denoted as the volatility threshold.

[0045] Furthermore, controlling the scheduling method based on the current time period's task volume and fluctuation indicators, and according to the corresponding indicator thresholds, includes the following sub-steps:

[0046] For any given time period, if the corresponding task volume index is greater than the task volume threshold and the corresponding fluctuation index is greater than the task volume threshold, then it is determined that the reinforcement learning scheduling method is suitable; otherwise, it is determined that the traditional scheduling method is suitable.

[0047] For the current time period, the previous time period is designated as L2 time period, and the previous time period of L2 time period is designated as L1 time period; obtain the fluctuation index and task volume index of L2 time period, and obtain the suitable scheduling method for L2 time period, which is designated as L2 suitable method; repeatedly obtain the suitable method for L1, which is designated as L1 suitable method.

[0048] It also obtains the scheduling method used in the L2 time period, denoted as the L2 usage method; if the L2 usage method is the same as the L1 usage method, the L2 usage method is used to schedule cloud-native computing resources in the current time period; if the L2 usage method is different from the L1 usage method, the L2 usage method is used to schedule cloud-native computing resources in the current time period.

[0049] The beneficial effects of this invention are as follows: This invention establishes both traditional scheduling methods and reinforcement learning scheduling methods, and collects task volume and computing resource requirements for different time periods on different dates to obtain task-related data for each time period; it calculates task volume indicators and fluctuation indicators for each time period based on the task-related data for each time period to obtain task indicator data for each time period; it classifies each time period based on the task indicator data for each time period and obtains corresponding indicator thresholds; it controls the scheduling method based on the task volume indicators and fluctuation indicators for the current time period and the corresponding indicator thresholds; when scheduling computing resources in a batch processing task scheduling scenario, the threshold for switching scheduling methods can be dynamically set based on the historical task volume and scheduling conditions for each time period, thereby dynamically controlling the scheduling method and improving scheduling efficiency;

[0050] This invention constructs a volatility index using intra-segment volatility V1 and period deviation V2, reflecting the degree of fluctuation in task volume within a time period and the degree of deviation from historical norms, thus better reflecting the complexity of real batch processing scenarios. By combining task completion rate and resource utilization, a scheduling adaptation score is constructed, which is then used to screen out reference high-volume periods. The advantage lies in that the selected reference high-volume periods not only consider the amount of task but also the suitability of the scheduling method for that period, making the resulting threshold closer to reality. By calculating a weighted density for the reference high-volume periods and then finding the boundary interval based on the weighted density peak, the volatility threshold can be determined more accurately, identifying the dividing points in the volatility index distribution and making the obtained volatility threshold more accurate and reliable. Attached Figure Description

[0051] Figure 1 This is a flowchart illustrating the steps of the method of the present invention;

[0052] Figure 2 This is a flowchart of the task indicator data acquisition process of the present invention;

[0053] Figure 3 This is a flowchart of the fluctuation threshold acquisition process of the present invention;

[0054] Figure 4 This is a schematic diagram of the electronic device of the present invention. Detailed Implementation

[0055] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0056] Example 1, please refer to Figure 1As shown, this application provides a dynamic scheduling and control method for cloud-native computing resources based on reinforcement learning, including the following steps:

[0057] Step S1 involves establishing traditional scheduling methods and reinforcement learning scheduling methods, and collecting task volume and computational resource requirements for different time periods on different dates to obtain task-related data for each time period. Step S1 includes the following sub-steps:

[0058] Step S101: In the batch processing task scheduling scenario, for the scheduling of cloud-native computing resources, a corresponding scheduling method is established based on manually predefined static rules, heuristic algorithms, and fixed thresholds, denoted as the traditional scheduling method; that is, the traditional scheduling method based on the K8s native scheduling framework; and a corresponding scheduling method is established based on reinforcement learning, denoted as the reinforcement learning scheduling method; Batch processing tasks refer to tasks submitted in batches and executed in a concentrated manner, with typical characteristics of a large number of tasks and execution time that may be concentrated in certain time periods, commonly seen in scenarios such as report statistics, offline computing, and log analysis;

[0059] Step S102: Based on the cloud-native computing resources required daily, the date type is divided into a first type of date and a second type of date. The cloud-native computing resources required for the first type of date are less than those required for the second type of date. The task pressure on different dates is different. Some dates have low computing resource requirements, while others have high computing resource requirements. For example, in some scenarios, the computing resource requirements for weekdays and holidays are significantly different, so they can be divided into two categories. The division can be made according to the actual situation.

[0060] Step S103: Divide each day into multiple uniform time periods according to a first time interval, where the first time interval is t1; record any date as the first date, and record any time period of the first date as the first time period; in this embodiment, t1 = 1 hour, that is, each hour is a time period, which can be flexibly set, generally [0.5 hours, 2 hours].

[0061] Step S104: During the first time period, tasks currently in the queued and running states are synchronously acquired every t2 seconds. The CPU requests, memory requests, and GPU requests values ​​of all corresponding Pods are accumulated and recorded as CPU usage, memory usage, and GPU usage in sequence. After completion, the CPU usage sequence, memory usage sequence, and GPU usage sequence of the first time period are obtained and recorded as the resource demand information of the first time period. Here, t2 is the set duration; in this embodiment, t2 = 1 minute, which can be flexibly set, generally [10 seconds, 2 minutes]; if there is no GPU, the collection of GPU-related data and subsequent processing of GPU-related data can be ignored.

[0062] CPU usage, memory usage, and GPU usage do not represent the actual amount of resources used, but rather the amount of resources requested by the task. The requests value represents the task's intended resource needs and reflects the actual resource pressure exerted by the task at the current time. Since queued tasks are also waiting for resources, they represent the upcoming load pressure. Only by including both queued and running tasks in the statistics can the overall task volume at the current time be more completely reflected.

[0063] Step S105: In the first time period, every t2 time interval, obtain all nodes in the Ready and schedulable states at the current time in sync with the resource demand information, and accumulate the Allocatable values ​​of CPU, memory, and GPU for all nodes respectively. Record them in order as the total CPU, total memory, and total GPU, and obtain the total CPU, total memory, and total GPU sequence for the first time period, which are recorded as the total resource information for the first time period.

[0064] Resource requirements refer to how much a task needs, while total resources refer to how much computing resources are actually available for allocation; only by combining the two can the task load level be determined for the current period; the Allocatable value refers to the amount of resources in a node that are actually available for allocation by the scheduling system.

[0065] Step S106: In the first time period, collect the sum of the number of tasks in the queue and running state at the current moment every t2 time interval, and record it as the instantaneous total number of tasks. After completion, the instantaneous task quantity sequence is obtained. The number of tasks is also the most intuitive representation of the load. The number of queued and running tasks can show how busy the system is at this moment.

[0066] Step S107: Record the resource demand information, total resource information, and instantaneous task volume sequence of the first time period as the task-related data of the first time period; repeatedly collect the task-related data for each date and each time period;

[0067] Step S108: Within the first time period, obtain the scheduling method used in the first time period, i.e., whether the traditional scheduling method or the reinforcement learning scheduling method is used; and obtain the proportion of tasks completed on time within the first time period, which is recorded as the task completion rate of the first time period; the task completion rate directly reflects whether the task is completed within the specified time and is the core indicator for measuring whether the scheduling method is appropriate.

[0068] Step S109: During the first time period, collect the ratio of the sum of the actual usage of CPU, memory and GPU of all schedulable nodes to the corresponding total allocable resources every t2 time interval, and record them as CPU utilization, memory utilization and GPU utilization respectively; after completion, obtain the CPU utilization sequence, memory utilization sequence and GPU utilization sequence.

[0069] High demand does not necessarily mean high utilization, so we also need to look at how much is actually used. CPU utilization, memory utilization, and GPU utilization are used to measure whether the scheduling method is appropriate from the perspective of resource utilization.

[0070] Step S110: Calculate the average values ​​of the CPU utilization sequence, memory utilization sequence, and GPU utilization sequence respectively, and take the largest average value as the resource utilization rate of the first time period; repeatedly obtain the task completion rate and resource utilization rate for each time period of each date;

[0071] In practical implementation, traditional scheduling methods rely on static rules and fixed thresholds. Their advantages include definite decision-making, simple implementation, and low maintenance costs, making them suitable for low-load or stable-load scenarios. However, the static and fixed scheduling rules can only make local optimal judgments based on the current state, resulting in significant lag in elastic adjustments and extremely weak multi-objective balancing capabilities. Faced with sudden urgent tasks and complex multi-priority queues, they are prone to problems such as core task queuing timeouts, resource fragmentation, and insufficient cluster throughput. Reinforcement learning scheduling methods are more suitable for scenarios with high task volume and large fluctuations. Their advantage is the ability to dynamically adapt to load changes and handle sudden fluctuations more effectively. However, the deep neural network models used have extremely poor interpretability, making it difficult to quickly trace the root cause when task anomalies occur. They cannot meet the requirements of highly compliant scenarios, and their decisions are inherently random, leading to stability issues. Furthermore, they require massive amounts of interactive samples for model training and iteration, resulting in high maintenance costs.

[0072] Step S2 involves calculating the task volume and fluctuation indicators for each time period based on the task-related data for that time period, thus obtaining the task indicator data for each time period. Step S2 includes the following sub-steps:

[0073] For step S201, please refer to... Figure 2 As shown, for the first time period, outliers in the CPU usage sequence are filtered out and removed, and the average value of the remaining CPU usage sequence is calculated and recorded as the effective mean of the CPU usage sequence, denoted as YC; the method for filtering outliers can be flexibly selected according to the actual application scenario.

[0074] Step S202: Repeatedly obtain the effective mean values ​​of the memory usage sequence, GPU usage sequence, CPU total sequence, memory total sequence, and GPU total sequence, and record them as YN, YG, ZC, ZN, and ZG respectively in order;

[0075] Step S203: Calculate YC / ZC, YN / ZN, and YG / ZG respectively, and record them as the resource usage ratio. Record the largest resource usage ratio as the resource demand saturation AE of the first time period, and mark it as the task volume indicator. The resource demand saturation AE quantifies the actual pressure of the task on the computing resources at the current time. It is an indicator of the level of task volume. The scheduling is often not limited by the average resources, but by the most scarce resources. Therefore, the largest resource usage ratio is used as AE.

[0076] For example, in the first time period, YC=108 cores, YN=192G, ZC=240 cores, ZN=480G, and there is no GPU; then YC / ZC=108 / 240=0.45, YN / ZN=192 / 480=0.40, and the resource demand saturation AE=0.45=45%.

[0077] Step S204: Filter out outliers in the instantaneous task volume sequence and remove them. Calculate the average value RP, maximum value RA, and minimum value RB of the remaining instantaneous task volume sequence. Calculate (RA-RB) / RP, denoted as the intra-segment volatility V1 of the first period. Intra-segment volatility V1 captures sudden changes in the number of tasks within the current period by comparing the peak-to-trough difference in task volume with the average value. Essentially, it shows how drastic the change in task volume is relative to the average level within this period, and is a core indicator for measuring short-term sudden fluctuations in task volume.

[0078] Step S205: Dates within k1 days before the first date that have the same date type as the first date are recorded as adjacent identical dates of the first date; time periods that are the same as the first time period and are adjacent identical dates are recorded as adjacent identical time periods of the first time period, where k1 is the set number of days; in this embodiment, k1=40, which can be flexibly set, generally [30, 60];

[0079] Step S206: Obtain the resource demand saturation of each adjacent time period, denoted as the adjacent demand saturation; calculate the mean XP and standard deviation XB of all adjacent demand saturations; calculate |AE-XP| / XB, denoted as the period deviation V2 of the first time period; the period deviation V2 is essentially a judgment on whether the current time period deviates from the historical pattern by comparing the current task volume with the normal level of the same period in history, and is the core indicator for measuring the long-term fluctuation of the task volume.

[0080] Step S207: Calculate q1×V1+q2×V2 to mark the fluctuation index of the first time period; repeatedly obtain the task volume index and fluctuation index for each time period to obtain the task index data for each time period, where q1 and q2 are the set weights; in this embodiment, q1=0.3, q2=0.7. Different business scenarios have different levels of emphasis on short-term fluctuations and periodic anomalies, which can be flexibly set according to the actual application scenario.

[0081] For example, the average value RP of the remaining instantaneous task quantity sequence corresponding to the first period is 20, the maximum value RA is 38, and the minimum value RB is 12. Then the intra-period volatility V1 = (RA - RB) / RP = 1.3; the adjacent demand saturation corresponding to the first period is {42%, 44%, 43%, 41%}, then the mean XP = 42.5%, the standard deviation XB = 1.118; AE = 45%, then the cycle deviation V2 = |AE - XP| / XB = 2.236; then the volatility index is q1 × V1 + q2 × V2 = 0.7 × 2.236 + 0.3 × 1.118 = 1.955;

[0082] In practice, the comprehensive fluctuation index reflects both the degree of fluctuation in the workload during the period and the degree of deviation between the workload during the period and the historical norm. It takes into account both short-term fluctuations and historical deviations, thus avoiding the bias of judgment based on a single dimension.

[0083] Step S3 involves classifying each time period based on its task indicator data and obtaining the corresponding indicator thresholds. Step S3 includes the following sub-steps:

[0084] Step S301: Obtain the task volume indicators for all time periods of all first-category dates within the most recent k2 dates, and record them as the historical task volume set; calculate the mean WP and standard deviation WB of the historical task volume set; record WP+2×WB as the task volume threshold, where k2 is the set number of days; in this embodiment, k2=60, which can be flexibly set, generally [50, 100]; the task volume threshold can also be set by yourself.

[0085] Step S302: For any given time period, if the corresponding task volume index is greater than the corresponding task volume threshold, it is marked as a high-volume time period; otherwise, it is marked as a low-volume time period. High-volume time periods often mean greater resource pressure and stronger task competition, while low-volume time periods are usually more suitable for stable and low-cost traditional scheduling methods.

[0086] For step S303, please refer to... Figure 3 As shown, obtain all high-volume time periods within the most recent k2 dates, and denote them as recent high-volume time periods. Denote any recent high-volume time period as the first high-volume time period. Obtain the time periods in all recent high-volume time periods that use the traditional scheduling method, and denote them as traditional time periods.

[0087] Step S304: Record the task completion rate and resource utilization rate of the first high-volume period as AR and AU respectively; calculate the average resource utilization rate of all traditional periods and record it as the utilization baseline UB; UB provides a unified calculation benchmark for the scheduling adaptation score of all samples, ensuring that the scoring standard of all samples is consistent; if the task completion rate of the first high-volume period is less than e1, the scheduling adaptation score of the first high-volume period is recorded as 0; no matter how high the resource utilization rate is, as long as the task cannot be completed according to the standard, the scheduling method is completely unacceptable and is directly recorded as 0, that is, it is not used as a subsequent reference; otherwise, calculate AR×0.6+(AU-UB)×0.4 and record it as the scheduling adaptation score FA of the first high-volume period; where e1 is the set threshold, in this embodiment, e1=99%, which can be flexibly set according to actual needs; the scheduling adaptation score is used to determine whether the selected scheduling method within the period is appropriate.

[0088] Step S305: Repeatedly obtain the scheduling adaptation score of all recent high-volume periods, and mark the recent high-volume periods with a scheduling adaptation score greater than 0.6 as reference high-volume periods; that is, only retain periods with suitable scheduling methods; some recent high-volume periods may have poor task completion rate and unsatisfactory resource utilization, that is, the scheduling method used is not suitable; therefore, these samples without reference significance can be removed to avoid interference with the subsequent acquisition of fluctuation thresholds;

[0089] Step S306: If the first high volume period is a reference high volume period, calculate (FA-0.6)×10+1 and record it as the sample weight of the first high volume period; repeatedly obtain the sample weight of all reference high volume periods; give higher weights to periods with better scheduling performance, so that the final threshold is more biased towards periods with better scheduling performance, making the threshold more accurate and reliable.

[0090] For example, in the first high-volume period, AR=100%, AU=56%, and UB=0.55, then the scheduling adaptation score FA=AR×0.6+(AU-UB)×0.4=0.604; and the sample weight is (A-0.6)×10+1=1.04.

[0091] Step S307: Arrange all the volatility indicators of the reference high-volume periods in ascending order to obtain a volatility indicator sequence. Mark any volatility indicator in the volatility indicator sequence as BV. i Where i represents the position number; the smaller the volatility index, the lower the volatility; the larger the volatility index, the higher the volatility; traditional scheduling is adapted to low volatility, while reinforcement learning scheduling is adapted to high volatility.

[0092] Step S308, take BV i The two sides have k3 volatility indicators before and after, denoted as BV. i The nearest neighbor indicator, calculate all nearest neighbor indicators and BV iThe sum of the sample weights, denoted as BH, is calculated as BH / (BV). i+k3 -BV i-k3 ), denoted as BV i The weighted density is set, where k3 is the number of samples. In this embodiment, k3=5, which can be set flexibly, generally [5, 10]. The larger the weighted density, the denser and more concentrated the samples with excellent scheduling effect are near the fluctuation index value, which is the core stable comfort zone of the corresponding scheduling method.

[0093] Step S309: Repeat the weighted density of all volatility indices in the volatility index sequence, arrange them in the order of the volatility index sequence, and denot them as the weighted density sequence. Obtain the two largest peaks and denot them as DM1 and DM2 respectively according to their order in the volatility index sequence. That is, DM1 is on the left side of the volatility index sequence, corresponding to the low volatility region, which is suitable for traditional scheduling methods; DM2 is on the right side of the volatility index sequence, corresponding to the high volatility region, which is suitable for reinforcement learning scheduling methods.

[0094] Step S310: Record the volatility indices corresponding to DM1 and DM2 as VF1 and VF2 respectively in sequence; in the volatility index sequence, take the volatility index with the lowest weighted density between VF1 and VF and record it as the volatility threshold.

[0095] For example, if the volatility index sequence is {0.4, 0.7, 0.9, 1.3, 1.9, 2.1, 2.3, 2.5}, and the corresponding sample weights are {1.04, 1.05, 1.04, 1.25, 1.36, 1.52, 1.41, 1.24}, then let k3=2, calculate the corresponding weighted density, and obtain the weighted density sequence {6.26, 4.87, 3.83, 4.44, 4.70, 5.65, 9.22, 10.43}, then DM1=6.26, DM2=10.43; therefore, the volatility index with the lowest weighted density between 0.4 and 2.5 is 0.9, and the volatility threshold is 0.9.

[0096] In practice, the point with the lowest weighted density indicates that there are very few samples of both scheduling methods performing well near the fluctuation index, meaning that the performance of both scheduling methods is only average. Switching at this point minimizes the risk of switching.

[0097] Step S4 involves controlling the scheduling method based on the current time period's task volume and fluctuation indicators, and according to the corresponding indicator thresholds. Step S4 includes the following sub-steps:

[0098] Step S401: For any given time period, if the corresponding task volume index is greater than the task volume threshold and the corresponding fluctuation index is greater than the task volume threshold, then it is determined that the reinforcement learning scheduling method is suitable; otherwise, it is determined that the traditional scheduling method is suitable. That is, the reinforcement learning scheduling method is only used when the task volume is high and the task volume fluctuation is large.

[0099] Step S402: For the current time period, the previous time period is denoted as L2 time period, and the previous time period of L2 time period is denoted as L1 time period; obtain the fluctuation index and task volume index of L2 time period, and obtain the scheduling method suitable for L2 time period, denoted as L2 suitable method; repeatedly obtain the L1 suitable method, denoted as L1 suitable method.

[0100] Step S403: Obtain the scheduling method used in the L2 time period, denoted as the L2 usage method; if the L2 usage method is the same as the L1 usage method, then use the L2 usage method to schedule cloud-native computing resources in the current time period; if the L2 usage method is different from the L1 usage method, then use the L2 usage method to schedule cloud-native computing resources in the current time period; reduce unnecessary switching and avoid frequent switching of scheduling methods;

[0101] In practical implementation, reinforcement learning scheduling methods are suitable for complex batch processing scenarios with high workloads and high volatility. However, if only reinforcement learning scheduling methods are used, issues such as cost-benefit imbalance and stability risks may arise. During periods of low workload and stability, traditional scheduling has the advantages of no maintenance costs and 100% decision-making certainty and controllability, which fully cover business needs. In contrast, reinforcement learning scheduling requires continuous consumption of computing power to run inference services and investment of manpower to maintain and iterate models, but it cannot bring any substantial optimization benefits, resulting in resource waste. At the same time, the characteristics of reinforcement learning cannot eliminate risks such as decision oscillations and illegal scheduling, and continuous use will pose a risk of business interruption. Therefore, in most cases, it is simply not possible to support the use of reinforcement learning scheduling around the clock.

[0102] Example 2, please refer to Figure 4 As shown, Figure 4A schematic diagram of an electronic device is provided, which may include a processor, a communication interface, a memory, and a communication bus. The processor, communication interface, and memory communicate with each other via the communication bus. The memory stores computer-readable instructions, and the processor can call these instructions. When the processor executes a computer-readable instruction, it performs steps similar to those in a reinforcement learning-based dynamic scheduling and control method for cloud-native computing resources to achieve the following functions: establishing a traditional scheduling method and a reinforcement learning scheduling method; collecting task volume and computing resource requirements for different time periods on different dates to obtain task-related data for each time period; calculating task volume indicators and fluctuation indicators for each time period based on the task-related data to obtain task indicator data for each time period; classifying each time period based on the task indicator data and obtaining corresponding indicator thresholds; and controlling the scheduling method based on the current time period's task volume indicators and fluctuation indicators, and the corresponding indicator thresholds.

[0103] Furthermore, when the logical instructions in the aforementioned memory can be implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0104] Example 3: This application also provides a computer-readable storage medium storing a computer program. When executed by a processor, the computer program performs the steps of a reinforcement learning-based dynamic scheduling and control method for cloud-native computing resources to achieve the following functions: establishing a traditional scheduling method and a reinforcement learning scheduling method, and collecting task volume and computing resource requirements for different time periods on different dates to obtain task-related data for each time period; calculating task volume indicators and fluctuation indicators for each time period based on the task-related data for each time period to obtain task indicator data for each time period; classifying each time period based on the task indicator data for each time period and obtaining corresponding indicator thresholds; controlling the scheduling method based on the task volume indicators and fluctuation indicators for the current time period and the corresponding indicator thresholds.

[0105] Based on the above description of the embodiments, the embodiments of the present invention can be provided as methods, systems, or computer program products. Based on this understanding, the above technical solutions, in essence or in terms of their contribution to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or certain parts of the embodiments.

[0106] In the embodiments provided in this application, it should be understood that the disclosed system or method can be implemented in other ways. The embodiments described above are merely illustrative. For example, the division of modules or units is only a logical functional division, and there may be other division methods in actual implementation. Furthermore, multiple modules or units may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the coupling or direct coupling or communication connection shown or discussed may be through some communication interfaces. The indirect coupling or communication connection between systems, modules, and units may be electrical, mechanical, or other forms.

[0107] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A dynamic scheduling and control method for cloud-native computing resources based on reinforcement learning, characterized in that, Includes the following steps: We established traditional scheduling methods and reinforcement learning scheduling methods, and collected task volume and computing resource requirements for different time periods on different dates to obtain task-related data for each time period. Calculate the task volume index and fluctuation index for each time period based on the task-related data for each time period, and obtain the task index data for each time period. The task indicator data for each time period are categorized, and the corresponding indicator thresholds are obtained. The scheduling method is controlled based on the current task volume and fluctuation indicators, and the corresponding indicator thresholds.

2. A dynamic scheduling and control method for cloud-native computing resources based on reinforcement learning as described in claim 1, characterized in that, Establish traditional scheduling methods and reinforcement learning scheduling methods, and collect task volume and computational resource requirements for different time periods on different dates to obtain task-related data for each time period, including the following sub-steps: In batch task scheduling scenarios, for the scheduling of cloud-native computing resources, there are two scheduling methods: one based on manually predefined static rules, heuristic algorithms, and fixed thresholds, which are called traditional scheduling methods; and another based on reinforcement learning, which are called reinforcement learning scheduling methods. Based on the daily cloud-native computing resources required, the date type is divided into Category I dates and Category II dates; Category I dates require fewer cloud-native computing resources than Category II dates. Each day is divided into multiple uniform time periods by a first time interval, where the first time interval is t1; any date is designated as the first date, and any time period of the first date is designated as the first time period.

3. The method for dynamic scheduling and control of cloud-native computing resources based on reinforcement learning according to claim 2, characterized in that, Establishing traditional scheduling methods and reinforcement learning scheduling methods, and collecting task volume and computational resource requirements for different time periods on different dates to obtain task-related data for each time period also includes the following sub-steps: Within the first time period, tasks currently in the queue and running states are synchronously retrieved every t2 seconds. The CPU, memory, and GPU requests values ​​of all corresponding Pods are accumulated and recorded in order as CPU usage, memory usage, and GPU usage, respectively. After completion, the CPU usage sequence, memory usage sequence, and GPU usage sequence of the first time period are obtained and recorded as the resource demand information of the first time period; where t2 is the set duration. Within the first time period, every t2 time interval, the system synchronously retrieves all nodes in the Ready and schedulable states at the current moment with the resource demand information. It then accumulates the Allocatable values ​​of CPU, memory, and GPU for all nodes, and records them in order as the total CPU, total memory, and total GPU. After completion, the system obtains the total CPU, total memory, and total GPU sequences for the first time period, which are recorded as the total resource information for the first time period.

4. The method for dynamic scheduling and control of cloud-native computing resources based on reinforcement learning according to claim 3, characterized in that, Establishing traditional scheduling methods and reinforcement learning scheduling methods, and collecting task volume and computational resource requirements for different time periods on different dates to obtain task-related data for each time period also includes the following sub-steps: Within the first time period, the sum of the number of tasks in the queue and running state at the current moment is collected every t2 time interval and recorded as the instantaneous total number of tasks. After completion, the instantaneous task quantity sequence is obtained. The resource demand information, total resource information, and instantaneous task volume sequence for the first time period are recorded as the task-related data for the first time period; Repeatedly collect task-related data for each date and time period; Within the first time period, obtain the scheduling method used in the first time period, and obtain the proportion of tasks completed on time within the first time period, which is recorded as the task completion rate of the first time period. Within the first time period, the sum of the actual usage of CPU, memory, and GPU of all schedulable nodes is collected every t2 time interval as a percentage of the corresponding total allocable resources, and recorded as CPU utilization, memory utilization, and GPU utilization, respectively; after completion, CPU utilization sequence, memory utilization sequence, and GPU utilization sequence are obtained. Calculate the mean values ​​of the CPU utilization sequence, memory utilization sequence, and GPU utilization sequence respectively, and take the largest mean value as the resource utilization rate of the first time period; repeatedly obtain the task completion rate and resource utilization rate for each time period of each date.

5. The cloud-native computing resource dynamic scheduling and control method based on reinforcement learning according to claim 4, characterized in that, Calculate the task volume and fluctuation indicators for each time period based on the task-related data for each time period. The process for obtaining the task indicator data for each time period includes the following sub-steps: For the first time period, outliers in the CPU usage sequence are filtered out and removed, and the average value of the remaining CPU usage sequence is calculated and recorded as the effective mean of the CPU usage sequence, denoted as YC. Repeatedly obtain the effective mean values ​​of the memory usage sequence, GPU usage sequence, CPU total usage sequence, memory total usage sequence, and GPU total usage sequence, and denote them as YN, YG, ZC, ZN, and ZG respectively in order; Calculate YC / ZC, YN / ZN, and YG / ZG separately, and record them as the resource usage ratio. Record the largest resource usage ratio as the resource demand saturation AE of the first period, and mark it as the task volume indicator.

6. The method for dynamic scheduling and control of cloud-native computing resources based on reinforcement learning according to claim 5, characterized in that, Calculating the task volume and fluctuation indicators for each time period based on task-related data for each time period, and obtaining the task indicator data for each time period, also includes the following sub-steps: Filter out outliers in the instantaneous task volume sequence and remove them. Calculate the average value RP, maximum value RA, and minimum value RB of the remaining instantaneous task volume sequence. Calculate (RA-RB) / RP, and record it as the intra-segment volatility V1 of the first time period. Dates within k1 days before the first date that have the same date type as the first date are recorded as adjacent identical dates of the first date; time periods that are adjacent identical dates and are the same as the first time period are recorded as adjacent identical time periods of the first time period, where k1 is the number of days set; Obtain the resource demand saturation of each adjacent time period, denoted as the adjacent demand saturation; calculate the mean XP and standard deviation XB of all adjacent demand saturations; calculate |AE-XP| / XB, denoted as the period deviation V2 of the first time period; Calculate q1×V1+q2×V2 to mark the fluctuation index of the first period; repeatedly obtain the task volume index and fluctuation index for each period to obtain the task index data for each period, where q1 and q2 are the set weights.

7. The method for dynamic scheduling and control of cloud-native computing resources based on reinforcement learning according to claim 6, characterized in that, The process of categorizing task metrics data for each time period and obtaining corresponding metric thresholds includes the following sub-steps: Obtain the task volume indicators for all time periods of all first-category dates within the most recent k2 dates, and record them as the historical task volume set; calculate the mean WP and standard deviation WB of the historical task volume set; record WP+2×WB as the task volume threshold, where k2 is the set number of days; For any given time period, if the corresponding task volume indicator is greater than the corresponding task volume threshold, it is marked as a high-volume time period; otherwise, it is marked as a low-volume time period.

8. The method for dynamic scheduling and control of cloud-native computing resources based on reinforcement learning according to claim 7, characterized in that, The process of categorizing task metrics data for each time period and obtaining corresponding metric thresholds also includes the following sub-steps: Get all high-volume periods within the last k2 dates and denote them as recent high-volume periods. Denote any one of the recent high-volume periods as the first high-volume period. Get the periods in all recent high-volume periods that use the traditional scheduling method and denote them as traditional periods. The task completion rate and resource utilization rate of the first high-volume period are recorded as AR and AU respectively in order; the average resource utilization rate of all traditional periods is calculated and recorded as the utilization baseline UB; if the task completion rate of the first high-volume period is less than e1, the scheduling adaptation score of the first high-volume period is recorded as 0; otherwise, AR×0.6+(AU-UB)×0.4 is calculated and recorded as the scheduling adaptation score FA of the first high-volume period. Repeatedly obtain the scheduling adaptation score for all recent high-volume periods, and mark the recent high-volume periods with a scheduling adaptation score greater than 0.6 as reference high-volume periods.

9. The method for dynamic scheduling and control of cloud-native computing resources based on reinforcement learning according to claim 8, characterized in that, The process of categorizing task metrics data for each time period and obtaining corresponding metric thresholds also includes the following sub-steps: If the first high-volume period is the reference high-volume period, calculate (FA-0.6)×10+1, and record it as the sample weight of the first high-volume period; Repeatedly obtain the sample weights for all reference high-volume periods; Arrange all volatility indicators from the reference high-volume periods in ascending order to obtain a volatility indicator sequence. Label any volatility indicator in this sequence as BV. i , where i represents the position number; Take BV i The two sides have k3 volatility indicators before and after, denoted as BV. i The nearest neighbor indicator, calculate all nearest neighbor indicators and BV i The sum of the sample weights, denoted as BH, is calculated as BH / (BV). i+k3 -BV i-k3 ), denoted as BV i The weighted density; where k3 is the number of settings; The weighted density of all volatility indicators in the repeating volatility indicator sequence is arranged in the order of the volatility indicator sequence and denoted as the weighted density sequence. The two largest peaks are obtained and denoted as DM1 and DM2 respectively according to their order in the volatility indicator sequence. The volatility indicators corresponding to DM1 and DM2 are denoted as VF1 and VF2 respectively in sequence; in the volatility indicator sequence, the volatility indicator with the lowest weighted density between VF1 and VF is taken and denoted as the volatility threshold.

10. A dynamic scheduling and control method for cloud-native computing resources based on reinforcement learning according to claim 9, characterized in that, Based on the current time period's task volume and fluctuation metrics, and using the corresponding metric thresholds, the scheduling method is controlled through the following sub-steps: For any given time period, if the corresponding task volume index is greater than the task volume threshold and the corresponding fluctuation index is greater than the task volume threshold, then it is determined that the reinforcement learning scheduling method is suitable; otherwise, it is determined that the traditional scheduling method is suitable. For the current time period, the previous time period is denoted as L2 time period, and the previous time period of L2 time period is denoted as L1 time period; Obtain the fluctuation index and task volume index for the L2 time period, and obtain the suitable scheduling method for the L2 time period, denoted as the L2 suitable method; repeatedly obtain the suitable method for L1, denoted as the L1 suitable method. And obtain the scheduling method used in the L2 time period, denoted as the L2 usage method; if the L2 fitting method is the same as the L1 fitting method, then use the L2 fitting method to schedule cloud-native computing resources in the current time period; If the L2 fitting method and the L1 fitting method are different, the L2 fitting method is used to schedule cloud-native computing resources in the current time period.

Images