Scheduling prompting method, system and equipment for timing tasks and medium
By obtaining the CRON expression and historical resource consumption data of the timing tasks, the current resource consumption is estimated and the task concurrency is combined, the container resources are inferred and the scheduling prompts are performed, which solves the problem of insufficient resources during peak task concurrency, and improves the task execution success rate and container reliability.
Patent Information
- Application Number
- CN202510335592.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-03-20
- Publication Date
- 2025-08-01
AI Technical Summary
In the prior art, the background distributed scheduling system has insufficient resources when the task is concurrently concurrent, resulting in failure of task scheduling, delayed execution or interruption, affecting the reliability and business continuity of the container.
By obtaining the CRON expression data of the timing task and the historical resource consumption data, estimate the current resource consumption, and combine the task concurrency within the current unit time period to infer whether the container has resource limitations and perform scheduling prompts to avoid resource shortages.
It improves the execution success rate of timed tasks, optimizes the utilization of container resources, solves the problem of insufficient resources during peak task concurrency, and ensures the reliability and business continuity of containers.
Smart Images
Figure CN120407099A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of task scheduling, and in particular to a scheduling prompt method, system, device and medium for timing tasks. Background Art
[0002] In the related art of backend distributed scheduling, for long-term backend task scheduling scenarios, pre-set scheduling policies are often used. These scheduling policies are generally configured based on static rules such as fixed time windows, task priorities, or resource quotas. However, in practical applications, these pre-set scheduling policies are limited by task concurrency peaks and resource contention.
[0003] Because task scheduling policies are statically configured in advance, a large number of tasks may be triggered simultaneously within the same time period. For example, common business system scenarios such as end-of-day batch processing and monthly report generation often involve the launch of numerous scheduled tasks at specific times (such as midnight). This peak in task concurrency can lead to a surge in container platform resource demand, exceeding the platform's capacity.
[0004] When a large number of tasks are triggered simultaneously, they trigger intense competition for computing resources (CPU, memory), storage resources, and network resources. Especially in a containerized environment, where resources are allocated and isolated on demand, insufficient resources can lead to task scheduling failures, task execution delays, or interruptions, impacting container reliability and business continuity. Summary of the Invention
[0005] In response to the deficiencies in the prior art, the present invention provides a scheduling prompt method, system, device and medium for timed tasks, which comprehensively consider CRON expression data, historical resource consumption data and task concurrency in the current unit time period to infer whether the container is resource-constrained. When the container resources are limited, prompts are given and timed tasks are scheduled to better utilize container resources and improve the success rate of timed task execution, thus solving the technical problem that the container may have insufficient resources during the peak of task concurrency.
[0006] The first object of the present invention is to provide a scheduling prompt method for timed tasks, comprising:
[0007] Obtain CRON expression data of the scheduled task and historical resource consumption data of the container, and estimate the current resource consumption data of the scheduled task based on the CRON expression data and the historical resource consumption data;
[0008] Query the total number of evaluation tasks within the current unit time period based on the registered task execution information, where the registered task execution information includes task quantity information and execution plan information, and the current unit time period is obtained based on the start time and time length of the timed task;
[0009] If it is estimated that the current resource consumption data of the timed task is greater than the preset maximum resource consumption, or if it is queried that the total number of evaluation tasks within the current unit time period is greater than the preset maximum task concurrency, then it is prompted that the container has resource constraints and the timed task is scheduled.
[0010] As a further improvement of the present invention, the estimation of the current resource consumption data of the timed task includes:
[0011] Calculate the historical average resource consumption data of the timed task based on the CRON expression data and the historical resource consumption data;
[0012] Estimate the current resource consumption data of the timed task based on the historical average resource consumption data of the timed task.
[0013] As a further improvement of the present invention, the calculation formula for the historical average resource consumption data of the timed task is:
[0014]
[0015] Wherein, is the historical average resource consumption data of the timed task, k m is the historical resource consumption data of the most recent timed task, k m-i is the historical resource consumption data of the i-th timed task before the most recent timed task, and n is the historical execution times of the timed task before the most recent timed task.
[0016] As a further improvement of the present invention, the calculation formula for the current resource consumption data of the timed task is:
[0017]
[0018] Wherein, is the current resource consumption data of the timed task, is the historical average resource consumption data, k m is the historical resource consumption data of the most recent timed task, k m-n is the historical resource consumption data of the n-th timed task before the most recent timed task, and ε is the minimum resource consumption data required for a single task.
[0019] As a further improvement of the present invention, the situation where the currently predicted resource consumption data of the timing task is greater than the preset maximum resource consumption is specifically as follows:
[0020]
[0021] Among them, is the currently predicted resource consumption data of the timing task, K max is the preset maximum resource consumption, and ε k is the resource threshold coefficient.
[0022] As a further improvement of the present invention, the situation where the total number of evaluation tasks in the current unit time period is greater than the preset maximum task concurrency is specifically as follows:
[0023] Tcount>ε t *T max
[0024] Among them, Tcount is the total number of evaluation tasks in the current unit time period, and T max is the preset maximum task concurrency, and ε t is the task threshold coefficient.
[0025] As a further improvement of the present invention, the scheduling of the timing task includes:
[0026] Moving the timing task in the current unit time period forward or backward so that the currently predicted resource consumption data of the timing task is less than or equal to the preset maximum resource consumption, and the total number of evaluation tasks in the current unit time period is less than or equal to the preset maximum task concurrency.
[0027] The second object of the present invention is to provide a scheduling prompt system for timing tasks, which is used to implement the above scheduling prompt method, including:
[0028] A resource consumption prediction module, which is used to obtain the CRON expression data of the timing task and the historical resource consumption data of the container, and predict the currently predicted resource consumption data of the timing task based on the CRON expression data and the historical resource consumption data
[0029] An evaluation task total query module, which queries the total number of evaluation tasks in the current unit time period based on the registered task execution information. Among them, the registered task execution information includes task quantity information and execution plan information, and the current unit time period is obtained based on the start time and time length of the timing task;
[0030] A scheduling prompt module is used to determine whether the container has resource constraints, and when the container has resources, it performs resource constraint prompts and schedules timed tasks; among them, determining whether the container has resource constraints includes: if it is estimated that the current resource consumption data of the timed task is greater than the preset maximum resource consumption, or if it is queried that the total number of evaluation tasks within the current unit time period is greater than the preset maximum task concurrency, then it is determined that the container has resource constraints.
[0031] A third object of the present invention is to provide an electronic device, including at least one processing unit and at least one storage unit, wherein the storage unit stores a computer program, and when the program is executed by the processing unit, the processing unit executes the above scheduling prompt method.
[0032] A fourth object of the present invention is to provide a storage medium, which stores a computer program executable by an electronic device, and when the program runs on the electronic device, the electronic device executes the above scheduling prompt method.
[0033] Compared with the prior art, the beneficial effects of the present invention are:
[0034] In the configuration stage of timed tasks, comprehensively consider CRON expression data, historical resource consumption data, and task concurrency in the current unit time period to infer whether the container has resource constraints. When the container resources are restricted, prompt and schedule the timed tasks to better utilize the container resources and improve the success rate of timed task execution, solving the technical problem that the container may have insufficient resources during the peak of task concurrency. Description of the Drawings
[0035] Figure 1 It is a flowchart of the scheduling prompt method;
[0036] Figure 2 It is a schematic flowchart of scheduling timed tasks;
[0037] Figure 3 It is a schematic diagram of scheduling prompts;
[0038] Figure 4 It is a schematic structural diagram of the scheduling prompt system. Detailed Embodiments
[0039] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Apparently, the described embodiments are some but not all of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts fall within the scope of protection of the present invention.
[0040] The following further describes the present invention in detail with reference to the accompanying drawings:
[0041] Please refer to Figure 1 , this embodiment provides a scheduling prompt method for timed tasks, including:
[0042] Obtain the CRON expression data of the timed task and the historical resource consumption data of the container, and estimate the current resource consumption data of the timed task based on the CRON expression data and the historical resource consumption data.
[0043] Timed tasks can be long-timed tasks or short-timed tasks. Among them, long-timed tasks generally refer to timed tasks that are executed only once after a very long period of time. The time period is default greater than one day, such as tasks executed once a day, or even tasks executed once a week, etc. Short-timed tasks are tasks executed multiple times a day.
[0044] CRON expression data can be a formatted string used to define the task scheduling time, which allows tasks to automatically execute scheduled work at specified times. By configuring the CRON expression, the container can automatically execute tasks according to the established plan without manual intervention.
[0045] Query the total number of evaluated tasks in the current unit time period based on the registered task execution information, where the registered task execution information includes task quantity information and execution plan information, and the current unit time period is obtained based on the start time and time length of the timed task.
[0046] Registered task execution information refers to all the registered tasks and their related execution details in the container, mainly including the following two types of information:
[0047] Task quantity information: Records the total number of registered tasks in the current container and the status of these tasks (such as waiting to be executed, in execution, completed, etc.). Execution plan information: Describes the scheduling rules and execution arrangements of each task, usually including the start time of the task, execution frequency (such as one-time task, periodic task), execution duration, resource requirements (such as CPU, memory), etc. These information are usually defined through CRON expressions or other scheduling configurations.
[0048] The current unit time period refers to a time interval divided based on the characteristics of timed tasks, which is determined based on the start time and time length of the timed tasks. The start time is based on the planned start time of the task and is usually determined by the scheduling rules of the task (such as CRON expressions). The time length is set according to the execution cycle or execution duration of the task, such as a time period per hour, per day, or per week.
[0049] Specifically, if a certain task is planned to start at 10:00 am every day and its execution cycle is 1 hour, then the current unit time period may be from 10:00 to 11:00. The container will evaluate the total amount of tasks within this time period to determine whether the resources are sufficient and whether the scheduling strategy needs to be adjusted.
[0050] Specifically, before configuring the timed task, query the task concurrency within the current unit time period, including the steps:
[0051] Obtain all the registered task execution information in the container. The task execution information includes: the number of tasks taskNum, and the execution plans of the task in the last n times NPL(d1, d2, d3, d4,... d n ); Using the unit time period as the filtering condition, query the total number of tasks within the current unit time period, that is, evaluate the total amount of tasks. The attributes of the unit time period include: start time Dstart, time length Dlength (in seconds).
[0052] The code for querying and evaluating the total amount of tasks is as follows:
[0053]
[0054] Among them, Tcount is the evaluated total amount of tasks.
[0055] If it is estimated that the current resource consumption data of the timed task is greater than the preset maximum resource consumption, or if it is queried that the evaluated total amount of tasks within the current unit time period is greater than the preset maximum task concurrency, then it is prompted that the container has resource constraints and the timed task is scheduled.
[0056] Scheduling, that is, task scheduling, is the process of arranging and executing task code at a specific time point or triggered by a specific action. This process usually involves two core components: one is the "scheduler", which is responsible for sending task execution notifications according to the predetermined plan or trigger conditions; the other is the "executor", which is responsible for specifically executing the task operations. Through the collaborative work of the scheduler and the executor, task scheduling realizes the functions of automation and on-demand execution.
[0057] Specifically:
[0058] When it is estimated that the current resource consumption data is greater than the preset maximum resource consumption, it is prompted that the container has resource constraints and the timing tasks are scheduled. The specific judgment conditions are as follows:
[0059]
[0060] Among them, is the current resource consumption data of the timing task, K max is the preset maximum resource consumption, ε k is the resource threshold coefficient, and its value is less than 1.
[0061] Or when it is queried that the total amount of evaluation tasks is greater than the preset maximum task concurrency, it is prompted that the container has resource constraints and the timing tasks are scheduled. The specific judgment conditions are as follows:
[0062] Tcount>ε t *T max
[0063] Among them, Tcount is the total amount of evaluation tasks in the current unit time period, T max is the preset maximum task concurrency, ε t is the task threshold coefficient, and its value is less than 1.
[0064] When the container has resource constraints, the timing tasks are scheduled: move the timing tasks in the current unit time period forward or backward so that the current resource consumption data of the timing tasks is less than or equal to the preset maximum resource consumption, and the total amount of evaluation tasks in the current unit time period is less than or equal to the preset maximum task concurrency.
[0065] The method provided in this embodiment comprehensively considers the CRON expression data, historical resource consumption data, and task concurrency in the current unit time period during the configuration stage of the timing task to infer whether the container has resource constraints. When the container has resource constraints, a prompt is given and the timing task is scheduled to better utilize the container resources and improve the success rate of the timing task execution, solving the technical problem that the container may have insufficient resources during the peak of task concurrency.
[0066] Further, the estimation of the current resource consumption data of the timing task includes:
[0067] Calculating the historical average resource consumption data of the timing task based on the CRON expression data and the historical resource consumption data;
[0068] Specifically, according to the CRON expression of the timing task, deduce its next n execution plans, and record the next n execution plans of a certain task as NPL(d1,d2,d3,d4,...dn );
[0069] Sampling is performed in unit time periods to collect historical data on the resource consumption of the container, and the historical data samples are recorded as (k m , k m-1 , k m-2 , k m-3 ,..., k m-n );
[0070] The calculation formula for the historical average resource consumption data of the timing type tasks is as follows:
[0071]
[0072] Among them, is the historical average resource consumption data of the timing type tasks, k m is the historical resource consumption data of the most recent timing type task, k m-i is the historical resource consumption data of the i-th timing type task before the most recent timing type task, and n is the historical execution times of the timing type tasks before the most recent timing type task.
[0073] Based on the historical average resource consumption data of the timing type tasks, the current resource consumption data of the timing type tasks is estimated. The calculation formula for the current resource consumption data of the timing type tasks is as follows:
[0074]
[0075] Among them, is the current resource consumption data of the timing type tasks, is the historical average resource consumption data, k m is the historical resource consumption data of the most recent timing type task, k m-n is the historical resource consumption data of the n-th timing type task before the most recent timing type task, and ε is the minimum resource consumption data required for a single task, which can be configured according to the actual situation.
[0076] Furthermore, if it is estimated that the current resource consumption data of the timing type tasks is less than or equal to the preset maximum resource consumption, and it is queried that the total number of evaluation tasks in the current unit time period is less than or equal to the preset maximum task concurrency, it is prompted that there is no resource limitation for the container.
[0077] Specifically, when the following two conditions are met simultaneously, that is, the resource consumption condition and the task concurrency condition, it can be determined that there is no resource limitation for the current container:
[0078] Resource consumption condition: Through real-time monitoring and data analysis, it is estimated that the current resource consumption data (including key resources such as CPU, memory, and storage) does not exceed the preset maximum resource threshold, which means that there is still enough margin in the resource pool of the container platform to support the operation of more tasks.
[0079] The resource consumption condition expression is:
[0080]
[0081] Among them, is the current resource consumption data of the timed task type, K max is the preset maximum resource consumption, ε k is the resource threshold coefficient, and its value is less than 1.
[0082] Task concurrency condition: Through query, it is evaluated that the total number of tasks to be executed currently does not exceed the preset maximum task concurrency, indicating that the task scheduling queue of the container is not saturated and can accommodate the addition of more tasks.
[0083] The task concurrency condition expression is:
[0084] Tcount>ε t *T max
[0085] Among them, Tcount is the total number of evaluated tasks in the current unit time period, T max is the preset maximum task concurrency, ε t is the task threshold coefficient, and its value is less than 1.
[0086] Only when the above two conditions are met simultaneously will it be prompted that there is no resource constraint situation in the container. This dual verification mechanism can more comprehensively evaluate the load status of the container and avoid misjudgment that may be caused by a single indicator evaluation. For example, even if the resource consumption does not reach the upper limit, but if the task concurrency has exceeded the preset threshold, the container may still experience performance degradation or task backlog due to excessive task scheduling pressure. Conversely, if the task concurrency does not reach the upper limit but the resource consumption is already close to the peak, it may also lead to resource competition and task execution failure. Therefore, only when the conditions of both resource consumption and task concurrency are met can it be ensured that the container is in a stable and reliable running state.
[0087] Furthermore, consider an extreme scenario: at the same moment, all configurators are configuring jobs and the CRON has the same policy, which may lead to the problem that the scheduling policy prediction of this solution fails. To avoid the above problem, considering the execution plan of timed tasks, the maximum number of tasks allowed to be executed within a unit time period can be set to better utilize container resources and prevent resource shortages or surpluses. At the same time, the operation and maintenance workload of the container can be reduced to more clearly understand the task execution status and resource consumption.
[0088] Furthermore, in practical applications, there may be a large error in the prediction result due to external factors. To reduce the above error, external factors (such as manual task execution, abnormal interruption or jamming of certain tasks) should be shielded as much as possible, and other indicators (such as the number of manual task executions, the number of automatic task executions, whether the task is jammed, etc.) should be comprehensively considered to improve the accuracy of prediction.
[0089] Furthermore, in practical applications, for short timed tasks, since the task execution plan is relatively frequent, it may be impossible to predict whether the resources for the nth future execution will be limited. To reduce the above error, after each task execution ends, the background immediately evaluates whether resource limitations may occur according to the next execution time and the scheduling hint algorithm provided in this embodiment.
[0090] Figure 3 It is a schematic diagram of the scheduling hint for timed tasks. When the container has resource limitations, it prompts the scheduling policy for task configurators. Further, after the configurator schedules the timed tasks according to the scheduling policy, the scheduling hint method provided in this embodiment can evaluate whether resource limitations will occur in the next execution of the timed tasks and prompt the configurator according to the evaluation.
[0091] Furthermore, it can also inform the configurator in the form of a switch item whether to accept the option of delaying the execution time of the timed tasks in the scheduling policy. If the option is selected, then after each task execution is completed, the next execution time of the task is automatically calculated, and then according to the scheduling hint algorithm provided in this embodiment, the container resource usage is predicted. The implementation method is as Figure 2 shown, and the specific implementation steps are as follows:
[0092] Step 1: After the task is executed this time, calculate its next execution plan;
[0093] Step 2: Infer the resources and task concurrency of the unit time for the next execution;
[0094] Step 3: Determine whether the container has a resource limitation scenario. If not, jump to Step 4; if so, jump to Step 5;
[0095] Step 4: If no resource - limited scenario occurs, the execution plan of the task is not modified, and the task is executed normally;
[0096] Step 5: After moving the next execution plan of the task one time unit later, determine whether the adjusted task execution time exceeds the preset maximum execution time range of the task. If so, execute the task according to the adjusted task execution time; if not, jump to Step 2 to re - judge whether the container has a resource - limited scenario.
[0097] Please refer to Figure 4 , this embodiment provides a scheduling prompt system for timed tasks to implement the above - mentioned scheduling prompt method, including:
[0098] A resource consumption estimation module, which is used to obtain the CRON expression data of the timed task and the historical resource consumption data of the container, and estimate the current resource consumption data of the timed task based on the CRON expression data and the historical resource consumption data
[0099] An evaluation task total query module, which queries the total amount of evaluation tasks in the current unit time period based on the registered task execution information. Among them, the registered task execution information includes task quantity information and execution plan information, and the current unit time period is obtained based on the start time and time length of the timed task;
[0100] A scheduling prompt module, which is used to judge whether the container has resource limitations, and give a resource - limited prompt and schedule the timed task when the container has resources; among them, judging whether the container has resource limitations includes: if it is estimated that the current resource consumption data of the timed task is greater than the preset maximum resource consumption, or if it is queried that the total amount of evaluation tasks in the current unit time period is greater than the preset maximum task concurrency, then it is judged that the container has resource limitations.
[0101] This embodiment provides an electronic device, including at least one processing unit and at least one storage unit. Among them, the storage unit stores a computer program, and when the program is executed by the processing unit, the processing unit executes the above - mentioned scheduling prompt method.
[0102] This embodiment provides a storage medium, which stores a computer program executable by an electronic device. When the program runs on the electronic device, the electronic device executes the above - mentioned scheduling prompt method.
[0103] The above are only the preferred embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, various modifications and variations can be made to the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.
Claims
1. A scheduling prompt method for timed tasks, characterized in that, Including: Obtain the CRON expression data of the timing task and the historical resource consumption data of the container, and estimate the current resource consumption data of the timing task based on the CRON expression data and the historical resource consumption data; Query the total number of evaluation tasks in the current unit time period based on the registered task execution information, where the registered task execution information includes task quantity information and execution plan information, and the current unit time period is obtained based on the start time and time length of the timing task; If it is estimated that the current resource consumption data of the timing task is greater than the preset maximum resource consumption, or if it is queried that the total number of evaluation tasks in the current unit time period is greater than the preset maximum task concurrency, then prompt that the container has resource constraints and schedule the timing task.
2. The scheduling prompt method according to claim 1, wherein The estimation of the current resource consumption data of the timing task includes: Calculate the historical average resource consumption data of the timing task based on the CRON expression data and the historical resource consumption data; Estimate the current resource consumption data of the timing task based on the historical average resource consumption data of the timing task.
3. The scheduling prompt method according to claim 2, wherein The calculation formula for the historical average resource consumption data of the timing task is: Among them, is the historical average resource consumption data of the timed tasks, k m is the historical resource consumption data of the most recent timed task, k m-i is the historical resource consumption data of the i-th timed task before the most recent timed task, and n is the historical execution times of the timed tasks before the most recent timed task.
4. The scheduling prompt method according to claim 2 or 3, characterized in that, The calculation formula for the current resource consumption data of the timing task is: Among them, is the current resource consumption data of the timed task, is the historical average resource consumption data, k m is the historical resource consumption data of the most recent timed task, k m-n is the historical resource consumption data of the nth timed task before the most recent timed task, and ε is the minimum resource consumption data required for a single task.
5. The scheduling prompt method according to claim 1, wherein The situation where it is estimated that the current resource consumption data of the timing task is greater than the preset maximum resource consumption is specifically: Among them, is the current resource consumption data of the timing task, K max is the preset maximum resource consumption, ε k is the resource threshold coefficient.
6. The scheduling prompt method according to claim 1, characterized in that, The situation where it is queried that the total number of evaluation tasks in the current unit time period is greater than the preset maximum task concurrency is specifically: Tcount > ε t *T max Among them, Tcount is the total amount of evaluation tasks in the current unit time period, T max is the preset maximum task concurrency, ε t is the task threshold coefficient.
7. The scheduling prompt method according to claim 1, wherein The scheduling of the timing task includes: Move the timing task in the current unit time period forward or backward so that the current resource consumption data of the timing task is less than or equal to the preset maximum resource consumption, and the total number of evaluation tasks in the current unit time period is less than or equal to the preset maximum task concurrency.
8. A scheduling prompt system for timed tasks, which is used to implement the scheduling prompt method according to any one of claims 1 to 7, characterized in that, Including: A resource consumption estimation module, configured to obtain the CRON expression data of the timing task and the historical resource consumption data of the container, and estimate the current resource consumption data of the timing task based on the CRON expression data and the historical resource consumption data An evaluation task total query module, which queries the total number of evaluation tasks in the current unit time period based on the registered task execution information, where the registered task execution information includes task quantity information and execution plan information, and the current unit time period is obtained based on the start time and time length of the timing task; A scheduling and prompting module, configured to determine whether the container has resource constraints, and perform resource constraint prompting and schedule the timing task when the container has resources; among them, determining whether the container has resource constraints includes: if it is estimated that the current resource consumption data of the timing task is greater than the preset maximum resource consumption, or if it is queried that the total number of evaluation tasks in the current unit time period is greater than the preset maximum task concurrency, then determine that the container has resource constraints.
9. An electronic device, characterized in that, Comprising at least one processing unit and at least one storage unit, wherein the storage unit stores a computer program, and when the program is executed by the processing unit, the processing unit is caused to execute the scheduling prompt method according to any one of claims 1 to 7.
10. A storage medium, characterized in that, It stores a computer program executable by an electronic device, and when the program runs on the electronic device, the electronic device is caused to execute the scheduling prompt method according to any one of claims 1 to 7.