A task scheduling method and system

Abstract

The application discloses a task scheduling method and system. The method comprises the following steps: a task scheduler initializes a multi-level time wheel, each level of the time wheel has an independent slot span and slot quantity; a to-be-scheduled task is received, time parameters of the to-be-scheduled task are analyzed, and the to-be-scheduled task is automatically routed to a corresponding level of the time wheel; the to-be-scheduled task is inserted into a dynamic expansion task queue of a corresponding slot through slot hash mapping, and task metadata is simultaneously persisted to an external storage medium; a clock watchdog thread is independently started for each level of the time wheel, a current slot is periodically scanned according to a slot span, a to-be-processed task is asynchronously executed, and an execution state is recorded; a monitoring process monitors a survival state of the scheduler in real time, the scheduler is automatically restarted when an abnormal termination is detected, all uncompleted task metadata is loaded from the external storage medium, a slot is recalculated, and the time wheel is joined. The application has the advantages of low delay, high throughput and recoverability without relying on a third-party middleware.

Description

Technical Field

[0001] This invention relates to the field of computer software technology, and more specifically, to a task scheduling method and system. Background Technology

[0002] Task scheduling is a fundamental function in computer systems, widely used in scenarios such as scheduled tasks, delayed messages, distributed coordination, and data synchronization. Whether it's a timer at the operating system level or a job scheduling framework at the application software level, the core objective is to trigger a preset operation at a specified future time or at a fixed period.

[0003] Currently, mainstream task scheduling solutions mainly fall into two categories: one is a lightweight solution based on the operating system's native timer or thread pool for delayed execution, such as ScheduledThreadPoolExecutor and DelayQueue in Java; the other is a distributed scheduling solution based on external middleware, such as Quartz combined with frameworks like JDBC, Elastic-Job, and XXL-JOB, which typically rely on message queues (such as RabbitMQ and Kafka), distributed coordination services (such as ZooKeeper), or relational databases.

[0004] However, existing technologies have the following technical problems in practical applications: First, it suffers from heavy dependency and high operational costs. While scheduling solutions based on external middleware are powerful, they require the additional deployment and maintenance of heavyweight components such as message queues and distributed coordination services. For resource-constrained or lightweight scenarios such as embedded devices, edge computing nodes, and small and medium-sized microservices, introducing these components significantly increases the system's deployment complexity, resource consumption, and operational costs, resulting in an issue of "over-engineering."

[0005] Second, there is redundancy and susceptibility to blocking. Traditional schedulers often employ single-threaded polling or global locking mechanisms when handling a large number of tasks. When task execution times are long or the number of tasks surges, scheduling blockages can easily occur, causing subsequent tasks to be delayed and impacting overall throughput and response time. Especially in high-concurrency scenarios, the scheduler's performance bottlenecks will be quickly exposed.

[0006] Third, they have weak fault recovery capabilities and are prone to task loss. Many lightweight schedulers store tasks entirely in memory, and once the service process crashes or the machine loses power, all unexecuted tasks will be permanently lost. Even some solutions that support persistence often require complex configurations and additional storage components, and the recovery process requires manual intervention, making it impossible to achieve automatic and lossless fault recovery.

[0007] Fourth, the task granularity is limited and flexibility is insufficient. While existing time wheel algorithms can perform task insertion and deletion, a single-level time wheel presents a trade-off between precision and span: high precision requires fine-grained slots, but consumes a large amount of memory; large spans require coarse-grained slots, but at the cost of precision. A single time wheel cannot simultaneously meet the scheduling requirements of millisecond-level precision and hourly-level spans.

[0008] Therefore, there is an urgent need for a task scheduling method and system with low dependency, low latency, high throughput, and recoverability to solve the above-mentioned technical problems. Summary of the Invention

[0009] This invention provides a task scheduling method and system to solve the problems of heavy reliance, high operation and maintenance costs, redundant links, easy blockage, weak anomaly recovery capability, easy task loss, and single task granularity and insufficient flexibility in the prior art.

[0010] To achieve the above objectives, the present invention provides a task scheduling method, comprising: S1, the task scheduler initializes a multi-level time wheel, each level of the time wheel being a circular array structure with a preset number of slots, each slot being associated with a dynamically expandable task queue, and each level of the time wheel having an independent slot span and total time span; S2, the task scheduler receives a task to be scheduled, parses the execution cycle or delay time of the task, automatically routes the task to the corresponding level of the time wheel according to a preset time threshold, calculates the target slot number through slot hash mapping, inserts the task into the task queue corresponding to the target slot number, and simultaneously persists the task's metadata to an external storage medium; S3, the task scheduler independently starts a clock watchdog line for each level of the time wheel. In the process, each clock watchdog thread periodically moves the current slot pointer according to the slot span of the corresponding time wheel level, scans the task queue of the current slot, and if there is a task to be executed, submits the task to the asynchronous thread pool for asynchronous execution, and records the execution start time, end time, execution status and exception information of the task; S4, monitors the liveness status of the task scheduler process in real time, and automatically restarts the task scheduler process when the abnormal termination of the task scheduler process is detected; S5, after restarting, the task scheduler loads all unfinished task metadata from the external storage medium, recalculates the time wheel level and target slot number of each task according to the difference between the next execution time of each task and the current time, and re-inserts the task into the corresponding task queue.

[0011] Optionally, the task scheduler initializes a multi-level time wheel including: initializing a second-level time wheel with 100 slots, a single slot span of 10 milliseconds, and a total time span of 1 second; initializing a minute-level time wheel with 60 slots, a single slot span of 1 second, and a total time span of 60 seconds; and initializing an hour-level time wheel with 60 slots, a single slot span of 1 minute, and a total time span of 60 minutes. The initial capacity of the task queue for each slot is 8. When the queue length reaches three-quarters of the initial capacity, a dynamic expansion of 2 times is triggered, and the original task order is maintained after the expansion.

[0012] Optionally, the calculation method for the slot hash mapping is as follows: target slot number = (current slot pointer position + step number) mod total number of slots in the current level time wheel; where, step number = task delay time ÷ single slot span of the current level time wheel; the task metadata includes: unique task identifier, task class path, execution method name, execution input parameters, execution cycle, next execution timestamp, priority, maximum number of retries, retry interval, and current number of retries.

[0013] Optionally, when inserting the task into the task queue corresponding to the target slot number, the method further includes: performing queue insertion processing according to the priority of the task: inserting high-priority tasks into the head of the queue and ordinary-priority tasks into the tail of the queue; the external storage medium is a hybrid storage of Redis cache and relational database, wherein Redis is used for high-speed read and write, and relational database is used for permanent storage.

[0014] Optionally, when submitting the task to the asynchronous thread pool for asynchronous execution, the method further includes: if the task execution throws an exception, automatically retrying according to the maximum number of retries and retry interval recorded in the task's task metadata; when the number of retries exceeds the maximum number of retries, marking the task as a failed state, writing it into the exception log, and triggering the monitoring and alarm module to send a notification.

[0015] On the other hand, the present invention provides a task scheduling system, comprising: a time wheel initialization unit, used by the task scheduler to initialize multi-level time wheels, each level of time wheel being a circular array structure with a preset number of slots, each slot being associated with a dynamically expandable task queue, and each level of time wheel having an independent slot span and total time span; a task routing and persistence unit, used by the task scheduler to receive tasks to be scheduled, parse the execution cycle or delay time of the task, automatically route the task to the corresponding level of time wheel according to a preset time threshold, calculate the target slot number through slot hash mapping, insert the task into the task queue corresponding to the target slot number, and persist the task's metadata to an external storage medium; and a clock watchdog and asynchronous execution unit, used by the task scheduler to independently start each level of time wheel. A clock watchdog thread periodically moves the current slot pointer according to the slot span of the corresponding time wheel level, scans the task queue of the current slot, and if there is a task to be executed, submits the task to the asynchronous thread pool for asynchronous execution, and records the start time, end time, execution status and exception information of the task; a monitoring unit is used to monitor the liveness status of the task scheduler process in real time, and automatically restarts the task scheduler process when an abnormal termination is detected; a restart unit is used for the restarted task scheduler to load all unfinished task metadata from the external storage medium, recalculate the time wheel level and target slot number of each task according to the difference between the next execution time of each task and the current time, and re-insert the task into the corresponding task queue.

[0016] Optionally, the task scheduler initializes a multi-level time wheel including: initializing a second-level time wheel with 100 slots, a single slot span of 10 milliseconds, and a total time span of 1 second; initializing a minute-level time wheel with 60 slots, a single slot span of 1 second, and a total time span of 60 seconds; and initializing an hour-level time wheel with 60 slots, a single slot span of 1 minute, and a total time span of 60 minutes. The initial capacity of the task queue for each slot is 8. When the queue length reaches three-quarters of the initial capacity, a dynamic expansion of 2 times is triggered, and the original task order is maintained after the expansion.

[0017] Optionally, the calculation method for the slot hash mapping is as follows: target slot number = (current slot pointer position + step number) mod total number of slots in the current level time wheel; where, step number = task delay time ÷ single slot span of the current level time wheel; the task metadata includes: unique task identifier, task class path, execution method name, execution input parameters, execution cycle, next execution timestamp, priority, maximum number of retries, retry interval, and current number of retries.

[0018] Optionally, when inserting the task into the task queue corresponding to the target slot number, the method further includes: performing queue insertion processing according to the priority of the task: inserting high-priority tasks into the head of the queue and ordinary-priority tasks into the tail of the queue; the external storage medium is a hybrid storage of Redis cache and relational database, wherein Redis is used for high-speed read and write, and relational database is used for permanent storage.

[0019] Optionally, when submitting the task to the asynchronous thread pool for asynchronous execution, the method further includes: if the task execution throws an exception, automatically retrying according to the maximum number of retries and retry interval recorded in the task's task metadata; when the number of retries exceeds the maximum number of retries, marking the task as a failed state, writing it into the exception log, and triggering the monitoring and alarm module to send a notification.

[0020] The beneficial effects of this invention are: This invention provides a task scheduling method and system. The method initializes a multi-level time wheel through a task scheduler, separating tasks with different time granularities into time wheels at various levels with independent slot spans and total time spans. Combined with a dynamically expandable task queue and slot hash mapping, it achieves low-latency, high-throughput task scheduling. By persisting task metadata to external storage and automatically restarting the task scheduler process when it terminates abnormally, the restarted scheduler loads all unfinished task metadata from the external storage, recalculates the time wheel level and target slot number based on the difference between the next execution time and the current time, and re-inserts the task into the task queue. This achieves automatic task recovery without loss, significantly improving system reliability and stability. Furthermore, the entire method does not rely on third-party middleware, reducing deployment and maintenance costs, and is suitable for resource-constrained scenarios such as embedded systems, edge computing, and small-to-medium-sized services. Attached Figure Description

[0021] Figure 1 This is a flowchart of a task scheduling method provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of the structure of a task scheduling system provided in an embodiment of the present invention. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and a specific embodiment throughout. Obviously, the described embodiments are only a part of the embodiments of this invention, and not all of them. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0023] Suppose we are developing a smart home control system that runs on a resource-constrained IoT gateway device. This device uses an ARM architecture processor and is equipped with 1GB of RAM and 4GB of storage. The system needs to schedule the following five typical tasks: The first category is temperature acquisition tasks. This system needs to read data from the temperature sensor every 50 milliseconds and update the memory cache. This task requires extremely high accuracy because temperature changes may trigger emergency logic (such as turning on air conditioning or heating equipment). Latency must be controlled within milliseconds.

[0024] The second category is device status reporting tasks. This system needs to report the status of all devices in the home (lights, air conditioners, door locks, etc.) to the cloud server every 30 seconds. This task allows for a second-level error, but requires reliable execution and cannot lose reported data due to system restarts.

[0025] The third type is the firmware version check task. This system needs to check the cloud every 10 minutes for new firmware releases, and if found, trigger a download prompt. This task has a longer cycle and doesn't require high precision, but it needs to be persistently recorded to avoid duplicate prompts.

[0026] The fourth category is smoke alarm detection tasks. This is an urgent task that requires immediate execution after the smoke sensor is triggered, with a delay of less than 100 milliseconds. The tasks include sending an alarm notification, starting the exhaust fan, and closing the gas valve. This task requires high-priority scheduling and will not be retried, as smoke alarms are emergencies and retrying is pointless.

[0027] The fifth category is log cleanup tasks. This system needs to clean up local log files older than 7 days every day at 2 AM to free up storage space. This is an extremely long-running task, allowing for delayed execution (if the system is shut down at 2 AM, it should be executed immediately upon recovery).

[0028] In traditional solutions, using a combination of Quartz, MySQL, and Redis is simply unsustainable due to the limited memory and storage of the gateway device. Furthermore, using a simple thread pool for delayed execution fails to address task persistence and fault recovery issues. The solution presented in this invention, however, is perfectly suited to this resource-constrained and highly reliable scenario.

[0029] Below, we will describe in detail how the smart home control system achieves task scheduling according to the five steps of the claims of this invention. Figure 1 This is a flowchart of a task scheduling method provided in an embodiment of the present invention; as shown below. Figure 1 As shown, the method includes: S1. The task scheduler initializes a multi-level time wheel. Each level of the time wheel is a circular array structure with a preset number of slots. Each slot is associated with a dynamically expandable task queue, and each level of the time wheel has an independent slot span and total time span. In an optional implementation, the task scheduler initializes the multi-level time wheel by: Initialize a second-level time wheel with 100 slots, a single slot span of 10 milliseconds, and a total time span of 1 second; Initialize a minute-level time wheel with 60 slots, a single slot span of 1 second, and a total time span of 60 seconds; Initialize the hourly time wheel with 60 slots, a single slot span of 1 minute, and a total time span of 60 minutes. The initial capacity of the task queue for each slot is 8. When the queue length reaches three-quarters of the initial capacity, a dynamic expansion of 2 times is triggered, and the original task order is maintained after the expansion.

[0030] Specifically, upon system startup, the task scheduler first initializes a multi-level time wheel. This invention employs a three-level circular time wheel architecture: a second-level time wheel, a minute-level time wheel, and an hour-level time wheel. Each level of the time wheel is a circular array structure, where each element is called a slot, and each slot is associated with a dynamically expandable task queue.

[0031] For the second-level time wheel, the task scheduler sets its number of slots to 100, with each slot having a span of 10 milliseconds. This means that the second-level time wheel takes 1 second to complete a full rotation. This time wheel is used to schedule tasks with execution cycles or delays of less than 1 second, providing scheduling accuracy at the 10-millisecond level.

[0032] For the minute-level time wheel, the task scheduler sets the number of slots to 60, with each slot spanning 1 second. The time wheel takes 60 seconds to complete a full rotation, and is used to schedule tasks with execution cycles between 1 and 60 seconds.

[0033] For the hourly time wheel, the task scheduler sets the number of slots to 60, with each slot spanning 1 minute. The time wheel takes 60 minutes to complete a full rotation, and is used to schedule tasks with execution periods between 1 minute and 60 minutes.

[0034] During initialization, the task scheduler also creates a dynamic task queue for each slot. The initial capacity of this queue is uniformly set to 8, meaning each slot can initially hold 8 tasks to be executed. When the task queue length of a slot reaches three-quarters of its initial capacity (i.e., 6 tasks), the task scheduler automatically doubles the queue's capacity to 16. If subsequent tasks continue to increase, when the queue length reaches 12, it will be expanded again to 32, and so on. This dynamic expansion mechanism ensures that the queue does not overflow and avoids the waste caused by pre-allocating excessive memory. During the expansion process, the original task order remains unchanged, and the expansion operation does not block the insertion of other tasks.

[0035] In our smart home implementation, a second-level time wheel will be used for temperature acquisition tasks (50 milliseconds cycle) and smoke alarm detection tasks (latency requirement less than 100 milliseconds). A minute-level time wheel will be used for device status reporting tasks (30-second cycle). An hour-level time wheel will be used for firmware version checking tasks (10-minute cycle). For log cleanup tasks (24-hour cycle), since its cycle exceeds the maximum span of the hour-level time wheel, the task scheduler will automatically expand to a day-level time wheel with 24 slots, each slot spanning 1 hour, for a total of 24 hours.

[0036] Through this multi-level design, the present invention ensures precise scheduling of millisecond-level tasks, covers task cycles up to one day, and ensures that tasks of different granularities do not interfere with each other.

[0037] S2. The task scheduler receives the task to be scheduled, parses the execution cycle or delay time of the task, automatically routes the task to the corresponding level time wheel according to the preset time threshold, calculates the target slot number through slot hash mapping, inserts the task into the task queue corresponding to the target slot number, and persists the task's task metadata to the external storage medium. In an optional implementation, the task scheduler initializes the multi-level time wheel by: Initialize a second-level time wheel with 100 slots, a single slot span of 10 milliseconds, and a total time span of 1 second; Initialize a minute-level time wheel with 60 slots, a single slot span of 1 second, and a total time span of 60 seconds; Initialize the hourly time wheel with 60 slots, a single slot span of 1 minute, and a total time span of 60 minutes. The initial capacity of the task queue for each slot is 8. When the queue length reaches three-quarters of the initial capacity, a dynamic expansion of 2 times is triggered, and the original task order is maintained after the expansion.

[0038] In an optional implementation, the slot hash map is calculated as follows: Target slot number = (current slot pointer position + number of steps) mod total number of slots in the current level time wheel; Where, the number of steps = task delay time ÷ current level time wheel single slot span; The task metadata includes: unique task identifier, task class path, execution method name, execution input parameters, execution cycle, next execution timestamp, priority, maximum number of retries, retry interval, and current number of retries.

[0039] In an optional implementation, inserting the task into the task queue corresponding to the target slot number further includes: Tasks are queued according to their priority: high-priority tasks are inserted at the head of the queue, and normal-priority tasks are inserted at the tail of the queue; the external storage medium is a hybrid storage of Redis cache and relational database, where Redis is used for high-speed read and write, and relational database is used for persistent storage.

[0040] Specifically, when a user or system submits a task to be scheduled via API, the task scheduler performs the following steps.

[0041] (1) Parse task parameters The task scheduler first receives the task to be scheduled and parses its parameters. Taking the temperature acquisition task as an example, this task contains the following key information: the task's unique identifier is "temp_sensor_task_001"; the task class path is "com.smarthome.TemperatureCollectorTask"; the execution method name is "collect"; the execution input parameter is "sensor_id_01"; the execution period is 50 milliseconds; the priority is set to 5 (ranging from 1 to 10, with higher values ​​indicating higher priority); the maximum number of retries is 3; and the retry interval is 1000 milliseconds.

[0042] For device status reporting tasks, the cycle is 30 seconds, priority is 5, maximum retries are 3, and retry interval is 1000 milliseconds. For firmware version checking tasks, the cycle is 10 minutes, priority is 3, maximum retries are 2, and retry interval is 5000 milliseconds. For smoke alarm detection tasks, the delay is 0 milliseconds (immediate execution), priority is 10 (highest), and maximum retries are 0 (no retries). For log cleanup tasks, the cycle is 24 hours, execution time is 2 AM, priority is 2, maximum retries are 1, and retry interval is 1 hour.

[0043] (2) Automatically route to the time wheel of the corresponding level The task scheduler automatically routes tasks to appropriate time wheels based on their execution cycle or delay time, according to preset time thresholds. The specific rules are as follows: if the task's cycle or delay time is less than or equal to 1 second, it is routed to a second-level time wheel; if it is greater than 1 second but less than or equal to 60 seconds, it is routed to a minute-level time wheel; if it is greater than 60 seconds but less than or equal to 3600 seconds (i.e., 1 hour), it is routed to an hour-level time wheel; and if it exceeds 1 hour, it is routed to a dynamically expanded day-level or higher-level time wheel.

[0044] According to this rule, the temperature acquisition task has a cycle of 50 milliseconds, which is less than 1 second, and is therefore routed to the second-level time wheel. The device status reporting task has a cycle of 30 seconds, which is greater than 1 second and less than 60 seconds, and is therefore routed to the minute-level time wheel. The firmware version check task has a cycle of 10 minutes, which is greater than 60 seconds and less than 3600 seconds, and is therefore routed to the hour-level time wheel. The smoke alarm detection task has a latency of 0 milliseconds and is also routed to the second-level time wheel. The log cleanup task has a cycle of 24 hours, which exceeds 1 hour, and is therefore routed to the day-level time wheel.

[0045] (3) Calculate the target slot number The task scheduler uses slot hash mapping to calculate which slot in the current time wheel a task should be placed in. The calculation method is as follows: First, calculate the step number, which is the task's delay time divided by the single slot span of the current level time wheel; then calculate the target slot number, which is equal to the current slot pointer position plus the step number, and then modulo the total number of slots in the current level time wheel.

[0046] Taking a temperature acquisition task as an example, assume the current pointer of the second-level time wheel is pointing to slot 0. The task's delay is 50 milliseconds, and the span of a single slot on the second-level time wheel is 10 milliseconds. Therefore, the number of steps is 50 divided by 10, which equals 5 steps. The target slot number is equal to the current slot pointer position 0 plus the number of steps 5, modulo 100, resulting in 5. Therefore, the task is placed in slot 5 of the second-level time wheel.

[0047] Taking the device status reporting task as an example, assume the current pointer of the minute-level time wheel is pointing to slot 10. The task's cycle is 30 seconds, and the span of a single slot on the minute-level time wheel is 1 second, so the step count is 30 steps. The target slot number is equal to 10 plus 30 modulo 60, resulting in 40. Therefore, the task is placed in slot 40 of the minute-level time wheel.

[0048] For the smoke alarm detection task, the delay time is 0 milliseconds and the step count is 0, so it is placed in the slot currently pointed to by the pointer. If the current second-level time wheel pointer points to slot 42, then the task is placed in slot 42 and inserted at the head of the queue due to its high priority.

[0049] (4) Insert into the task queue and handle priority. After calculating the target slot number, the task scheduler inserts the task into the task queue for the corresponding slot. During insertion, the task scheduler processes tasks according to their priority: high-priority tasks are inserted at the head of the queue, and tasks of normal priority are inserted at the tail. If there are already multiple tasks with the same priority in the queue, they are arranged in a first-in, first-out (FIFO) order.

[0050] In this embodiment, the smoke alarm detection task has a priority of 10, which is the highest priority. When it is inserted into the task queue of a certain slot, it will be placed at the front of the queue, regardless of how many tasks are already in the queue, ensuring that it will be executed first in the next scan. The temperature acquisition task has a priority of 5, which is a medium priority. It will be inserted after all tasks with a priority higher than 5 and before all tasks with a priority lower than 5 in the queue.

[0051] (5) Persistence of task metadata While inserting a task into the memory queue, the task scheduler writes the task's core metadata to external storage. This invention employs a hybrid storage strategy: a Redis cache is used to store metadata for tasks to be executed within the last hour, providing high-speed read and write capabilities; a relational database (such as MySQL) is used to store complete metadata for all tasks, providing persistent data reliability.

[0052] Task metadata includes the following: unique task identifier, task class path, execution method name, execution parameters, execution cycle, next execution timestamp, priority, maximum number of retries, retry interval, current number of retries, and task status (pending execution, in execution, successful, failed).

[0053] Taking the temperature acquisition task as an example, its persistent record includes: task ID "temp_sensor_task_001"; task class path "com.smarthome.TemperatureCollectorTask"; execution method "collect"; input parameter "sensor_id_01"; period of 50 milliseconds; next execution time is a specific timestamp; priority is 5; maximum number of retries is 3; retry interval is 1000 milliseconds; current number of retries is 0; status is pending execution.

[0054] This metadata is written to both Redis and MySQL. The records in Redis are set to expire in one hour because the tasks within the last hour are hot data that need to be accessed quickly; the records in MySQL are stored permanently for full recovery and historical queries after a system restart.

[0055] For periodic tasks, the task scheduler also records their periodicity so that it can recalculate the next execution time and re-add the task to the time wheel after each execution. For one-off tasks, only the time of a single execution is recorded.

[0056] Through the above process, all tasks are correctly routed, located, and inserted into the corresponding slots of the corresponding time wheels, and their metadata is reliably persisted to external storage. This ensures that these tasks will not be lost even if the system subsequently crashes.

[0057] S3. The task scheduler starts a clock watchdog thread independently for each level of time wheel. Each clock watchdog thread moves the current slot pointer periodically according to the slot span of the corresponding level of time wheel, scans the task queue of the current slot, and if there is a task to be executed, submits the task to the asynchronous thread pool for asynchronous execution, and records the execution start time, end time, execution status and exception information of the task. In an optional implementation, when submitting the task to the asynchronous thread pool for asynchronous execution, the method further includes: If an exception is thrown during task execution, the task will be automatically retried according to the maximum number of retries and the retry interval recorded in the task's metadata. When the number of retries exceeds the maximum number of retries, the task will be marked as failed, written to the exception log, and the monitoring alarm module will be triggered to send a notification.

[0058] Specifically, the task scheduler starts a separate clock watchdog thread for each time wheel. Each watchdog thread is responsible for advancing the pointer of the corresponding time wheel and scanning the task queue of the current slot.

[0059] (1) Startup and execution of the watchdog thread The task scheduler starts a second-level watchdog thread for the second-level time wheel. This thread wakes up every 10 milliseconds, moves the pointer of the second-level time wheel forward one position (i.e., points to the next slot), and then scans the task queue in the newly arrived slot. Similarly, the task scheduler starts a minute-level watchdog thread for the minute-level time wheel, which wakes up every second, moves the pointer, and scans the slots. The hour-level watchdog thread for the hour-level time wheel wakes up every minute. If the system extends to a day-level time wheel, a day-level watchdog thread will also be started for the day-level time wheel, waking up every hour.

[0060] These watchdog threads run independently and do not interfere with each other. Even if the second-level watchdog is briefly blocked for some reason, the minute-level watchdog will still run on time, ensuring the scheduling accuracy of tasks at different granularities.

[0061] (2) Slot scanning and asynchronous execution When the watchdog thread moves its pointer to a slot, it checks if the task queue for that slot is empty. If it's not empty, the watchdog thread retrieves all tasks from the queue and submits them one by one to a separate asynchronous thread pool for execution. The watchdog thread itself doesn't execute any business logic; it's only responsible for task distribution. The advantage of this design is that even if a task takes a long time to execute (e.g., waiting for a response after calling an external API), it won't block the watchdog thread's pointer movement, thus ensuring the timely scheduling of subsequent tasks.

[0062] In this embodiment, assume the second-level watchdog timer moves its pointer to slot 5, where the task queue contains a temperature acquisition task. The watchdog thread detects the queue is not empty and submits the temperature acquisition task to the asynchronous thread pool. An idle thread in the thread pool receives the task, calls the `collect` method of the `TemperatureCollectorTask` class via reflection, passes the `sensor_id_01` parameter, and performs temperature data reading and cache update. After the task is completed, the thread pool thread records the task's start time, end time, execution status (success or failure), and execution time. Since the temperature acquisition task is periodic, after completion, the task scheduler recalculates the next execution time based on its 50-millisecond period and places the task back into the corresponding slot of the second-level time wheel, achieving cyclic scheduling.

[0063] Similarly, when the minute-level watchdog moves its pointer to slot 40, it triggers the execution of a device status reporting task. This task reports the device status via a network call to the cloud API. Because network calls can be unstable, this task may throw a timeout exception. When an exception occurs, the task scheduler will handle it according to the retry policy recorded in its task metadata, which will be detailed below.

[0064] When the hourly watchdog timer moves its pointer to slot 10, a firmware version check task is triggered. This task sends a version query request to the cloud, and if a new version is found, it logs the changes and triggers a notification.

[0065] When the smoke alarm sensor is triggered, the system immediately submits a smoke alarm detection task to the task scheduler with a delay of 0 milliseconds and a priority of 10. This task is routed to the current pointer slot of the second-level time wheel and, due to its high priority, is inserted at the head of the queue. When the second-level watchdog scans this slot, it prioritizes submitting this task to the asynchronous thread pool for execution. When the thread pool executes the task, it sends an alarm SMS, starts the exhaust fan, and closes the gas valve. Since the maximum number of retries for this task is 0, if execution fails, no retry will be performed; instead, a failure log will be logged and an alarm will be triggered.

[0066] (3) Recording and updating execution status During the execution of each task, the task scheduler records detailed execution information. For successfully executed tasks, it records the start timestamp, end timestamp, execution time (milliseconds), and the status "success." For tasks that fail, it records the exception stack trace and updates the status to "failed" or "retrying." This information is updated in real time to persistent storage for subsequent queries and fault analysis.

[0067] (4) Task retry and failure handling When a task throws an exception, the task scheduler will automatically retry according to the maximum number of retries and the retry interval recorded in the task's metadata. Taking the device status reporting task as an example, the maximum number of retries configured for this task is 3, and the retry interval is 1000 milliseconds. Assuming that the first execution fails due to a network timeout, after the task scheduler catches the exception, it first checks whether the current number of retries (initially 0) is less than the maximum number of retries (3). Since 0 is less than 3, the task scheduler increases the current number of retries to 1, and then, based on the retry interval of 1000 milliseconds, sets the next execution time of the task to the current time plus 1000 milliseconds, and re-invokes the routing logic to put the task back into the appropriate time wheel (since the delay is 1 second, it is still put into the minute-level time wheel). At the same time, the task scheduler updates the task metadata in persistent storage, updating the current number of retries to 1.

[0068] If the second execution also fails, the task scheduler increases the current retry count to 2 and reschedules after a 1-second delay. If the third execution still fails, the task scheduler increases the current retry count to 3. At this point, the current retry count equals the maximum retry count, and the retry condition is no longer met. The task scheduler marks the task's status as "failed," writes it to the exception log file, and triggers the monitoring and alarm module to send a notification (such as an email or SMS) to the operations and maintenance personnel, informing them that the task failed after 3 retries and requires manual intervention.

[0069] Different retry strategies can be configured for different tasks. For example, the firmware version check task is configured to retries twice, with a retry interval of 5 seconds; the temperature acquisition task is configured to retries three times, with a retry interval of 1 second; and the smoke alarm detection task is configured to retries zero times, because if an alarm event fails, it needs to be handled manually immediately, rather than being automatically retried.

[0070] S4. Monitor the liveness status of the task scheduler process in real time, and automatically restart the task scheduler process when the abnormal termination of the task scheduler process is detected. Specifically, the present invention also includes a key mechanism: real-time monitoring of the liveness of the task scheduler process, and automatic restart of the process when abnormal termination is detected.

[0071] (1) Implementation of survival monitoring The system can deploy a standalone monitoring module (e.g., a lightweight daemon process), which runs on the same device as the task scheduler process or on a separate monitoring node. The monitoring module sends a heartbeat check to the task scheduler process at regular intervals (e.g., every second). Heartbeat checks can be performed in several ways: checking for the existence of the task scheduler process's PID file; sending an HTTP health check request to a specific port of the task scheduler process; or using shared memory for inter-process communication.

[0072] In the smart home embodiment of this application, a simple daemon script runs on the IoT gateway device. This script checks the PID of the task scheduler process every second. If the PID file exists and the corresponding process is running, the task scheduler is considered to be working normally; if the PID file does not exist or the corresponding process has terminated, the task scheduler is determined to be abnormal.

[0073] (2) Detection of abnormal termination and restart triggering Suppose that at 2:15 AM, the IoT gateway device suddenly loses power (or the operating system crashes, or the task scheduler process is forcibly killed). In the next detection cycle (1 second later), the monitoring module detects that the task scheduler process's PID file has disappeared or the process no longer exists, thus determining that the task scheduler process has terminated abnormally.

[0074] At this point, the monitoring module performs an automatic restart operation. Specifically, the monitoring module first cleans up any remaining old process resources (e.g., by sending a kill signal to ensure that the old process is completely terminated), then waits for a short period of time (e.g., 1 second) to allow the system to stabilize, and finally executes the task scheduler's start command to restart the task scheduler process.

[0075] This process is fully automated and requires no manual intervention. In this embodiment, after the device is powered back on, the monitoring module will restart the task scheduler within seconds, and the user will hardly be aware that the system had ever crashed.

[0076] S5. After restarting, the task scheduler loads all unfinished task metadata from the external storage medium, recalculates the time wheel level and target slot number of each task based on the difference between the next execution time of each task and the current time, and re-inserts the task into the corresponding task queue.

[0077] Specifically, the restarted task scheduler process first initializes, then loads all unfinished task metadata from external storage, and recalculates the time wheel level and target slot number of each task based on the difference between the next execution time of each task and the current time. Finally, it reinserts the tasks into the corresponding task queue.

[0078] (1) Load task metadata from persistent storage After restarting, the task scheduler first connects to the Redis cache and attempts to load metadata for all tasks whose keys are prefixed with "task:". Redis stores hot data for tasks scheduled to be executed within the last hour, resulting in fast loading. If the data in Redis is incomplete (e.g., some tasks have been cleared because they expired after one hour), the task scheduler will fall back to the MySQL database, execute queries, and retrieve metadata for all tasks with a status of "pending" or "in execution".

[0079] In this embodiment, the following tasks existed before the system crashed: temperature acquisition task (cycle 50 milliseconds, with a large number of unexecuted instances in memory), device status reporting task (next execution time 30 seconds after the crash), firmware version check task (next execution time 5 minutes after the crash), and log cleanup task (next execution time at 2:00 AM, but the crash occurred at 2:15 AM, and the task had expired 15 minutes ago).

[0080] The restarted task scheduler reads the metadata records of these tasks from MySQL.

[0081] (2) Filter expired tasks For each loaded task, the task scheduler compares its next execution timestamp with the current system timestamp. If the next execution timestamp is greater than or equal to the current time, the task has not expired and can be rescheduled directly. If the next execution timestamp is less than the current time, the task has expired, and the task scheduler needs to decide how to handle it based on whether the task allows delayed execution.

[0082] For tasks that allow delayed execution (such as log cleanup tasks), the task scheduler adjusts their next execution time to the current time (i.e., executes immediately) and then re-adds them to the time wheel. For tasks that do not allow delayed execution (such as certain tasks with high real-time requirements), the task scheduler marks their status as "skipped," records the reason for skipping, and does not reschedule them.

[0083] In this embodiment, since the temperature acquisition task is a high-frequency periodic task, only a new task with a period of 50 milliseconds needs to be registered during recovery; it is not necessary to restore every historical instance. The task scheduler detects that the task ID already exists and that it is a periodic task, so it directly recreates it and adds it to the second-level time wheel. The device status reporting task's next execution time is 30 seconds after the crash. This time has not yet arrived when the system recovers (assuming recovery takes 30 seconds, which is exactly the time), so the task is rescheduled for immediate execution at the current time. The firmware version check task's next execution time is 5 minutes after the crash. There are 4 minutes and 30 seconds left before execution when the system recovers. The task scheduler calculates the step count and places it in the corresponding slot of the hour-level time wheel. The log cleanup task's next execution time is 2:00 AM. The system recovers at 2:15:30 AM, meaning the task has expired for 15 minutes and 30 seconds. Since the log cleanup task allows delayed execution, the task scheduler reschedules it for immediate execution, cleaning the log files immediately after recovery.

[0084] (3) Recalculate the time wheel level and target slot. For each task requiring rescheduling, the task scheduler, based on the difference between its next execution time and the current time (i.e., the remaining delay time), and following the same routing rules as in step S2, re-determines which time wheel level the task should enter. Then, based on the remaining delay time and the current time wheel pointer position, the target slot number is recalculated. The calculation method is also the same: the step number equals the remaining delay time divided by the single slot span, and the target slot number equals the current slot pointer position plus the step number modulo the total number of slots.

[0085] (4) Re-insert into the task queue After calculating the new time wheel level and target slot number, the task scheduler inserts the task into the task queue of the corresponding slot. Insertion also follows the priority rule: high-priority tasks are inserted at the head of the queue, and normal-priority tasks are inserted at the tail.

[0086] Through the above recovery process, all unfinished tasks were reloaded and correctly scheduled, achieving task recovery without loss. Users are completely unaware that the system had ever crashed. Functions such as device status reporting, temperature acquisition, and firmware checks continue to operate normally after recovery, and log cleanup tasks are also executed immediately after recovery, without any omissions due to the system crash.

[0087] In summary, this invention achieves low-dependency, low-latency, high-throughput, and highly reliable task scheduling through the organic combination of a multi-level time wheel architecture, dynamically expanding queues, persistent task metadata, independent watchdog threads with asynchronous execution, and automatic monitoring and recovery mechanisms. The above smart home embodiments fully demonstrate the technical solution and beneficial effects of this invention. Those skilled in the art should understand that these embodiments are for illustrative purposes only and not for limiting the scope of the invention. Any modifications and variations based on the principles of this invention fall within the protection scope of this invention.

[0088] Figure 2 This is a schematic diagram of the structure of a task scheduling system provided in an embodiment of the present invention; as shown below. Figure 2 As shown, the system includes: The time wheel initialization unit 201 is used for the task scheduler to initialize a multi-level time wheel. Each level of the time wheel is a circular array structure with a preset number of slots. Each slot is associated with a dynamically expandable task queue, and each level of the time wheel has an independent slot span and total time span. The task routing and persistence unit 202 is used for the task scheduler to receive the task to be scheduled, parse the execution cycle or delay time of the task, automatically route the task to the corresponding level time wheel according to the preset time threshold, calculate the target slot number through slot hash mapping, insert the task into the task queue corresponding to the target slot number, and persist the task metadata of the task to the external storage medium. The clock watchdog and asynchronous execution unit 203 are used by the task scheduler to independently start a clock watchdog thread for each level of time wheel. Each clock watchdog thread periodically moves the current slot pointer according to the slot span of the corresponding level of time wheel, scans the task queue of the current slot, and if there is a task to be executed, submits the task to the asynchronous thread pool for asynchronous execution, and records the execution start time, end time, execution status and exception information of the task. The monitoring unit 204 is used to monitor the survival status of the task scheduler process in real time, and automatically restart the task scheduler process when the abnormal termination of the task scheduler process is detected. The restart unit 205 is used to load all unfinished task metadata from the external storage medium after the task scheduler is restarted, recalculate the time wheel level and target slot number of each task based on the difference between the next execution time of each task and the current time, and re-insert the task into the corresponding task queue.

[0089] In an optional implementation, the task scheduler initializes the multi-level time wheel by: Initialize a second-level time wheel with 100 slots, a single slot span of 10 milliseconds, and a total time span of 1 second; Initialize a minute-level time wheel with 60 slots, a single slot span of 1 second, and a total time span of 60 seconds; Initialize the hourly time wheel with 60 slots, a single slot span of 1 minute, and a total time span of 60 minutes. The initial capacity of the task queue for each slot is 8. When the queue length reaches three-quarters of the initial capacity, a dynamic expansion of 2 times is triggered, and the original task order is maintained after the expansion.

[0090] In an optional implementation, the slot hash map is calculated as follows: Target slot number = (current slot pointer position + number of steps) mod total number of slots in the current level time wheel; Where, the number of steps = task delay time ÷ current level time wheel single slot span; The task metadata includes: unique task identifier, task class path, execution method name, execution input parameters, execution cycle, next execution timestamp, priority, maximum number of retries, retry interval, and current number of retries.

[0091] In an optional implementation, inserting the task into the task queue corresponding to the target slot number further includes: Tasks are queued according to their priority: high-priority tasks are inserted at the head of the queue, and normal-priority tasks are inserted at the tail of the queue; the external storage medium is a hybrid storage of Redis cache and relational database, where Redis is used for high-speed read and write, and relational database is used for persistent storage.

[0092] In an optional implementation, when submitting the task to the asynchronous thread pool for asynchronous execution, the method further includes: If an exception is thrown during task execution, the task will be automatically retried according to the maximum number of retries and the retry interval recorded in the task's metadata. When the number of retries exceeds the maximum number of retries, the task will be marked as failed, written to the exception log, and the monitoring alarm module will be triggered to send a notification.

[0093] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A task scheduling method, characterized in that, include: S1. The task scheduler initializes a multi-level time wheel. Each level of the time wheel is a circular array structure with a preset number of slots. Each slot is associated with a dynamically expandable task queue, and each level of the time wheel has an independent slot span and total time span. S2. The task scheduler receives the task to be scheduled, parses the execution cycle or delay time of the task, automatically routes the task to the corresponding level time wheel according to the preset time threshold, calculates the target slot number through slot hash mapping, inserts the task into the task queue corresponding to the target slot number, and persists the task's task metadata to the external storage medium. S3. The task scheduler starts a clock watchdog thread independently for each level of time wheel. Each clock watchdog thread moves the current slot pointer periodically according to the slot span of the corresponding level of time wheel, scans the task queue of the current slot, and if there is a task to be executed, submits the task to the asynchronous thread pool for asynchronous execution, and records the execution start time, end time, execution status and exception information of the task. S4. Monitor the liveness status of the task scheduler process in real time, and automatically restart the task scheduler process when the abnormal termination of the task scheduler process is detected. S5. After restarting, the task scheduler loads all unfinished task metadata from the external storage medium, recalculates the time wheel level and target slot number of each task based on the difference between the next execution time of each task and the current time, and re-inserts the task into the corresponding task queue.

2. The method according to claim 1, characterized in that, The task scheduler initializes the multi-level time wheel, including: Initialize a second-level time wheel with 100 slots, a single slot span of 10 milliseconds, and a total time span of 1 second; Initialize a minute-level time wheel with 60 slots, a single slot span of 1 second, and a total time span of 60 seconds; Initialize the hourly time wheel with 60 slots, a single slot span of 1 minute, and a total time span of 60 minutes. The initial capacity of the task queue for each slot is 8. When the queue length reaches three-quarters of the initial capacity, a dynamic expansion of 2 times is triggered, and the original task order is maintained after the expansion.

3. The method according to claim 1, characterized in that: The calculation method for the slot hash mapping is as follows: Target slot number = (current slot pointer position + number of steps) mod total number of slots in the current level time wheel; Where, the number of steps = task delay time ÷ current level time wheel single slot span; The task metadata includes: unique task identifier, task class path, execution method name, execution input parameters, execution cycle, next execution timestamp, priority, maximum number of retries, retry interval, and current number of retries.

4. The method according to claim 1, characterized in that, When inserting the task into the task queue corresponding to the target slot number, the method further includes: Tasks are queued according to their priority: high-priority tasks are inserted at the head of the queue, and normal-priority tasks are inserted at the tail of the queue; the external storage medium is a hybrid storage of Redis cache and relational database, where Redis is used for high-speed read and write, and relational database is used for persistent storage.

5. The method according to claim 1, characterized in that: When submitting the task to the asynchronous thread pool for asynchronous execution, the following is also included: If an exception is thrown during task execution, the task will be automatically retried according to the maximum number of retries and the retry interval recorded in the task's metadata. When the number of retries exceeds the maximum number of retries, the task will be marked as failed, written to the exception log, and the monitoring alarm module will be triggered to send a notification.

6. A task scheduling system, characterized in that, include: The time wheel initialization unit is used by the task scheduler to initialize multi-level time wheels. Each level of time wheel is a circular array structure with a preset number of slots. Each slot is associated with a dynamically expandable task queue, and each level of time wheel has an independent slot span and total time span. The task routing and persistence unit is used by the task scheduler to receive the task to be scheduled, parse the execution cycle or delay time of the task, automatically route the task to the corresponding level time wheel according to the preset time threshold, calculate the target slot number through slot hash mapping, insert the task into the task queue corresponding to the target slot number, and persist the task's task metadata to the external storage medium. The clock watchdog and asynchronous execution unit are used by the task scheduler to independently start a clock watchdog thread for each level of time wheel. Each clock watchdog thread periodically moves the current slot pointer according to the slot span of the corresponding level of time wheel, scans the task queue of the current slot, and if there is a task to be executed, submits the task to the asynchronous thread pool for asynchronous execution, and records the execution start time, end time, execution status and exception information of the task. The monitoring unit is used to monitor the liveness status of the task scheduler process in real time, and automatically restart the task scheduler process when the abnormal termination of the task scheduler process is detected. The restart unit is used to load all unfinished task metadata from the external storage medium after the task scheduler restarts, recalculate the time wheel level and target slot number of each task based on the difference between the next execution time of each task and the current time, and re-insert the task into the corresponding task queue.

7. The system according to claim 6, characterized in that, The task scheduler initializes the multi-level time wheel, including: Initialize a second-level time wheel with 100 slots, a single slot span of 10 milliseconds, and a total time span of 1 second; Initialize a minute-level time wheel with 60 slots, a single slot span of 1 second, and a total time span of 60 seconds; Initialize the hourly time wheel with 60 slots, a single slot span of 1 minute, and a total time span of 60 minutes. The initial capacity of the task queue for each slot is 8. When the queue length reaches three-quarters of the initial capacity, a dynamic expansion of 2 times is triggered, and the original task order is maintained after the expansion.

8. The system according to claim 6, characterized in that: The calculation method for the slot hash mapping is as follows: Target slot number = (current slot pointer position + number of steps) mod total number of slots in the current level time wheel; Where, the number of steps = task delay time ÷ current level time wheel single slot span; The task metadata includes: unique task identifier, task class path, execution method name, execution input parameters, execution cycle, next execution timestamp, priority, maximum number of retries, retry interval, and current number of retries.

9. The system according to claim 6, characterized in that, When inserting the task into the task queue corresponding to the target slot number, the method further includes: Tasks are queued according to their priority: high-priority tasks are inserted at the head of the queue, and normal-priority tasks are inserted at the tail of the queue; the external storage medium is a hybrid storage of Redis cache and relational database, where Redis is used for high-speed read and write, and relational database is used for persistent storage.

10. The system according to claim 6, characterized in that: When submitting the task to the asynchronous thread pool for asynchronous execution, the following is also included: If an exception is thrown during task execution, the task will be automatically retried according to the maximum number of retries and the retry interval recorded in the task's metadata. When the number of retries exceeds the maximum number of retries, the task will be marked as failed, written to the exception log, and the monitoring alarm module will be triggered to send a notification.

Images