Distributed task scheduling method and device, electronic equipment and computer readable storage medium

Abstract

The present disclosure provides a distributed task scheduling method and device, electronic equipment and computer readable storage medium. The distributed task scheduling method comprises: creating a timing task on a function as a service platform; triggering a periodically scheduled task; implementing asynchronous distribution of the task by taking a message queue as a task distribution middleware; executing the task on the function as a service platform; and updating the execution and state of the task. By deploying the distributed task scheduling module and the task execution module on the FaaS platform, the system architecture is simplified, the system stability and scalability are solved, and the operation and learning costs of the system are reduced.

Description

Technical Field

[0001] This disclosure relates to the field of task scheduling, and more particularly to a distributed task scheduling method, apparatus, electronic device, and computer-readable storage medium. Background Technology

[0002] With the rapid development of computer technology, the volume of data being processed is increasing, placing ever higher demands on computer performance. In computer applications, every process involves task scheduling and processing, and task scheduling is a crucial component of the operating system. For real-time operating systems, task scheduling directly impacts their real-time performance. Existing task scheduling systems often employ complex overall architectures, placing significant pressure on the computer's real-time performance and failing to support high concurrency, maximize resource utilization, lack fault tolerance, and are inconvenient to manage. While some open-source solutions exist, maintaining them presents significant challenges, with heavy overall architectures, high learning costs, and potential unsuitability for business needs. Therefore, in many computer systems where there is a need to periodically schedule large-scale tasks, minimizing the impact of task scheduling on operating system performance and efficiently handling task scheduling is a pressing issue that needs to be addressed in this field. Summary of the Invention

[0003] This summary section is provided to briefly introduce the concepts, which will be described in detail in the detailed description section below. This summary section is not intended to identify key or essential features of the claimed technical solution, nor is it intended to limit the scope of the claimed technical solution.

[0004] To address the aforementioned technical issues, simplify the system architecture, improve system stability and scalability, and reduce operation, maintenance, and learning costs, the embodiments of this disclosure propose the following technical solutions.

[0005] In a first aspect, embodiments of this disclosure provide a distributed task scheduling method, including:

[0006] Set up a timed scheduling service, a task distribution middleware, and a task execution service on the Function as a Service platform;

[0007] The tasks that need to be periodically scheduled are triggered periodically by triggers in the timed scheduling service.

[0008] The task distribution middleware asynchronously distributes the tasks to multiple task execution services in the form of message queues;

[0009] The tasks issued by the task distribution middleware are executed through the multiple task execution services;

[0010] Update the execution and status of the task.

[0011] Furthermore, the step of periodically triggering tasks that require periodic scheduling through triggers in the timed scheduling service includes:

[0012] The Function as a Service platform triggers multiple of the aforementioned tasks;

[0013] When a task running in the schedule fails, the failed task is handled by one of the launched tasks that did not fail.

[0014] Furthermore, the task distribution middleware asynchronously distributes the task to multiple task execution services in the form of a message queue, including:

[0015] Establish task message queues in chronological order of task release time;

[0016] Obtain the load status of multiple task execution services, and asynchronously distribute tasks in the task message queue based on the load balancing principle.

[0017] Furthermore, the task is executed periodically at set times, and the execution of the task is sequential; that is, the next cycle cannot be executed until the same task has been completed in the current cycle.

[0018] Furthermore, the Function as a Service platform initiates a task scheduling distributed component to synchronize data between multiple nodes corresponding to multiple tasks through sharing, and the distributed lock in the distributed component is used to elect the master node.

[0019] Furthermore, the method also includes:

[0020] During task scheduling, the task scheduling capacity is expanded through dynamic partitioning;

[0021] Each task is bound to a partition, and each partition corresponds to a timer of the function i.e. service platform. When the number of tasks exceeds a certain threshold, a new partition is added, and the newly added tasks are bound to the new partition.

[0022] Furthermore, the method also includes:

[0023] During task execution, the message queue is expanded to include more partitions;

[0024] Based on the scalability of the Function as a Service platform, when the number of tasks increases, the platform will automatically add more tasks for task execution.

[0025] Furthermore, the method also includes:

[0026] The Online Transaction Processing (OLTP) storage component is used as persistent storage to save the created scheduled tasks and the state of task execution.

[0027] Furthermore, the state of the task includes the different states the task is in during the scheduling process and the transition information of the states.

[0028] Furthermore, the method also includes:

[0029] When an anomaly occurs in the scheduling of the task, the abnormal task is reprocessed.

[0030] Furthermore, the reprocessing of the abnormal task includes:

[0031] If writing to the message queue fails or the task fails to execute after the task is published, the task will be republished after the task execution exceeds a first time threshold.

[0032] If the task exits abnormally during execution and remains in a pending state, the task will be re-released after the pending state exceeds a second time threshold.

[0033] If the task fails, it will be reissued in the next task cycle; or

[0034] If the task execution is stopped, the task status is updated to paused.

[0035] Secondly, embodiments of this disclosure provide a distributed task scheduling apparatus, comprising:

[0036] The configuration module is used to configure the scheduled service, task distribution middleware, and task execution service on the Function as a Service platform.

[0037] The triggering module is used to periodically trigger tasks that need to be scheduled through triggers in the timed scheduling service.

[0038] The task distribution module is used by the task distribution middleware to asynchronously distribute the task to multiple task execution services in the form of a message queue.

[0039] The execution module is used to execute the tasks issued by the task distribution middleware through the multiple task execution services;

[0040] The update module is used to update the execution and status of the task.

[0041] Furthermore, the device also includes:

[0042] The synchronization election module is used by the Function as a Service platform to start the task scheduling distributed component, synchronize the data between multiple nodes corresponding to multiple tasks through sharing, and elect the master node through the distributed lock in the distributed component.

[0043] Furthermore, the device also includes:

[0044] The expansion module is used to expand the task scheduling capacity through dynamic partitioning during task scheduling, or to expand the message queue and add more partitions during task execution.

[0045] Furthermore, the device also includes:

[0046] The exception handling module is used to reprocess the exception task when an exception occurs in the scheduling of the task.

[0047] Thirdly, embodiments of this disclosure provide an electronic device, including:

[0048] Memory, used to store computer-readable instructions; and

[0049] A processor for executing the computer-readable instructions, causing the electronic device to implement the method according to any one of the first aspects above.

[0050] Fourthly, embodiments of this disclosure provide a non-transitory computer-readable storage medium for storing computer-readable instructions that, when executed by a computer, cause the computer to implement the method described in any one of the first aspects above.

[0051] This disclosure provides a distributed task scheduling method, apparatus, electronic device, and computer-readable storage medium. The distributed task scheduling method includes: creating a scheduled task on a Function as a Service (FaaS) platform; periodically triggering the periodically scheduled task; asynchronously allocating tasks using a message queue as a task distribution middleware; executing the task on the FaaS platform; and updating the execution and status of the task. This method, by deploying the distributed task scheduling module and task execution module on the FaaS platform, simplifies the system architecture, solves system stability and scalability issues, and reduces system operation and maintenance costs and learning curves.

[0052] The above description is merely an overview of the technical solution disclosed herein. In order to better understand the technical means of this disclosure and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this disclosure more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0053] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent when taken in conjunction with the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the originals and elements are not necessarily drawn to scale.

[0054] Figure 1 A flowchart illustrating a distributed task scheduling method provided in an embodiment of this disclosure;

[0055] Figure 2 This is a schematic diagram of a task scheduling architecture provided in an embodiment of the present disclosure;

[0056] Figure 3 This is a schematic diagram illustrating the flow of task states according to an embodiment of this disclosure.

[0057] Figure 4 A schematic diagram of a distributed task scheduling device provided in another embodiment of this disclosure;

[0058] Figure 5 A schematic diagram of the structure of an electronic device provided in another embodiment of this disclosure. Detailed Implementation

[0059] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0060] It should be understood that the steps described in the method embodiments of this disclosure may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of this disclosure is not limited in this respect.

[0061] The term "comprising" and its variations as used herein are open-ended inclusions, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the description below.

[0062] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.

[0063] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0064] Figure 1 This is a flowchart illustrating an embodiment of the distributed task scheduling method provided in this disclosure. The distributed task scheduling method provided in this embodiment can be executed by a distributed task scheduling device. This distributed task scheduling device can be implemented as software, or as a combination of software and hardware. This device can be integrated into a device within the task scheduling system, such as a terminal device. Figure 1 As shown, the method includes the following steps:

[0065] Step S101: Configure the scheduled service, task distribution middleware, and task execution service on the Function as a Service (FaaS) platform 220.

[0066] Function as a Service (FaaS), as an important development direction in the era of cloud computing 2.0, can bring tremendous value to developers in terms of engineering efficiency, reliability, performance, and cost, especially in greatly improving R&D efficiency. Therefore, embracing FaaS has become an important technical field of concern for developers. In step S101, a scheduled service (i.e., a scheduled component), a task distribution middleware, and a task execution service (i.e., a task execution component) are created on the FaaS platform. The scheduled service component is used to schedule and manage scheduled tasks and set the scheduled execution period. Due to business logic, all tasks are required to be executed periodically, and the execution must be serial. That is, the same task cannot be executed in the next period until the current period is completed. This is somewhat similar to the exclusive mode of the crontab command. Tasks will be in different states during the scheduling process, and the transition between states needs to be managed. The scheduled module starts multiple tasks on the FaaS platform and sets the task scheduling algorithm. When a running instance fails, a new instance will be used to handle the task. The scheduling component is deployed on the FaaS platform. FaaS features a shared high-speed processor and high-transmission network, without consuming system resources of the terminal device. When the application system on the terminal device needs to schedule a large number of tasks, it accesses the scheduling component on the FaaS platform in real time to schedule a large number of tasks reasonably. The periodically scheduled tasks are scheduled and their status changes are adjusted on the FaaS platform. On this FaaS platform, a task queue is used as middleware to obtain the load status of multiple execution entities, and the tasks are asynchronously allocated based on the load balancing principle. The FaaS platform has multiple task execution services (i.e., task execution components), which asynchronously allocate tasks based on the load balancing principle. After scheduling and allocation, tasks are assigned to execution components with lower load and within their capacity. By placing the execution components on the FaaS platform, the execution efficiency of large-scale tasks is greatly improved, and the resource load of the terminal system can be effectively saved.

[0067] Step S102: Trigger the task that needs to be periodically scheduled at regular intervals through the trigger in the timed scheduling service.

[0068] In step S102, the scheduling of tasks is periodically triggered by the scheduling component on the FaaS platform. In this embodiment of the disclosure, the scheduling of tasks is triggered by setting a timer in the scheduling component. By setting the period of timed triggering execution, tasks that need to be periodically scheduled are triggered periodically.

[0069] In this embodiment, the Timer class is used, a timer utility specifically provided by the JDK, to schedule the execution of specified tasks in a background thread. In the `java` package, it is used in conjunction with TimerTask. The Timer is essentially a task scheduler containing a TimerThread that loops infinitely through the TaskQueue to retrieve TimerTasks (which implement the Runnable interface). Calling its `run` method executes the scheduled task asynchronously. We need to inherit from the TimerTask class, implement its `run` method, and add our own business logic within it. First, instantiate a Timer class, then call its `schedule` method. In this method, instantiate a TimerTask class, and write the business logic in the `run` method. The last two parameters of the `schedule` method represent the delay time and the interval time, respectively, in milliseconds. In the example above, the scheduled task is set to execute every 1 second with a 2-second delay. This Timer class is very convenient for implementing multiple periodic scheduled tasks, supports delayed execution, and also supports execution after a specified time, making it quite powerful.

[0070] Additionally, this embodiment can use the Thread class to perform the simplest scheduled tasks. The `run` method contains an infinite `while` loop (there are other methods as well) to execute our own tasks. A crucial point to note is the need to use `try...catch` to catch exceptions; otherwise, if an exception occurs, the loop will exit immediately, preventing further execution. Scheduled tasks implemented this way can only execute periodically and cannot be scheduled to run at a specific time. Furthermore, this thread can be defined as a daemon thread, running silently in the background. Use cases include simple periodic tasks such as downloading a file every 10 minutes or reading a template file to generate a static HTML page every 5 minutes. This type of scheduled task is very simple, has a low learning curve, and is easy to learn, making it a good choice for simple periodic tasks.

[0071] Step S103: The task distribution middleware asynchronously distributes the task to multiple task execution services in the form of a message queue.

[0072] In step S103, the periodically scheduled task is scheduled and its state changes are adjusted on a Function as a Service (FaaS) platform. On this FaaS platform, a task queue acts as middleware to obtain the load status of multiple execution entities, and the tasks are asynchronously allocated based on load balancing principles. The task scheduling includes: the FaaS platform starts multiple tasks; when a task running in the schedule fails, a task that has not failed among the started tasks is used to handle the failed task. The FaaS platform starts a distributed task scheduling component, synchronizing data between multiple nodes corresponding to multiple tasks through sharing, and electing a master node through a distributed lock in the distributed component. During task scheduling, task scheduling is expanded through dynamic partitioning. Each task is bound to a partition, and each partition corresponds to a timer on the FaaS platform. When the number of tasks exceeds a certain threshold, a new partition is added, and the newly added tasks are bound to the new partition.

[0073] In this embodiment, the FaaS platform uses the Etcd component to implement the election of the master node in task scheduling. Etcd is a distributed, consistent key-value store, mainly used for shared configuration and service discovery. Developed and maintained by CoreOS, Etcd uses the Raft consensus algorithm to handle log replication to ensure strong consistency. Raft is a new consensus algorithm from Stanford, suitable for log replication in distributed systems. Raft achieves consistency through election; in Raft, any node can become the master node (leader). Managing the state between nodes has always been a challenge in distributed systems. Etcd seems specifically designed for service discovery and registration in cluster environments. It provides functions such as data TTL expiration, data change monitoring, multi-value, directory listening, and distributed lock atomic operations, making it easy to track and manage the state of cluster nodes. In all distributed systems, Etcd can solve the data sharing problem between multiple nodes. This is similar to team collaboration; members can work separately, but they always need to share some essential information, such as who the leader is, which members are present, and the order coordination between dependent tasks.

[0074] In this embodiment, Etcd provides interfaces for storing and retrieving data, ensuring strong consistency of data across multiple nodes in the Etcd cluster through a protocol. It is used for storing metadata and sharing configurations. A listening mechanism is provided, allowing clients to monitor changes to a specific key or a set of keys. Changes are monitored and pushed. Key expiration and renewal mechanisms are provided, with clients renewing keys through periodic refreshes. Cluster monitoring and service registration / discovery are handled. Atomic CAS (Compare-and-Swap) and CAD (Compare-and-Delete) support is provided, implemented through interface parameters or batch transactions. Distributed locks and leader election are also included. Specifically, when a task on one node fails, tasks on another node will elect a new service through Etch, ensuring uninterrupted task execution.

[0075] Etcd's leader election mechanism primarily relies on two core built-in mechanisms: TTL (Time to Live) and Atomic Compare-and-Swap (CAS). TTL sets an expiration time for a key, after which it is automatically deleted. This is used in many distributed lock implementations to ensure the lock's real-time validity. Atomic Compare-and-Swap (CAS) requires the client to provide certain conditions before assigning a value to a key; the assignment succeeds only if these conditions are met.

[0076] Step S104: Execute the task issued by the task distribution middleware through the multiple task execution services.

[0077] In step S104, multiple task execution components are set up on the FaaS platform. Based on load balancing principles, the tasks are asynchronously allocated. After scheduling, the allocated tasks are assigned to execution components with lower loads and within their capabilities. By placing these execution components on the FaaS platform, the execution efficiency of large batches of tasks is greatly improved, and the resource load of the terminal system is effectively saved. Typically, this execution component is a function module on the FaaS platform. When a task needs to be executed, the FaaS platform activates the execution component, projecting the function module's registry cache into the terminal system's registry without occupying physical space. Similarly, the file system projects its file cache into the terminal's file system without occupying physical space. At this time, system components and other processes installed on the terminal system can see the function module's registration entries and file sets on the FaaS platform and consider them real. They can access and use each other's registry entries and file sets as usual. The tasks are executed periodically on a timer basis, and the execution of tasks is sequential; that is, the next cycle cannot begin until the current cycle of the same task is completed.

[0078] In addition, during task execution, the message queue is expanded to add more partitions. Based on the expansion capability of the Function as a Service platform, when the number of tasks increases, the Function as a Service platform will automatically add more tasks for task execution.

[0079] In addition, the embodiments of this disclosure also include the case of reprocessing abnormal tasks when the scheduling of the task is abnormal.

[0080] The reprocessing of the abnormal task includes:

[0081] i. If writing to the message queue fails or the task execution fails after the task is published, the task will be republished after the task execution exceeds a first time threshold;

[0082] ii. If the task exits abnormally during execution and remains in a pending state, the task will be re-released after the pending state exceeds a second time threshold.

[0083] iii. If the task fails to execute, it will be reissued in the next task cycle;

[0084] iv. If the task execution is shut down, the task status is updated to paused.

[0085] Step S105: Update the execution and status of the task.

[0086] In step S105, the task scheduling component of the FaaS platform updates the execution and status of tasks in real time to ensure real-time task invocation.

[0087] This disclosure uses an Online Transaction Processing (OLTP) storage component as persistent storage to save the created scheduled tasks and their execution state. OLTP is a specialized, memory-optimized relational data management engine and a native compiler for stored procedures integrated into SQL Server. It handles the most demanding OLTP workloads. To achieve this, in-memory OLTP introduces two entirely new concepts: memory-optimized tables and natively compiled stored procedures. Data in memory-optimized tables resides in memory, and transactions are logged for recovery, without on-disk paging like traditional disk-based tables. Memory-optimized tables use hash and non-clustered sequential indexes to provide a highly optimized data access structure. The internal structure of these indexes differs from traditional B-trees, providing a new, high-performance way to access data in memory. Data access and transaction isolation are handled through a multi-version concurrency control mechanism that provides an optimistic, non-blocking implementation.

[0088] This embodiment provides an optimized method for migrating traditional disk-based tables and interpreted stored procedures to in-memory optimized tables and locally compiled stored procedures. It fully integrates the in-memory OLTP engine into SQL Server, with these task objects residing within the in-memory OLTP engine. This integration allows access to data stored in both in-memory optimized and disk-based tables using standard Transact-SQL (interpreted T-SQL) calls. This integration helps minimize application changes during the transition to in-memory OLTP. This efficient OLTP storage method enables real-time task execution and status updates while ensuring storage stability. The task status includes, but is not limited to, the different states the task is in during scheduling and state transition information.

[0089] like Figure 2 The diagram illustrates a task scheduling architecture, which includes a task scheduling component (taskScheduler), a task distribution middleware (dispatchMiddleware), a task execution component (taskExecWorker), persistent storage for task states (persistentStorage), and a task management component (taskManage). Specifically...

[0090] a) Task scheduling component (taskScheduler): A timed scheduling service deployed on the FaaS platform. It has a Timer trigger that periodically schedules tasks.

[0091] b) Task distribution middleware (message queue): Uses a message queue as the middleware for task distribution to achieve asynchronous task distribution and load balancing.

[0092] c) Task Execution Component (taskExecWorker): The task execution service is deployed on the FaaS platform and ultimately implements the execution of each task and the updating of task status.

[0093] d) Persistent Storage for Task Status: Uses OLTP storage components as persistent storage to save the created periodic tasks and the status of task execution.

[0094] e) Task Management Component (taskManage): Responsible for adding, deleting, modifying, and querying tasks.

[0095] like Figure 3 As shown, a diagram illustrating the flow of task states is presented. Due to business logic, all tasks are required to be executed periodically, and the execution must be sequential. That is to say, the same task cannot be executed in the next cycle until the current cycle is completed. This is somewhat similar to the exclusive mode of crontab. Tasks will be in different states during the scheduling process, and the flow of task states needs to be managed.

[0096] The specific task status transition process is as follows:

[0097] 1. The task is published and its status is set to running.

[0098] 2. The task begins execution, and the task status is set to pending.

[0099] 2.1 If a single cycle of the task is executed successfully, set the task status to "execution successful".

[0100] 2.2 The task will proceed to the next execution cycle, and its status will be set to pending.

[0101] 2.3 If a task fails to execute in a single cycle, set the task status to "execution failed" and proceed to step 3.1;

[0102] 2.4 If the task remains in a pending state and exceeds the first time threshold, the task will be reissued, and steps 1 and 2.1 will be repeated.

[0103] 3. Task execution successful or failed;

[0104] 3.1 If the task fails to execute, reissue the task.

[0105] 3.2. All executable periodic tasks have been executed successfully. Waiting for the next period's tasks to be released, proceed to steps 1 and 5.1.

[0106] 4. If task deployment fails and the task remains in the running state for an extended period beyond the second time threshold, the task will be re-deployed.

[0107] 5. Task paused;

[0108] 5.1 After successful execution, the task is closed and its status is set to suspended.

[0109] 5.2 If the task fails to execute, it will be closed and the task status will be set to suspended.

[0110] In addition, the task status also includes abnormal status, and the handling of abnormal status is as follows:

[0111] i. If writing to the message queue fails or the task execution fails after the task is published, the task will be republished after the task execution exceeds a first time threshold;

[0112] ii. If the task exits abnormally during execution and remains in a pending state, the task will be re-released after the pending state exceeds a second time threshold; iii. If the task fails to execute, it will be re-released in the next task cycle;

[0113] iv. If the task execution is shut down, the task status is updated to paused.

[0114] Figure 4 This is a schematic diagram of the structure of a distributed task scheduling device provided in another embodiment of the present disclosure. Figure 4 As shown, the device 400 includes: a setting module 401, a triggering module 402, a sending module 403, an execution module 404, and an update module 405. Wherein:

[0115] Module 401 is used to configure scheduled services, task distribution middleware, and task execution services on the Function as a Service (FaaS) platform. This module first creates a scheduled component on the FaaS platform. This component manages the scheduling of scheduled tasks and sets the execution cycle. Due to business logic, all tasks must be executed periodically and sequentially; that is, a task cannot start a new cycle until the current cycle is completed. This is similar to the exclusive mode of the crontab command. Tasks will be in different states during scheduling, requiring management of state transitions. The scheduled module starts multiple tasks on the FaaS platform and sets the task scheduling algorithm. When a running instance fails, a new instance is used to handle the task. This scheduling component is set up on the FaaS platform. FaaS has shared high-speed processors and high-transmission networks, without consuming system resources on terminal devices. When application systems on terminal devices need to schedule a large number of tasks, they access the scheduling component on the FaaS platform in real time to schedule a large number of tasks appropriately. Secondly, this module first sets up middleware on the FaaS platform. On this FaaS platform, a task message queue is used as middleware to obtain the load status of multiple execution entities, and the tasks are asynchronously allocated based on load balancing principles. The FaaS platform also sets up multiple task execution services (i.e., task execution components), which asynchronously allocate tasks based on load balancing principles. The scheduled tasks are assigned to execution components with lower loads and within their capabilities. By placing the execution components on the FaaS platform, the execution efficiency of large batches of tasks is greatly improved, and the resource load of the terminal system can be effectively saved.

[0116] The triggering module 402 is used to periodically trigger tasks that require periodic scheduling through triggers in the timed scheduling service. By periodically triggering task scheduling through the scheduling component on the FaaS platform, this embodiment of the disclosure triggers task scheduling by setting a timer in the scheduling component and setting the period for timed execution. In this embodiment, the timer tool provided by the Timer class or Thread class timer is used to schedule the execution of specified tasks in a background thread.

[0117] The distribution module 403 is used by the task distribution middleware to asynchronously distribute the tasks to multiple task execution services in the form of message queues. The periodically scheduled tasks are scheduled and adjusted for state changes on a Function as a Service (FaaS) platform. On this FaaS platform, a task queue is used as middleware to obtain the load status of multiple execution entities, and the tasks are asynchronously allocated based on load balancing principles. The FaaS platform starts a distributed task scheduling component, synchronizing data between multiple nodes corresponding to multiple tasks through sharing, and using a distributed lock in the distributed component to elect a master node. During task scheduling, task scheduling is expanded through dynamic partitioning. Each task is bound to a partition, and each partition corresponds to a timer of the FaaS platform. When the number of tasks exceeds a certain threshold, a new partition is added, and the newly added tasks are bound to the new partition. In this embodiment, the FaaS platform uses the Etcd component to implement master node election in task scheduling. Etcd can solve the data sharing problem between multiple nodes, ensuring strong consistency of data across multiple nodes in the Etcd cluster through a protocol, and is used for distributed locks and leader election. Specifically, when a task on one node fails, a task on another node will elect a new service through Etch, thus ensuring that the task is not interrupted.

[0118] The execution module 404 is used to execute the tasks distributed by the task distribution middleware through the multiple task execution services. Multiple task execution modules are set up on the FaaS platform, and the tasks are asynchronously allocated based on load balancing principles. The scheduled tasks are assigned to execution components with lower load and within their capabilities. Typically, this execution module is a function module on the FaaS platform. When a task needs to be executed, the FaaS platform activates the execution module, projecting the function module's registry cache into the terminal system's registry without occupying physical space. The same applies to the file system, where file caches are projected into the terminal's file system without occupying physical space. Their mutual access and use of each other's registry entries and file sets proceed as normal. The tasks are executed periodically on a timer, and the execution of tasks is sequential; that is, the next cycle cannot begin until the current cycle of the same task is completed.

[0119] The update module 405 is used to update the execution and status of the tasks. In the task scheduling component of the FaaS platform, the execution and status of tasks are updated in real time to maintain real-time task invocation. In this embodiment, an Online Transaction Processing (OLTP) storage component is used as persistent storage to save the created scheduled tasks and their execution status. The in-memory OLTP engine is fully integrated into SQL Server, and these task objects reside in the in-memory OLTP engine. This integration allows access to data stored in memory-optimized tables and disk-based tables using standard Transact-SQL (interpreted T-SQL) calls. This integration helps minimize application changes when migrating to in-memory OLTP. This efficient OLTP storage method enables real-time task execution and status updates while ensuring storage stability.

[0120] Furthermore, the triggering module 402 is further configured to: initiate multiple tasks through the Function as a Service (FaaS) platform, and when a task running in the schedule fails, use a task that has not failed among the initiated multiple tasks to handle the failed task.

[0121] The distributed task scheduling device 400 further includes: a synchronization election module, used to start the task scheduling distributed component through the function-as-a-service platform, synchronize the data between multiple nodes corresponding to multiple tasks through sharing, and elect the master node through the distributed lock in the distributed component.

[0122] The distributed task scheduling device 400 further includes: an expansion module, used to expand the task scheduling capacity by dynamic partitioning during task scheduling, or to expand the message queue and add more partitions during task execution.

[0123] The expansion module is further used to: expand the message queue and add more partitions during task execution; wherein, based on the expansion capability of the Function as a Service platform, when the number of tasks increases, the Function as a Service platform will automatically add more tasks for task execution.

[0124] The distributed task scheduling device 400 further includes: a state storage module for saving the created timed tasks and the state of task execution, wherein an online transaction processing storage component is used as persistent storage.

[0125] The distribution module is further configured to: obtain the load status of multiple execution entities and asynchronously distribute the tasks based on the load balancing principle.

[0126] The distributed task scheduling device 400 further includes: an exception handling module, used to reprocess the exception task when an exception occurs during task scheduling.

[0127] The exception handling module is further configured to: if writing to the message queue fails or the task execution fails after the task is published, republish the task after the task execution exceeds a first time threshold; if the task exits abnormally during execution and remains in a pending state, republish the task after the pending state exceeds a second time threshold; if the task execution fails, republish the task in the next task cycle; or if the task execution is closed, update the task status to a paused state.

[0128] Figure 4 The device shown can perform Figure 1 For the methods shown in the embodiments, the parts not described in detail in this embodiment can be referred to the following: Figure 1 The relevant descriptions of the illustrated embodiments are provided below. For the execution process and technical effects of this technical solution, please refer to [link / reference]. Figure 1 The descriptions in the illustrated embodiments will not be repeated here.

[0129] The following is for reference. Figure 5 This illustration shows a structural schematic of an electronic device 500 suitable for implementing another embodiment of the present disclosure. The terminal devices in the embodiments of the present disclosure may include, but are not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 5 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0130] like Figure 5 As shown, electronic device 500 may include a processing unit (e.g., central processing unit, graphics processor, etc.) 501, which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 502 or a program loaded from storage device 508 into random access memory (RAM) 503. RAM 503 also stores various programs and data required for the operation of electronic device 500. Processing unit 501, ROM 502, and RAM 503 are interconnected via communication line 504. Input / output (I / O) interface 505 is also connected to communication line 504.

[0131] Typically, the following devices can be connected to I / O interface 505: input devices 506 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 507 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 508 including, for example, magnetic tapes, hard disks, etc.; and communication devices 509. Communication device 509 allows electronic device 500 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 5 An electronic device 500 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0132] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 509, or installed from a storage device 508, or installed from a ROM 502. When the computer program is executed by the processing device 501, it performs the functions defined in the methods of embodiments of this disclosure.

[0133] It should be noted that the computer-readable medium described in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0134] In some implementations, clients and servers can communicate using any currently known or future-developed network protocol, such as HTTP (Hypertext Transfer Protocol), and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future-developed networks.

[0135] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0136] The aforementioned computer-readable medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to perform the interaction method described in the above embodiments.

[0137] Computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including but not limited to object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0138] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0139] The units described in the embodiments of this disclosure can be implemented in software or hardware. The names of the units are not, in some cases, intended to limit the specific unit.

[0140] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application Standard Products (ASSPs), System-on-Chip (SoCs), Complex Programmable Logic Devices (CPLDs), and so on.

[0141] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0142] According to one or more embodiments of the present disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform any of the distributed task scheduling methods described in the first aspect above.

[0143] According to one or more embodiments of the present disclosure, a non-transitory computer-readable storage medium is provided, characterized in that the non-transitory computer-readable storage medium stores computer instructions for causing a computer to execute any of the distributed task scheduling methods described in the first aspect above.

[0144] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

Claims

1. A distributed task scheduling method, characterized in that, include: A scheduled service, a task distribution middleware, and a task execution service are set up on the Function as a Service platform. The scheduled service, the task distribution middleware, and the task execution service are deployed and run on the Function as a Service platform. Based on the expansion capability of the Function as a Service platform, when the number of tasks increases, the Function as a Service platform adds more task instances for task execution. The tasks that need to be periodically scheduled are triggered periodically by triggers in the timed scheduling service. The task distribution middleware asynchronously distributes the tasks to multiple task execution services in the form of message queues; The tasks issued by the task distribution middleware are executed through the multiple task execution services; Update the execution and status of the task.

2. The distributed task scheduling method according to claim 1, characterized in that, The method of periodically triggering tasks that require periodic scheduling through triggers in the timed scheduling service includes: The Function as a Service platform triggers multiple of the aforementioned tasks; When a task running in the schedule fails, the failed task is handled by one of the launched tasks that did not fail.

3. The distributed task scheduling method according to claim 1, characterized in that, The task distribution middleware asynchronously distributes the tasks to multiple task execution services in the form of message queues, including: Establish task message queues in chronological order of task release time; Obtain the load status of multiple task execution services, and asynchronously distribute tasks in the task message queue based on the load balancing principle.

4. The distributed task scheduling method according to claim 1, characterized in that, The task is executed periodically at set times, and the execution of the task is sequential; that is, the next cycle cannot be executed until the current cycle of the same task is completed.

5. The distributed task scheduling method according to claim 1, characterized in that, The method further includes: The Function as a Service platform initiates a task scheduling distributed component to synchronize data between multiple nodes corresponding to multiple tasks through sharing, and elects a master node through a distributed lock in the distributed component.

6. The distributed task scheduling method according to claim 1, characterized in that, The method further includes: During task scheduling, the task scheduling capacity is expanded through dynamic partitioning; Each task is bound to a partition, and each partition corresponds to a timer of the function i.e. service platform. When the number of tasks exceeds a certain threshold, a new partition is added, and the newly added tasks are bound to the new partition.

7. The distributed task scheduling method according to claim 1, characterized in that, The method further includes: During task execution, the message queue is expanded to include more partitions; Based on the scalability of the Function as a Service platform, when the number of tasks increases, the platform will automatically add more tasks for task execution.

8. The distributed task scheduling method according to claim 1, characterized in that, The method further includes: The online transaction processing (OLTP) storage component is used as persistent storage to save the created scheduled tasks and the state of task execution.

9. The distributed task scheduling method according to claim 1, characterized in that, The method further includes: When an anomaly occurs in the scheduling of the task, the abnormal task is reprocessed.

10. The distributed task scheduling method according to claim 9, characterized in that, The reprocessing of the abnormal task includes: If writing to the message queue fails or the task fails to execute after the task is published, the task will be republished after the task execution exceeds a first time threshold. If the task exits abnormally during execution and remains in a pending state, the task will be re-released after the pending state exceeds a second time threshold. If the task fails, it will be reissued in the next task cycle; or If the task execution is stopped, the task status is updated to paused.

11. A distributed task scheduling device, characterized in that, include: The configuration module is used to configure a timed scheduling service, a task distribution middleware, and a task execution service on the Function as a Service platform. The timed scheduling service, the task distribution middleware, and the task execution service are deployed and run on the Function as a Service platform. Based on the scalability of the Function as a Service platform, when the number of tasks increases, the Function as a Service platform adds more task instances for task execution. The triggering module is used to periodically trigger tasks that need to be scheduled through triggers in the timed scheduling service. The task distribution module is used by the task distribution middleware to asynchronously distribute the task to multiple task execution services in the form of a message queue. The execution module is used to execute the tasks issued by the task distribution middleware through the multiple task execution services; The update module is used to update the execution and status of the task.

12. The distributed task scheduling device according to claim 11, characterized in that, The device further includes: The synchronization election module is used by the Function as a Service platform to start the task scheduling distributed component, synchronize the data between multiple nodes corresponding to multiple tasks through sharing, and elect the master node through the distributed lock in the distributed component.

13. The distributed task scheduling device according to claim 11, characterized in that, The device further includes: The expansion module is used to expand the task scheduling capacity through dynamic partitioning during task scheduling, or to expand the message queue and add more partitions during task execution.

14. The distributed task scheduling device according to claim 11, characterized in that, The device further includes: The exception handling module is used to reprocess the exception task when an exception occurs in the scheduling of the task.

15. An electronic device comprising: Memory, used to store computer-readable instructions; as well as A processor for executing the computer-readable instructions, causing the electronic device to perform the method according to any one of claims 1-10.

16. A computer-readable storage medium for storing computer-readable instructions that, when executed by a computer, cause the computer to perform the method of any one of claims 1-10.

Images