Task scheduling method and device, storage medium and computer program product

Abstract

The invention discloses a task scheduling method and device, a storage medium and a computer program product, and relates to the technical field of batch data processing, and the method comprises the steps: obtaining a batch processing task initiation instruction, and registering a scheduling task in a tree structure management mode according to the batch processing task initiation instruction; registering the scheduling task to a message queue or a background thread to perform sub-library batch processing execution to obtain a sub-library batch processing execution result; updating a tree scheduling task control record according to the sub-library batch processing execution result; and synchronizing the tree-shaped scheduling task control record to a scheduling library to trigger a next batch processing scheduling task. Parallel scheduling of multi-sub-library batch processing tasks is achieved through sub-library batch processing execution and updating of tree-shaped scheduling task control records, and the batch processing performance is effectively improved.

Description

Technical Field

[0001] The present application relates to the technical field of batch data processing, and in particular to a task scheduling method, device, storage medium and computer program product. Background Art

[0002] With the advent of the Internet and the era of big data, the scale of data has expanded dramatically, placing higher demands on data storage and processing capabilities. In order to achieve high availability of services and improve database performance, the distributed cloud architecture manages data in separate databases. In particular, for large amounts of data, multiple sub-databases are used for storage, and different application services are deployed in different sub-databases. However, serial batch processing or summary statistics of data from each sub-database cannot effectively utilize the advantages of multiple sub-databases and execute batch processing tasks in parallel, resulting in poor batch processing performance. In addition, the serial processing method is also difficult to deal with interface timeouts and trigger batch processing tasks for multiple sub-databases, affecting the execution of the overall batch processing tasks.

[0003] Therefore, how to parallelize the batch processing tasks of multiple databases to effectively improve the batch processing performance has become a technical problem that needs to be solved urgently in this application.

[0004] The above contents are only used to assist in understanding the technical solution of the present application and do not constitute an admission that the above contents are prior art. Summary of the invention

[0005] The main purpose of this application is to provide a task scheduling method, device, storage medium and computer program product, aiming to solve the technical problem of how to parallel schedule multi-library batch processing tasks to effectively improve batch processing performance.

[0006] To achieve the above objectives, the present application proposes a task scheduling method, which includes:

[0007] Obtaining a batch task initiation instruction and registering a scheduling task in a tree structure management manner according to the batch task initiation instruction;

[0008] Register the scheduling task to the message queue or background thread for batch processing and execution, and obtain the batch processing execution result;

[0009] Update the tree-like scheduling task control record according to the sub-library batch processing execution result;

[0010] Synchronize the tree-like scheduling task control record to the scheduling library to trigger the next batch of processing scheduling tasks.

[0011] In one embodiment, the step of obtaining a batch task initiation instruction and registering a scheduling task in a tree structure management manner according to the batch task initiation instruction includes:

[0012] Get the batch task initiation instruction;

[0013] Calling a registration scheduling task interface according to the batch task initiation instruction, and generating a tree-like scheduling task control record in a tree-like structure management manner through the registration scheduling task interface;

[0014] Writing the tree-like scheduling task control record into the scheduling database and determining whether cross-database processing is required;

[0015] If cross-database processing is required, the tree-like scheduling task control record is synchronized to the sub-database for scheduling task registration;

[0016] If cross-database processing is not required, directly register the scheduling task.

[0017] In one embodiment, the step of registering the scheduling task to a message queue or a background thread for database-based batch processing and obtaining database-based batch processing execution results includes:

[0018] Registering the scheduling task to a message queue or a background thread to configure a message queue consumer or a thread pool;

[0019] The corresponding sub-library batch processing execution program is called through the message queue consumer or the thread pool to execute the sub-library batch processing and obtain the sub-library batch processing execution result.

[0020] In one embodiment, the step of calling the corresponding sub-library batch processing execution program through the message queue consumer or the thread pool to execute sub-library batch processing includes:

[0021] When the execution time of the sub-database batch processing exceeds the execution alarm duration, the tree-like scheduling task control record corresponding to the scheduling task is kept in a pending execution state, and a timeout non-execution alarm and an automatic retry mechanism are set.

[0022] In one embodiment, the step of updating the tree-like scheduling task control record according to the sub-library batch processing execution result includes:

[0023] Update the current scheduling task control record result according to the sub-library batch processing execution result and determine whether there is a first upper-level scheduling task;

[0024] When there is a first upper-layer scheduling task, the number of successful subtasks and the number of failed subtasks of the first upper-layer scheduling task are updated upward based on the control record result of the current scheduling task to obtain the control record result of the first upper-layer scheduling task;

[0025] Determine whether there is a second upper-layer scheduling task according to the control record result of the first upper-layer scheduling task;

[0026] When there is a second upper-layer scheduling task, the number of successful subtasks and the number of failed subtasks of the second upper-layer scheduling task are updated upward based on the control record result of the first upper-layer scheduling task to obtain the control record result of the second upper-layer scheduling task;

[0027] The tree-shaped scheduling task control record is updated according to the current scheduling task control record result, the first upper-layer scheduling task control record result, and the second upper-layer scheduling task control record result.

[0028] In one embodiment, the step of synchronizing the tree-like scheduling task control record to the scheduling library to trigger the next batch of processing scheduling tasks includes:

[0029] Synchronize the tree-like scheduling task control record to the scheduling library to update the scheduling library task control record in a tree-like management manner;

[0030] The next batch of processing tasks is automatically registered as a sibling task or a subtask of the scheduling task.

[0031] In one embodiment, the step of synchronizing the tree-like scheduling task control record to the scheduling library to update the scheduling library task control record in a tree-like management manner includes:

[0032] When the synchronization execution result of the tree-like scheduling task control record synchronized to the scheduling library is abnormal, an automatic retry mechanism is started, and the synchronization execution result includes execution time and synchronization result;

[0033] When the synchronization execution result of the tree-like scheduling task control record to the scheduling library is normal, the scheduling library task control record is updated in a tree-like management manner. In addition, to achieve the above purpose, the present application also proposes a task scheduling device, which includes:

[0034] A scheduling task registration module is used to obtain a batch task initiation instruction and register a scheduling task according to the batch task initiation instruction;

[0035] The sub-database batch processing execution module is used to register the scheduling task into the message queue for sub-database batch processing execution and obtain the sub-database batch processing execution result;

[0036] A control record updating module, used for updating the tree-like scheduling task control record according to the sub-library batch processing execution result;

[0037] The next batch of processing scheduling tasks triggering module is used to synchronize the tree-like scheduling task control records to the scheduling library to trigger the next batch of processing scheduling tasks.

[0038] In addition, to achieve the above objectives, the present application also proposes a storage medium, which is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the task scheduling method described above are implemented.

[0039] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, the steps of the task scheduling method described above are implemented.

[0040] One or more technical solutions proposed in this application have at least the following technical effects:

[0041] The scheduling task is registered by initiating the command of the acquired batch task. The registered scheduling task supports tree structure management (parent-child-grandchild), allowing complex batch tasks to be broken down into smaller, easy-to-manage subtasks. The scheduling task is registered to the message queue or background thread for batch execution of the sub-database. By sending the task to the message queue, each database or database partition can independently and simultaneously process its assigned tasks, thereby significantly improving the overall performance of batch processing. After the batch processing of the sub-database is completed, the tree-like scheduling task control record is updated according to the execution result, the execution status of each task is grasped in real time, and accurate data support is provided for subsequent processing. After the tree-like scheduling task control record of the sub-database is synchronized back to the scheduling library, the next batch processing scheduling task is triggered and registered according to user needs, thereby realizing the continuous execution and automatic management of batch processing tasks, and further improving processing efficiency. In this way, parallel scheduling of batch processing tasks of multiple sub-databases is realized and batch processing performance is effectively improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0043] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0044] Figure 1 A flowchart of the first embodiment of the task scheduling method of the present application is provided;

[0045] Figure 2 A tree-like scheduling task diagram provided for the task scheduling method of this application;

[0046] Figure 3A schematic diagram of a second embodiment of the task scheduling method provided in this application;

[0047] Figure 4 A schematic diagram of a third embodiment of the task scheduling method provided in this application;

[0048] Figure 5 A brief flowchart of updating a tree-like scheduling task control record for a task scheduling method provided in the fourth embodiment of the present application;

[0049] Figure 6 This is a complete flowchart of the task scheduling method for this application;

[0050] Figure 7 This is a schematic diagram of the module structure of the task scheduling device according to an embodiment of the present application;

[0051] Figure 8 Schematic diagram of the device structure of the hardware operating environment involved in the task scheduling method in the embodiment of the present application.

[0052] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0053] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0054] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0055] The main solution of the embodiment of the present application is: register the scheduling task in a tree-like management manner according to the acquired batch task initiation instruction, register the scheduling task to the message queue or background thread for parallel sub-library batch execution, and obtain the sub-library batch execution result. The sub-library batch execution result includes: execution failure and execution success, and the tree-like scheduling task control record is updated upward in a tree-like management manner according to the sub-library batch execution result. The tree-like scheduling task control record of the sub-library is updated to the scheduling library to trigger the next batch of scheduling tasks.

[0056] The embodiments of the present application take into account that: with the advent of the Internet and the era of big data, the scale of data has expanded dramatically, placing higher demands on data storage and processing capabilities. In order to achieve high availability of services and improve database performance, the distributed cloud architecture manages data in sub-libraries. In particular, for larger volumes of data, multiple sub-libraries are split for storage, and different application services are deployed in different sub-libraries. However, serial batch processing or summary statistics of sub-library data one by one cannot effectively utilize the advantages of multiple sub-libraries and execute batch processing tasks in parallel, resulting in poor batch processing performance. In addition, the serial processing method is also difficult to deal with interface timeouts and trigger batch processing tasks in multiple sub-libraries, affecting the execution of the overall batch processing tasks.

[0057] Therefore, the present application provides a solution, which registers the scheduling task through the acquired batch task initiation instruction. The registered scheduling task supports tree structure management (parent-child-grandchild), allowing complex batch tasks to be disassembled into smaller, easy-to-manage subtasks. The scheduling task is registered to the message queue or background thread for sub-library batch execution. By sending the task to the message queue, each database or database partition can independently and simultaneously process its assigned tasks, thereby significantly improving the overall performance of the batch processing. After the sub-library batch processing is completed, the tree-like scheduling task control record is updated according to the execution result, the execution status of each task is grasped in real time, and accurate data support is provided for subsequent processing. After the tree-like scheduling task control record of the sub-library is synchronized back to the scheduling library, the next batch of processing scheduling tasks is triggered and registered according to user needs, thereby realizing the continuous execution and automated management of batch processing tasks, and further improving the processing efficiency. Then, parallel scheduling of multi-library batch processing tasks is realized and batch processing performance is effectively improved.

[0058] It should be noted that the execution subject of this embodiment can be a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device capable of realizing the above functions, a task scheduling system, etc. The following takes the task scheduling system as an example to illustrate this embodiment and the following embodiments.

[0059] Based on this, the present application embodiment provides a task scheduling method, referring to Figure 1 , Figure 1 This is a flowchart of the first embodiment of the task scheduling method of the present application.

[0060] In this embodiment, the task scheduling method includes steps S10 to S40:

[0061] Step S10, obtaining a batch task initiation instruction and registering a scheduling task in a tree structure management manner according to the batch task initiation instruction;

[0062] The batch task initiation instruction contains the key information required for batch processing, such as batch type, target database, data range, execution time, etc. Obtaining the batch task initiation instruction is the starting point for triggering the entire scheduling process. The batch task initiation instruction comes from the business system, user interface (UI), API call or other automation tools. Users send batch task initiation instructions to the task scheduling system through the business system, user interface (UI), API call or other automation tools. The task scheduling system listens, polls, and responds to specific events to obtain the batch task initiation instruction.

[0063] Furthermore, after obtaining the batch task initiation instruction, the task scheduling system will verify the instruction legitimacy of the batch task initiation instruction, parse the parameters and options in the instruction, and extract the detailed information required for executing the batch processing.

[0064] Finally, the scheduling task is created in a tree-structured management mode based on the detailed information extracted from the instruction. The scheduling task supports a tree-structured management mode (parent-child-grandchild), which breaks down complex batch tasks into smaller, easy-to-manage subtasks. The tree structure is a data structure that is very suitable for representing hierarchies and dependencies. In the task scheduling system, the entire task set is regarded as a tree, where each node represents a task, and the lines between nodes represent the dependencies between tasks.

[0065] Based on the extracted detailed information, the task scheduling system starts to create scheduled tasks. Specifically, each task is created as a node in a tree and placed in an appropriate position according to its dependencies. Based on the basic information and execution parameters of the task, the task object is instantiated. Based on the dependencies between tasks, the task nodes are connected in a hierarchical relationship to form a tree structure. Scheduling rules are configured for each task, such as timed execution, event-based triggering, etc., to complete the registration of the scheduled task.

[0066] Step S20, registering the scheduling task to a message queue or a background thread for batch processing in a sub-library to obtain a batch processing execution result;

[0067] For example, if you choose the message queue as the distribution method, since the scheduling task has been registered and the relevant parameters have been configured, the task scheduling system will encapsulate the scheduling task into a message and send it to the message queue. It should be noted that the message queue is an asynchronous processing mechanism that can decouple the producer and consumer of the task and improve the scalability and reliability of the system. When there are available messages in the message queue, the consumer of the message queue (usually the batch executor of the sub-library) will consume these messages and execute the corresponding batch tasks.

[0068] In addition, if you choose background thread as the distribution method, the task scheduling system will submit the scheduled task to the background thread pool for management. The background thread pool is responsible for maintaining a group of worker threads, which will continuously take tasks from the task queue and execute them. When a scheduled task is submitted, the background thread pool will assign an idle thread to execute the task, or put the task into the task queue to wait for execution.

[0069] It is understandable that no matter whether you choose message queue or background thread as the distribution method, the ultimate goal is to distribute the scheduling task to each sub-database to perform batch operations. The sub-database batch execution program will connect to the corresponding database according to the information in the task control record and perform the specified batch operation. Batch operations may include data query, data update, data aggregation and other types, depending on the needs of the scheduling task.

[0070] After the database batch processing is completed, the execution program will generate the execution results. The database batch processing execution results include the number of records successfully processed, the number of records failed to be processed, exception information, etc. The database batch processing execution results will be recorded and may be fed back to the user through methods such as database update and message sending.

[0071] Step S30, updating the tree-like scheduling task control record according to the sub-library batch processing execution result;

[0072] The sub-library batch processing tasks have been distributed to each sub-library and executed, and the corresponding execution results have been generated. Since there are sub-library batch processing execution results for multiple sub-libraries, it is necessary to continuously update the tree-like scheduling task control records upward according to the sub-library batch processing execution results.

[0073] The task scheduling system first needs to parse the batch execution results of the sub-database. The purpose of the analysis is to extract information directly related to the scheduling task, such as whether the task is successfully executed, how many data are processed, whether an error is encountered, etc.

[0074] Furthermore, based on the task identifier (such as task ID) or other relevant information in the execution result, the scheduling system needs to find the corresponding scheduling task in its internal task control record. If the scheduling task is managed in a tree structure, it is necessary to further locate the specific position of the task in the tree structure. Based on the parsed execution results, the scheduling system needs to update the status information in the task control record. The status information includes whether the task is completed, whether it is successful, the reason for failure, etc. In addition, in addition to the status information, the scheduling system also needs to update other important information in the execution results (such as the amount of data processed, exception information, etc.) to the task control record.

[0075] It is understandable that if the scheduling task is managed in a tree structure, then when a subtask is executed and its task control record is updated, the task scheduling system also needs to check and update the control record of its parent task. This usually involves updating the statistical information such as the number of successful subtasks and the number of failed subtasks of the parent task, so as to more accurately reflect the status of the entire task tree, thereby completing the update of the entire tree-like scheduling task control record.

[0076] Step S40, synchronizing the tree-like scheduling task control record to the scheduling library to trigger the next batch of processing scheduling tasks;

[0077] The scheduling library is one of the core components of the task scheduling system, responsible for storing task control records, scheduling rules, execution history and other information. The scheduling library not only stores task control records, but also contains the logic or rules for triggering the next batch of processing scheduling tasks.

[0078] When new control records are synchronized to the scheduling library, these rules will be triggered and executed. For example, if a task has been successfully completed, and according to the scheduling rules, there are subsequent tasks that depend on the completion result of this task, then these subsequent tasks will be added to the queue to be executed.

[0079] Specifically, since there are usually one or more task queues in the task scheduling system to manage the tasks to be executed. After the synchronous tree-structured scheduling task control is recorded in the scheduling library, the task scheduling system will add the next batch of processing scheduling tasks to be executed to the corresponding queue according to the scheduling rules and the dependencies between tasks. The next batch of processing scheduling tasks will wait for execution and resource allocation according to the order or priority in the queue.

[0080] This embodiment provides a task scheduling method, which registers the scheduling task through the acquired batch task initiation instruction. The registered scheduling task supports tree structure management (parent-child-grandchild), allowing complex batch tasks to be disassembled into smaller, easy-to-manage subtasks. The scheduling task is registered to the message queue or background thread for sub-library batch execution. By sending the task to the message queue, each database or database partition can independently and simultaneously process its assigned tasks, thereby significantly improving the overall performance of the batch processing. After the sub-library batch processing is completed, the tree-like scheduling task control record is updated according to the execution result, the execution status of each task is grasped in real time, and accurate data support is provided for subsequent processing. After the tree-like scheduling task control record of the sub-library is synchronized back to the scheduling library, the next batch processing scheduling task is triggered and registered according to user needs, thereby realizing the continuous execution and automated management of batch processing tasks, and further improving the processing efficiency. In this way, parallel scheduling of multi-library batch processing tasks is realized and batch processing performance is effectively improved.

[0081] Specifically, in a feasible implementation, step S10 may include steps S11 to S15:

[0082] Step S11, obtaining a batch task initiation instruction;

[0083] Batch task initiation instructions can come from multiple data sources, including but not limited to:

[0084] User interface: Users can manually trigger tasks through the graphical user interface (GUI) or command line interface (CLI); Scheduled tasks: The task scheduling system can automatically trigger tasks according to a preset schedule, such as performing database backup every morning; External systems: Other systems or applications can send task initiation instructions to the task scheduling system through API calls, message queues, etc.; Triggers: Trigger task execution based on specific events (such as file arrival, database changes, etc.).

[0085] Once the task initiator is identified, the task scheduling system obtains the batch task initiation instructions in the following ways: Listening port or queue: The task scheduling system can listen to a specific network port or message queue to receive instructions from an external system; Scheduled check: For scheduled tasks, the task scheduling system can periodically check the schedule to determine if there are any tasks to be executed. Event monitoring: For tasks triggered by events, the task scheduling system needs to be able to monitor and respond to specific events.

[0086] Step S12, calling the registration scheduling task interface according to the batch task initiation instruction, and generating a tree-like scheduling task control record in a tree-like structure management manner through the registration scheduling task interface;

[0087] The task scheduling system provides a programming interface (API) for registering task information into the system and generating corresponding task control records. The task scheduling system calls the task scheduling interface and passes a series of parameters for registering the scheduling task to the task scheduling interface. The parameters for registering the scheduling task usually include the basic information of the task (such as task ID, task name, execution time, etc.), execution parameters (such as input data, configuration options, etc.) and any necessary dependencies or preconditions. After receiving these parameters, the interface will perform a series of verification operations to ensure the validity of the parameters and the feasibility of the task.

[0088] After verification, the interface will generate one or more tree-like scheduling task control records based on the passed parameters. These control records are data structures used within the task scheduling system to track and manage the task execution status. Figure 2 As shown, Figure 2A tree-like scheduling task diagram provided for the task scheduling method of this application. In the tree-like scheduling model, a task may contain multiple subtasks, and there may be dependencies between these subtasks. Therefore, the control record needs to be able to reflect this tree structure and dependency.

[0089] It should be noted that the tree-like scheduling task control record contains the following information: basic task information: such as task ID, task name, task type, execution time, etc.; execution parameters: parameters required for task execution, such as input data, configuration options, etc.; status information: the current status of the task, such as pending, executing, completed, failed, etc.; dependency: the dependency between tasks, including parent tasks, child tasks, grandchild tasks, and the execution order between them. Execution result: the result information after the task is executed, such as the status code of success or failure, the amount of data processed, exception information, etc. (the above information is filled in after the task is executed).

[0090] Step S13, writing the tree-like scheduling task control record into the scheduling library and determining whether cross-library processing is required;

[0091] In order to store the detailed information and execution status of the scheduled tasks persistently, so that the task status can be retrieved after the task scheduling system is restarted or recovered from a fault, and to ensure the continuity and reliability of task execution, the tree-like scheduling task control records need to be written into the scheduling library.

[0092] After generating the tree-like scheduling task control records, these records need to be organized into a format acceptable to the database, including fields such as task ID, task name, status, execution parameters, dependencies, etc. Further, establish a database connection with the scheduling library to ensure that data can be written to the scheduling library. Through database operations (such as INSERT statements), write the tree-like scheduling task control records into the corresponding tables in the scheduling library. It should be noted that this process may need to deal with complex situations such as concurrent writing and transaction control to ensure the accuracy and consistency of the data. Therefore, it is necessary to reasonably allocate computing resources to cope with complex situations such as concurrent writing and transaction control. During the writing process, if any errors are encountered (such as database connection failure, write conflict, etc.), corresponding error handling is required, such as retrying writing and recording error logs.

[0093] In addition, based on the characteristics of the task (such as data volume, access frequency, dependencies, etc.) and the performance of the database (such as processing power, storage capacity, network latency, etc.), decide whether to assign the task to different database instances (i.e., sub-databases) for processing to improve the overall performance and scalability of the task scheduling system.

[0094] For example, if the amount of data processed by a task is very large, a single database instance may not be able to withstand such a high load, and cross-database processing may be considered. If there are complex dependencies between tasks, and these dependencies span different business modules or data domains, cross-database processing may help better manage these dependencies.

[0095] Step S14, if cross-sub-library processing is required, the tree-shaped scheduling task control record is synchronized to the sub-library to register the scheduling task;

[0096] When it is determined that a tree-like scheduling task needs to be processed across databases, the task scheduling system will synchronize the control record of the task to the specified database. The synchronization process mainly involves copying the control record data to another database instance, or distributing the data to different database instances according to a certain strategy (such as sharding, partitioning).

[0097] Once the control record is successfully synchronized to the sub-library, the task scheduling system will register the scheduling task in the sub-library. This means that the task will be identified, managed and executed in the scheduling environment of the sub-library. The registration process includes inserting the task record in the corresponding table of the sub-library, updating the task status, setting the execution time, etc.

[0098] Step S15: If cross-database processing is not required, the scheduling task registration is performed directly.

[0099] If it is determined that a tree-like scheduling task does not need to be processed across sub-databases (i.e. the task can be efficiently processed in the current database instance), the task scheduling system will directly register the task in the current scheduling environment. This means that the control record of the task will be directly inserted into the corresponding table of the current database instance, and the execution, monitoring and management of the task will all be carried out in this instance.

[0100] During this process, the task scheduling system does not need to perform additional data synchronization or distribution operations, because all necessary information already exists in the current database instance. The registration process also includes steps such as inserting task records into the table, updating task status, and setting execution time, but unlike cross-database processing, these operations are all completed in the same database instance.

[0101] Based on the first embodiment of the present application, the second embodiment of the present application is proposed. In the second embodiment of the present application, the same or similar contents as those of the above-mentioned embodiment 1 can be referred to the above introduction, and will not be repeated in the following.

[0102] On this basis, please refer to Figure 3 , Figure 3 A flowchart of a second embodiment of the task scheduling method provided in this application.

[0103] In this embodiment, the step S20 of registering the scheduling task to a message queue or a background thread for database batch processing execution and obtaining the database batch processing execution result may include steps S21 to S22:

[0104] Step S21, registering the scheduling task to a message queue or a background thread to configure a message queue consumer or a thread pool;

[0105] Register the relevant information of the scheduled task to be executed (such as task ID, execution time, parameters, etc.), and choose to register the scheduled task to the message queue or background thread pool based on the system architecture and performance requirements.

[0106] The message queue decouples the initiator and executor of the task, so that the production and consumption of the task can be carried out independently without interfering with each other; after the task is sent to the message queue, the result can be returned immediately without waiting for the actual execution of the task to be completed; the message queue can automatically assign tasks to multiple consumers for processing, thereby achieving load balancing.

[0107] The thread pool reduces the number of thread creation and destruction by reusing threads, thereby improving performance. The thread pool can limit the number of threads executed simultaneously to avoid performance degradation caused by too many threads competing for system resources.

[0108] For example, if the task scheduling system requires high reliability and decoupling to better execute parallel scheduling tasks, a message queue is usually selected. Task information is encapsulated into messages and sent to the message queue, which is processed asynchronously by the consumer of the message queue (which may be an independent process or service). In addition, if the task scheduling system has high real-time requirements or the task volume is not particularly large, a thread pool can be considered. Tasks are submitted to the thread pool and executed by threads in the thread pool.

[0109] Configure the corresponding message queue consumer or thread pool according to the specific mechanism selected to ensure that tasks can be received and executed correctly.

[0110] Step S22, calling the corresponding sub-library batch processing execution program through the message queue consumer or the thread pool to execute the sub-library batch processing and obtain the sub-library batch processing execution result.

[0111] Both message queue consumers and threads in the thread pool can process multiple tasks in parallel. For multi-database batch processing tasks, the task scheduling system splits the task into multiple subtasks, each of which corresponds to a database or part of a database, and then executes these subtasks in parallel. Message queue consumers receive tasks from message queues, or threads in thread pools get tasks from task queues, and determine task types, parameters, etc. by parsing task information, and then determine the sub-database batch execution program that needs to be executed.

[0112] Furthermore, according to the parsed task information, the corresponding sub-database batch execution program is called. These programs will connect to the corresponding database, execute batch processing operations, and obtain records or process the results of batch execution.

[0113] In this embodiment, the scheduling task is registered to the message queue or background thread to configure the message queue consumer or thread pool; the corresponding sub-library batch execution program is called through the message queue consumer or thread pool to execute the sub-library batch processing and obtain the sub-library batch processing execution result. Among them, the message queue can automatically assign tasks to multiple consumers for processing to achieve load balancing; the threads in the thread pool can also be dynamically adjusted according to factors such as task priority and execution time to optimize performance. Under the joint effect of decoupling and asynchronous processing, parallel execution, load balancing and other measures, the task scheduling system can efficiently handle large-scale, high-concurrency batch processing tasks and improve overall performance and user experience.

[0114] Based on the first embodiment and / or the second embodiment of the present application, the third embodiment of the present application is proposed. In the third embodiment of the present application, the same or similar contents as those of the first embodiment and / or the second embodiment can be referred to the above introduction, and will not be repeated in the following.

[0115] On this basis, please refer to Figure 4 , Figure 4 A flowchart of the third embodiment of the task scheduling method provided in this application.

[0116] In this embodiment, the corresponding sub-library batch processing execution program is called by the message queue consumer or the thread pool to execute the sub-library batch processing, and after step S22 of obtaining the sub-library batch processing execution result, the following steps are further included:

[0117] Step S23, when the execution time of the sub-library batch processing exceeds the execution alarm duration, the tree-like scheduling task control record corresponding to the scheduling task is kept in a pending execution state, and a timeout non-execution alarm and an automatic retry mechanism are triggered.

[0118] When the task scheduling system detects that the execution time of a sub-database batch processing task exceeds the set alarm duration, the task status in the scheduling system is first updated to "pending" or similar status. This is to prevent the system from mistakenly believing that the task has succeeded or failed because the task has not been completed for a long time, which may further lead to data inconsistency or process interruption.

[0119] Furthermore, the task scheduling system will trigger a timeout alarm. This alarm can be sent to relevant personnel in a variety of ways, such as email, SMS, instant message, etc. The alarm content usually includes basic information of the task (such as task ID, task name, system, etc.), timeout time, possible reasons and recommended solutions, etc., to help relevant personnel understand the status of the task in a timely manner and take corresponding actions.

[0120] In addition to the timeout alarm, the task scheduling system will also try to automatically retry the sub-database batch processing task. Automatic retry can solve the task execution failure caused by temporary factors (such as temporary database unavailability, network fluctuations, etc.) to a certain extent. The number and interval of retries can be configured according to the characteristics of the task and the actual situation of system resources. If the task still cannot be successfully executed after multiple retries, the task scheduling system may need further processing, such as manual intervention or marking it as a failed state.

[0121] In this embodiment, when the execution time of the sub-library batch processing exceeds the execution alarm duration, the tree-like scheduling task control record corresponding to the scheduling task is maintained in a pending state, and the timeout non-execution alarm and automatic retry mechanism are triggered. Through the automatic retry mechanism, the task scheduling system can cope with temporary failures to a certain extent and ensure the final completion of the task. Through the timeout non-execution alarm, the execution status of the task is notified to the relevant personnel in a timely manner, helping to quickly locate the problem and take action. By triggering a series of measures such as the timeout non-execution alarm and the automatic retry mechanism, the reliability and stability of the scheduling task are ensured, and the overall performance of the system is also improved.

[0122] Based on the above embodiments of the present application, a fourth embodiment of the present application is proposed. In the fourth embodiment of the present application, the same or similar contents as those of the above embodiments can be referred to the above introduction, and will not be described in detail later.

[0123] In this embodiment, the step S30 of updating the tree-like scheduling task control record according to the sub-library batch processing execution result may include steps S31 to S35:

[0124] Step S31, updating the current scheduling task control record result according to the sub-library batch processing execution result and determining whether there is a first upper-level scheduling task;

[0125] First, the control record result of the current scheduling task (i.e., the scheduling task that executes the batch processing of the sub-library) is updated according to the actual execution result (success, failure, partial success, etc.) of the batch processing of the sub-library. Since in the tree-like task structure, each task may have one or more parent tasks, these parent tasks rely on the execution results of their child tasks to complete their own execution logic. Therefore, further check whether the current scheduling task has a direct upper-level scheduling task, that is, the current task is its child task and the first upper-level scheduling task is the parent task.

[0126] Step S32, when there is a first upper-level scheduling task, the number of successful subtasks and the number of failed subtasks of the first upper-level scheduling task are updated upward based on the control record result of the current scheduling task to obtain the control record result of the first upper-level scheduling task;

[0127] When there is a first upper scheduling task, the control record of the first upper scheduling task is updated according to the result (success or failure) of the current scheduling task, and the number of successful subtasks or failed subtasks of the upper scheduling task is increased, depending on the result of the current scheduling task.

[0128] Step S33, judging whether there is a second upper-layer scheduling task according to the control record result of the first upper-layer scheduling task;

[0129] Since the dependencies between tasks in a complex task tree may be very deep, a task may indirectly depend on multiple upper-level tasks. Similarly, after updating the control record of the first upper-level scheduling task, check whether this upper-level task has its own upper-level task (i.e., the second upper-level scheduling task). The second upper-level scheduling task is the parent task of the first upper-level scheduling task.

[0130] Step S34, when there is a second upper-layer scheduling task, the number of successful subtasks and the number of failed subtasks of the second upper-layer scheduling task are updated upward based on the control record result of the first upper-layer scheduling task to obtain the control record result of the second upper-layer scheduling task;

[0131] When there is a second upper scheduling task, the control record of the second upper scheduling task is updated according to the result (success or failure) of the current scheduling task, and the number of successful subtasks or failed subtasks of the upper scheduling task is increased, depending on the result of the current scheduling task.

[0132] Step S35 , updating the tree-like scheduling task control record according to the current scheduling task control record result, the first upper-layer scheduling task control record result, and the second upper-layer scheduling task control record result.

[0133] Updating the tree-like scheduling task control record is a key step to ensure the consistency and accuracy of all task status in the task tree. Therefore, it is necessary to update the control record of the entire tree-like scheduling task based on the control record results of all related scheduling tasks (including the current task, the first upper-level task, and the second upper-level task, if any) in order to correctly manage the dependencies and execution order between tasks.

[0134] In this embodiment, the tree-like scheduling task control record is gradually updated upward through the currently processed batch task, focusing on the update of task status and the management of dependency relationships to achieve parallel scheduling of multi-database batch tasks.

[0135] For example, in order to help understand the implementation process of the task scheduling method obtained by combining the fourth embodiment of the present application with the first embodiment, please refer to Figure 5 , Figure 5 A brief flowchart of a task scheduling method is provided, specifically:

[0136] like Figure 5 As shown, the batch processing of the sub-database is completed, including successful and failed processing, and the control record results of the current scheduling task G1 are updated. It supports the management of execution results of scheduling tasks in a "parent-child-grandchild" tree structure. After updating the control record results of the current scheduling task, it will look up for the task S1 on the previous level. If it exists, the number of successful subtasks / failed subtasks on task S1 will be updated. Similarly, after all the subtasks of task S1 are executed, it will also look up for the task F1 on the previous level of task S1. If it exists, the number of successful subtasks / failed subtasks on task F1 will be updated.

[0137] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the task scheduling method of the present application. More simple transformations based on this technical concept are all within the scope of protection of the present application.

[0138] Based on the above embodiments of the present application, a fifth embodiment of the present application is proposed. In the fifth embodiment of the present application, the same or similar contents as those of the above embodiments can be referred to the above introduction, and will not be described in detail later.

[0139] In this embodiment, the step S40 of synchronizing the tree-like scheduling task control record to the scheduling library to trigger the next batch of processing scheduling tasks includes steps S41-S42:

[0140] Step S41, synchronizing the tree-like scheduling task control record to the scheduling library to update the scheduling library task control record in a tree-like management manner;

[0141] The control records of the current batch task (including task status, execution results, subtask information, etc.) are safely stored in the scheduling library, and the task control records in the scheduling library are updated to reflect the latest task status. This ensures the state consistency of the task scheduling system and provides accurate data support for subsequent task scheduling.

[0142] Specifically, in a feasible implementation manner, step S41 may include steps S411 to S412:

[0143] Step S411, when the synchronization execution result of the tree-like scheduling task control record to the scheduling library is abnormal, an automatic retry mechanism is started, and the synchronization execution result includes execution time and synchronization result;

[0144] When performing synchronization operations, you may encounter various abnormal situations, such as network failure, database lock waiting, database write timeout, etc. In order to ensure the integrity and consistency of the data, the task scheduling system needs to be able to automatically retry when the synchronization execution result is abnormal. Specifically, the task scheduling system sets a retry strategy, such as automatically retrying the synchronization operation at a certain time interval (such as retrying immediately, retrying after a few seconds, retrying after a few minutes, etc.) until the synchronization is successful or the preset retry limit is reached.

[0145] Step S412: when the synchronization result of the tree-like scheduling task control record to the scheduling library is normal, the scheduling library task control record is updated in a tree-like management manner.

[0146] If the synchronization operation is successfully executed without any exceptions, the task scheduling system will update the task control record in the scheduling library in a tree-like management manner. The "tree-like management manner" here means that the system organizes and manages task control records through the parent-child or sibling relationship between tasks. For example, a scheduling task may have multiple subtasks, and these subtasks have their own subtasks, forming a tree structure. The task scheduling system will update the control record of each task in the scheduling library based on this relationship between tasks, including information such as the task status, execution results, and execution time.

[0147] Step S42, automatically registering the next batch of processing tasks as sibling tasks or subtasks of the scheduling task.

[0148] After successfully updating the task control record in the scheduling library, the task scheduling system needs to automatically register the next batch of processing tasks as sibling tasks or subtasks of the corresponding scheduling task based on the current task status and business logic. This step is one of the keys to achieving automation and intelligence in the task scheduling system.

[0149] If the next batch of processing tasks is parallel to the current scheduling task in business logic, that is, there is no direct parent-child relationship between them, then the next batch of processing tasks can be registered as brother tasks of the current scheduling task, exist in parallel in the scheduling library, and be independently scheduled and executed by the task scheduling system.

[0150] If the next batch of processing tasks is triggered by the current scheduled task, that is, there is a clear parent-child relationship between them, then the next batch of processing tasks can be registered as subtasks of the current scheduled task. The task scheduling system can automatically schedule and execute these tasks in the correct order based on the dependencies between the tasks.

[0151] In this embodiment, the tree-like scheduling task control records of the sub-library are synchronized to the scheduling library. When the synchronization execution result is abnormal, the automatic retry mechanism is started; when the synchronization execution result is normal, the scheduling library task control record is updated in a tree-like management manner. By synchronizing the tree-like scheduling task control records, the automatic retry mechanism, the real-time update of the task control records, and the automatic registration of new tasks, the data consistency, robustness, automation degree and ability to support complex scheduling scenarios of the scheduling task scheduling system are significantly improved.

[0152] The following will be combined Figure 6 The complete process of task scheduling is described in detail. Figure 6 This is a complete flowchart of the task scheduling method for this application.

[0153] like Figure 6 As shown, first, after the batch processing business is initiated, the scheduling task is registered:

[0154] The batch processing service initiator registers the scheduling task through the provided registration scheduling task interface. The registered tasks support tree structure management and support registration of scheduling tasks according to the "parent-child-grandchild" structure.

[0155] The implementation of registered scheduling tasks includes: writing scheduling task control records to the scheduling library; synchronizing scheduling task control to the sub-library, supporting automatic retry for synchronization exceptions, and writing scheduling task control to the sub-library if synchronization is successful; registering the sub-library batch processing program to the message queue or background thread, supporting automatic retry. If it is a batch processing of the same database, there is no need to synchronize scheduling task control to other sub-libraries.

[0156] Furthermore, after the scheduling task is registered, the batch processing business is executed in the sub-database:

[0157] Sub-database batch processing application / / Configure message queue consumers or configure thread pools. When the application instance has available threads, the corresponding sub-database batch processing execution program will be called. If the message packet is lost abnormally, resulting in a long period of non-execution of the batch processing, the corresponding scheduling task control record of the sub-database will remain in the pending execution state. If the configured timeout non-execution alarm duration is exceeded, the scheduling task abnormality monitoring will push an alarm; if the configured long-term non-execution automatic retry duration is exceeded, the scheduling task abnormal retry will re-register the sub-database batch processing program to the message queue or background thread.

[0158] Furthermore, the scheduling task result management is performed based on the execution results of the sub-database batch processing business:

[0159] The batch processing of the sub-database is completed, including successful and failed processing, and the control record results of the current scheduling task G1 are updated. It supports the management of execution results of scheduling tasks in a "parent-child-grandchild" tree structure. After updating the control record results of the current scheduling task, it will look up for the task S1 of the previous layer. If it exists, the number of successful subtasks / failed subtasks on task S1 will be updated. Similarly, after all the subtasks of task S1 are executed, it will also look up for the task F1 of the previous layer of task S1. If it exists, the number of successful subtasks / failed subtasks on task F1 will be updated.

[0160] After the batch processing of the dispatching tasks of the sub-library is completed, since the dispatching tasks of the sub-library are synchronized from the dispatching library, the sub-library needs to be synchronized back to the dispatching library. Automatic retry is supported for synchronization exceptions. After updating the corresponding dispatching library task control record results, the tree-like dispatching task management execution results are also supported.

[0161] Finally, the next batch of processing operations is triggered.

[0162] The task execution results of all sub-databases are synchronized back to the scheduling library, and the scheduling library updates the execution results of the corresponding sub-databases. If the registered batch scheduling task needs to trigger the next batch processing, the scheduling library will register the scheduling task for the next batch processing after updating the execution results of all sub-databases; if the registered batch scheduling task does not need to trigger the next batch processing, the batch processing will be completed after the scheduling library updates the execution results of all sub-databases.

[0163] In addition, the task scheduling method of this application also provides a scheduling task execution result query mechanism, a scheduling task exception maintenance mechanism, a scheduling task exception monitoring mechanism, and a scheduling task exception retry mechanism. Through the above mechanisms acting together on the task scheduling system, the transparency, stability, reliability and automation of the system are improved. It helps to optimize task execution efficiency and resource utilization, and can also respond quickly and reduce losses in abnormal situations, providing a strong guarantee for business continuity and efficient operation.

[0164] To facilitate the management of the next batch of processing scheduling tasks, the next batch of processing tasks can be registered as the brother tasks of the same set of numbers of the current scheduling tasks, or as the subtasks of the current scheduling tasks. The next batch of processing scheduling tasks also supports tree management, distributing the next layer of scheduling tasks; of course, if the logic of the next batch is relatively simple, the next layer of scheduling tasks can also be not distributed.

[0165] This application also provides a task scheduling device, please refer to Figure 7 , the task scheduling device comprises:

[0166] The scheduling task registration module 10 is used to obtain the batch task initiation instruction and register the scheduling task in a tree structure management manner according to the batch task initiation instruction;

[0167] The sub-database batch processing execution module 20 is used to register the scheduling task into the message queue for sub-database batch processing execution and obtain the sub-database batch processing execution result;

[0168] A control record updating module 30, used to update the tree-like scheduling task control record according to the sub-library batch processing execution result;

[0169] The next batch of processing scheduling tasks triggering module 40 is used to synchronize the tree-like scheduling task control records to the scheduling library to trigger the next batch of processing scheduling tasks.

[0170] The task scheduling device provided by the present application adopts the task scheduling method in the above embodiment to solve the technical problem of task scheduling. Compared with the prior art, the beneficial effects of the task scheduling device provided by the present application are the same as the beneficial effects of the task scheduling method provided by the above embodiment, and other technical features in the task scheduling device are the same as the features disclosed in the above embodiment method, which will not be repeated here.

[0171] The present application provides a task scheduling device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the task scheduling method in the above-mentioned embodiment 1.

[0172] Reference below Figure 8, which shows a schematic diagram of the structure of a task scheduling device suitable for implementing the embodiment of the present application. The task scheduling device in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 8 The task scheduling device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0173] like Figure 8 As shown, the task scheduling device may include a processing device 1001 (e.g., a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for the operation of the task scheduling device are also stored. The processing device 1001, ROM1002, and RAM1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the task scheduling device to communicate with other devices wirelessly or by wire to exchange data. Although the task scheduling device with various systems is shown in the figure, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems can be implemented or have alternatively.

[0174] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0175] The task scheduling device provided by the present application adopts the task scheduling method in the above embodiment to solve the technical problem of task scheduling. Compared with the prior art, the beneficial effects of the task scheduling device provided by the present application are the same as the beneficial effects of the task scheduling method provided by the above embodiment, and the other technical features in the task scheduling device are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0176] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0177] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0178] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, computer programs) stored thereon, and the computer-readable program instructions are used to execute the task scheduling method in the above-mentioned embodiment.

[0179] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination of the above.

[0180] The computer-readable storage medium may be included in the task scheduling device; or may exist independently without being assembled into the task scheduling device.

[0181] The above-mentioned computer-readable storage medium carries one or more programs. When the above-mentioned one or more programs are executed by the task scheduling device, the task scheduling device: obtains a batch task initiation instruction and registers a scheduling task according to the batch task initiation instruction; registers the scheduling task to a message queue or a background thread for sub-library batch execution to obtain a sub-library batch execution result; updates a tree-like scheduling task control record according to the sub-library batch execution result; and synchronizes the tree-like scheduling task control record to the scheduling library to trigger the next batch scheduling task.

[0182] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0183] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0184] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0185] The readable storage medium provided by the present application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned task scheduling method, and can solve the technical problems of task scheduling. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided by the present application are the same as the beneficial effects of the task scheduling method provided by the above-mentioned embodiment, and will not be repeated here.

[0186] The present application also provides a computer program product, including a computer program, which implements the steps of the task scheduling method as described above when executed by a processor.

[0187] The computer program product provided by the present application can solve the technical problem of task scheduling. Compared with the prior art, the beneficial effects of the computer program product provided by the present application are the same as the beneficial effects of the task scheduling method provided by the above embodiment, which will not be repeated here.

[0188] The above descriptions are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect applications in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A task scheduling method, characterized in that: The method comprises: Obtaining a batch task initiation instruction and registering a scheduling task in a tree structure management manner according to the batch task initiation instruction; Register the scheduling task to the message queue or background thread for batch processing and execution, and obtain the batch processing execution result; Update the tree-like scheduling task control record according to the sub-database batch processing execution result; Synchronize the tree-like scheduling task control record to the scheduling library to trigger the next batch of processing scheduling tasks.

2. The method according to claim 1, characterized in that The step of obtaining a batch task initiation instruction and registering a scheduling task in a tree structure management manner according to the batch task initiation instruction comprises: Get the batch task initiation instruction; Calling a registration scheduling task interface according to the batch task initiation instruction, and generating a tree-like scheduling task control record in a tree-like structure management manner through the registration scheduling task interface; Writing the tree-like scheduling task control record into the scheduling database and determining whether cross-database processing is required; If cross-database processing is required, the tree-like scheduling task control record is synchronized to the sub-database for scheduling task registration; If cross-database processing is not required, directly register the scheduling task.

3. The method according to claim 1, characterized in that The step of registering the scheduling task to a message queue or a background thread for database batch processing execution and obtaining the database batch processing execution result comprises: Registering the scheduling task to a message queue or a background thread to configure a message queue consumer or a thread pool; The corresponding sub-library batch processing execution program is called through the message queue consumer or the thread pool to execute the sub-library batch processing and obtain the sub-library batch processing execution result.

4. The method according to claim 3, characterized in that After the step of calling the corresponding sub-library batch processing execution program through the message queue consumer or the thread pool to execute sub-library batch processing and obtain the sub-library batch processing execution result, the following step is also included: When the execution time of the sub-library batch processing exceeds the execution alarm duration, the tree-like scheduling task control record corresponding to the scheduling task is kept in a pending execution state, and a timeout non-execution alarm and an automatic retry mechanism are triggered.

5. The method according to claim 1, characterized in that The step of updating the tree-like scheduling task control record according to the sub-library batch processing execution result comprises: Update the current scheduling task control record result according to the sub-library batch processing execution result and determine whether there is a first upper-level scheduling task; When there is a first upper-layer scheduling task, the number of successful subtasks and the number of failed subtasks of the first upper-layer scheduling task are updated upward based on the control record result of the current scheduling task to obtain the control record result of the first upper-layer scheduling task; Determine whether there is a second upper-layer scheduling task according to the control record result of the first upper-layer scheduling task; When there is a second upper-layer scheduling task, the number of successful subtasks and the number of failed subtasks of the second upper-layer scheduling task are updated upward based on the control record result of the first upper-layer scheduling task to obtain the control record result of the second upper-layer scheduling task; The tree-shaped scheduling task control record is updated according to the current scheduling task control record result, the first upper-layer scheduling task control record result, and the second upper-layer scheduling task control record result.

6. The method according to claim 1, characterized in that The step of synchronizing the tree-like scheduling task control record to the scheduling library to trigger the next batch of processing scheduling tasks includes: Synchronize the tree-like scheduling task control record to the scheduling library to update the scheduling library task control record in a tree-like management manner; The next batch of processing tasks is automatically registered as a sibling task or a subtask of the scheduling task.

7. The method according to claim 6, characterized in that The step of synchronizing the tree-like scheduling task control record to the scheduling library to update the scheduling library task control record in a tree-like management manner includes: When the synchronization execution result of the tree-like scheduling task control record synchronized to the scheduling library is abnormal, an automatic retry mechanism is started, and the synchronization execution result includes execution time and synchronization result; When the synchronization execution result of the tree-like scheduling task control record to the scheduling library is normal, the scheduling library task control record is updated in a tree-like management manner.

8. A task scheduling device, characterized in that: The device comprises: A scheduling task registration module is used to obtain a batch task initiation instruction and register a scheduling task according to the batch task initiation instruction; The sub-database batch processing execution module is used to register the scheduling task into the message queue for sub-database batch processing execution and obtain the sub-database batch processing execution result; A control record updating module, used for updating the tree-like scheduling task control record according to the sub-library batch processing execution result; The next batch of processing scheduling tasks triggering module is used to synchronize the tree-like scheduling task control records to the scheduling library to trigger the next batch of processing scheduling tasks.

9. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the task scheduling method according to any one of claims 1 to 7 are implemented.

10. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the steps of the task scheduling method according to any one of claims 1 to 7 are implemented.

Images