Task scheduling method and system, electronic device and computer readable storage medium

Abstract

The application discloses a task scheduling method and system, an electronic device and a computer readable storage medium, relates to the field of task scheduling, and is applied to a main service node in a task scheduling platform. The task scheduling platform further comprises a database for storing task instructions and at least one queue for issuing the task instructions. The task scheduling method comprises the following steps: in a current query period, querying the number of tasks corresponding to the task instructions in each queue; determining whether there is a target queue with a task number less than a preset number; if yes, reading the task instructions from the database and writing the task instructions into the target queue, so that the target queue issues the task instructions to corresponding working service nodes in the task scheduling platform. The application can avoid network communication failures and data loss caused by frequent reading of the database, and ensure the stability of high-frequency reading and writing when a large number of tasks exist.

Description

Technical Field

[0001] This application relates to the field of task scheduling, and in particular to task scheduling methods, systems, electronic devices, and computer-readable storage media. Background Technology

[0002] Dolphinscheduler is a distributed, decentralized, and easily scalable visual DAG (Directed Acyclic Graph) workflow task scheduling platform. The existing overall architecture diagram of Dolphinscheduler is shown below. Figure 1 As shown, the task distribution process is as follows: the UI layer sends task instructions to the API layer, the API layer persists the task instructions in a traditional relational database (DB), and then the master server scans the task instructions in the relational database and sends the scanned task instructions to the worker servers. Under the existing persistent storage structure of dolphinscheduler, stability requirements can be met when executing a small number of tasks. However, when a large number of tasks are executed concurrently, the master server's monitoring thread needs to frequently read and write to the relational database to ensure the normal distribution of task instructions. Frequent database reads and writes can cause network communication failures and data loss.

[0003] Therefore, how to provide a solution to the above-mentioned technical problems is a problem that needs to be solved by those skilled in the art. Summary of the Invention

[0004] The purpose of this application is to provide a task scheduling method, system, electronic device, and computer-readable storage medium that can avoid network communication failures and data loss caused by frequent database reads, and ensure the stability of high-frequency read and write operations when a large number of tasks exist.

[0005] To address the aforementioned technical problems, this application provides a task scheduling method applied to the main service node of a task scheduling platform. The task scheduling platform further includes a database for storing task instructions and at least one queue for issuing the task instructions. The task scheduling method includes:

[0006] In the current query period, query the number of tasks corresponding to the task instructions in each of the queues;

[0007] Determine whether there exists a target queue with a task count less than a preset number;

[0008] If so, the task instruction is read from the database and written into the target queue, so that the target queue can send the task instruction to the corresponding work service node in the task scheduling platform.

[0009] Optionally, after determining whether there exists a target queue with a task quantity less than a preset quantity, the task scheduling method further includes:

[0010] If the target queue is determined to exist, the current instruction write count corresponding to the target queue is determined;

[0011] The process of reading the task instructions from the database includes:

[0012] Read the task instruction of the current instruction write quantity from the database.

[0013] Optionally, after determining whether there exists a target queue with a task quantity less than a preset quantity, the task scheduling method further includes:

[0014] If there are multiple target queues, determine the reading priority of each target queue;

[0015] The process of reading the task instructions from the database includes:

[0016] The task instructions corresponding to each target queue are read from the database in descending order of read priority.

[0017] Optionally, the process of reading the task instructions from the database includes:

[0018] Read the task instructions marked with pending execution from the database.

[0019] Optionally, the task scheduling method further includes:

[0020] Obtain the response information of the task instructions fed back by each of the work service nodes, and store the response information in a preset temporary storage space;

[0021] When the storage conditions are met, each of the response information stored in the preset temporary storage space is stored in the database.

[0022] Optionally, before storing each of the response information stored in the preset temporary storage space into the database when the storage conditions are met, the task scheduling method further includes:

[0023] Determine whether the number of tasks in each queue is greater than a preset number;

[0024] If so, determine that the storage conditions are met;

[0025] If not, the storage conditions are not met.

[0026] Optionally, after obtaining the response information of the task instructions fed back by each of the work service nodes, the task scheduling method further includes:

[0027] Determine whether any of the aforementioned response messages contains a target response message that meets the alarm conditions;

[0028] If so, generate an alarm message based on the target response information and issue a notification.

[0029] To address the aforementioned technical problems, this application also provides a task scheduling system applied to the main service node of a task scheduling platform. The task scheduling platform further includes a database for storing task instructions and at least one queue for issuing the task instructions. The task scheduling system includes:

[0030] The query module is used to query the number of tasks corresponding to the task instructions in each queue during the current query period.

[0031] The first judgment module is used to determine whether there is a target queue with a number of tasks less than a preset number. If so, the scheduling module is triggered.

[0032] The scheduling module is used to read the task instruction from the database and write the task instruction into the target queue, so that the target queue can send the task instruction to the corresponding work service node in the task scheduling platform.

[0033] To address the aforementioned technical problems, this application also provides an electronic device, comprising:

[0034] Memory, used to store computer programs;

[0035] A processor, used to implement the steps of the task scheduling method as described in any of the preceding descriptions when executing the computer program.

[0036] To address the aforementioned technical problems, this application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the task scheduling method described in any of the preceding claims.

[0037] This application provides a task scheduling method. In a task scheduling platform, at least one queue for issuing task instructions is set up. Task instructions read from a database are first written to each queue in batches, and then the queues issue the task instructions. The database is only read when the number of tasks corresponding to a queue is less than a certain value. This avoids network communication failures and data loss caused by frequent database reads, ensuring stability during high-frequency read / write operations with a large number of tasks. This application also provides a task scheduling system, electronic device, and computer-readable storage medium, which have the same beneficial effects as the above-described task scheduling method. Attached Figure Description

[0038] To more clearly illustrate the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0039] Figure 1 This is a schematic diagram of the structure of a task scheduling platform in the prior art;

[0040] Figure 2 A schematic diagram of the structure of a task scheduling platform provided in this application;

[0041] Figure 3 A flowchart of the steps of a task scheduling method provided in this application;

[0042] Figure 4 This is a schematic diagram of the structure of a task scheduling system provided in this application. Detailed Implementation

[0043] The core of this application is to provide a task scheduling method, system, electronic device, and computer-readable storage medium that can avoid network communication failures and data loss caused by frequent database reads, and ensure the stability of high-frequency read and write operations when a large number of tasks exist.

[0044] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0045] Firstly, to facilitate understanding of the task scheduling method provided in this application, the task scheduling platform to which this method is applicable will be described below. Please refer to [link / reference needed]. Figure 2 As shown, Figure 2 This is a schematic diagram of the structure of a task scheduling platform provided in this application. The task scheduling platform includes a UI layer, an API layer, multiple master server nodes, multiple worker server nodes, a Zookeeper cluster, a database, an alarm module, and at least one queue. Figure 2 The example shows three queues, three master servers, and three worker servers. It can be understood that the number of queues, master servers, and worker servers can be selected according to the actual project needs, and this embodiment does not impose specific limitations.

[0046] The Master server adopts a distributed, decentralized design, primarily responsible for DAG task splitting, task submission monitoring, and simultaneously monitoring the health status of other Master and Worker servers. The Worker servers also employ a distributed, decentralized design, primarily responsible for task execution and providing log services. Upon startup, each Worker server registers an ephemeral node with the ZooKeeper cluster and maintains a heartbeat. ZooKeeper is an open-source distributed coordination system. The registration of Master servers and / or Worker servers with ZooKeeper ensures a decentralized Master and Worker cluster, and uses ZooKeeper's distributed locks to elect a Master server or Worker server as the manager to execute the corresponding tasks.

[0047] For a detailed description of the task scheduling method provided in this application, please refer to [link / reference]. Figure 3 , Figure 3 This application provides a flowchart of the steps of a task scheduling method, which can be implemented through any one of the master service nodes in the task scheduling platform described above. The task scheduling method includes:

[0048] S101: In the current query period, query the number of tasks corresponding to the task instructions in each queue;

[0049] Understandably, the UI layer initiates task commands to the API layer. The API layer parses the task commands from the UI layer and persistently stores them in the database. A monitoring thread is created in the main service node. This thread reads task commands in batches from the database, writes them to the corresponding queues, and monitors the distribution of task commands in the queues. The main service node can have one monitoring thread monitoring each queue, or multiple monitoring threads can be created, each monitoring a specific queue. In one optional implementation, each monitoring thread has the same query cycle; in another optional implementation, each monitoring thread has a different query cycle. After task instructions are written to the queue, they are sequentially sent to the corresponding worker service nodes in a first-in, first-out (FIFO) order. Therefore, the number of task instructions in the queue varies in each query cycle. However, considering that some tasks include many task instructions, the number of tasks in the queue may remain unchanged in adjacent query cycles. To avoid frequent database reads when there are too many batch-executed tasks, the monitoring thread first queries the number of tasks corresponding to the task instructions in each queue in each query cycle. This allows the task instructions to be read from the database only when the number of tasks in the queue is small. The query cycle can be determined based on the actual number of tasks.

[0050] A task includes at least one task instruction, and each task instruction carries task identification information corresponding to its task. It can be understood that all task instructions included in the same task carry the same task identification information. Considering the case that there are multiple tasks corresponding to task instructions in a certain queue, the number of tasks corresponding to the queue can be determined according to the number of different task identification information in the queue.

[0051] For example, suppose there are m task instructions in a queue, where n task instructions have task identifier information 'a' and mn task instructions have task identifier information 'b', then the number of tasks corresponding to this queue is 2.

[0052] S102: Determine if there is a target queue with a number of tasks less than the preset number. If so, execute S103.

[0053] S103: Read the task instructions from the database and write them into the target queue so that the target queue can send the task instructions to the corresponding work service nodes in the task scheduling platform.

[0054] Understandably, to avoid problems caused by frequent database reads when there are too many batch tasks, after obtaining the number of tasks corresponding to each queue in the current query cycle, it is determined whether the number of tasks corresponding to each queue is less than a preset number. The queue with a number of tasks less than the preset number is determined as the target queue for the current query cycle. The monitoring thread reads task instructions from the database in batches and writes the batch of task instructions into the target queue. If the number of tasks corresponding to each queue obtained in the current query cycle is greater than or equal to the preset number, the monitoring thread is suspended and does not perform any processing, waiting for the next query cycle.

[0055] The number of task instructions that the monitoring thread reads from the database in batches each time can be set according to the actual project needs, such as based on the hardware processing capabilities of the task scheduling platform or dynamically calculated. This application does not limit this.

[0056] As can be seen, in this embodiment, at least one queue for issuing task instructions is set in the task scheduling platform. The task instructions read from the database are first written into each queue in batches, and the queues issue the task instructions. When the number of tasks corresponding to the queue is less than a certain value, the database is read again. This avoids network communication failures and data loss caused by frequent database reads, and ensures the stability of high-frequency read and write when a large number of tasks exist.

[0057] Based on the above embodiments:

[0058] As an optional embodiment, after determining whether there is a target queue with a number of tasks less than a preset number, the task scheduling method further includes:

[0059] If a target queue is determined to exist, the current number of instructions written to the target queue is determined.

[0060] The process of reading task instructions from the database includes:

[0061] Read the task instruction from the database to record the current number of instructions to be written.

[0062] Understandably, each queue has a preset target number of instructions to write, which is the maximum number of instructions that can be written to that queue. When it is determined that there is a target queue with fewer tasks than the preset number in the current query period, the current number of task instructions in that target queue is determined. The difference between the target number of instructions to write and the current number of task instructions can be used to determine the current number of instructions to write to that target queue. Task instructions of the current number of instructions to write are then read from the database and written to that target queue.

[0063] For example, suppose the number of target instructions written to a target queue is p, and the number of task instructions in the target queue is q in the current query period. Then the number of target instructions written to the target queue is pq, and pq task instructions are read from the database and written to the target queue.

[0064] As an optional embodiment, after determining whether there is a target queue with a number of tasks less than a preset number, the task scheduling method further includes:

[0065] If there are multiple target queues, determine the reading priority of each target queue;

[0066] The process of reading task instructions from the database includes:

[0067] The task instructions corresponding to each target queue are read from the database in descending order of read priority.

[0068] Considering the possibility of multiple target queues within the same query period, to avoid multiple monitoring threads corresponding to multiple queues simultaneously reading task instructions from the database, and to ensure that only one monitoring thread reads data from the database within a single query period, this embodiment first determines the reading priority of each target queue. The reading priority can be pre-set for each queue or determined in real time based on the current number of tasks in each queue. For example, queues with fewer tasks have higher reading priority. Then, task instructions corresponding to each target queue are read from the database in descending order of reading priority. The number of task instructions read by the monitoring thread from the database each time can be determined based on the current instruction write count obtained in the previous embodiment.

[0069] As an optional embodiment, the process of reading task instructions from the database includes:

[0070] Read the task instructions with pending execution flags from the database.

[0071] Specifically, users can output the execution command for a task on the UI layer page. At this time, all task instructions corresponding to the task stored in the database include the pending execution mark. If the monitoring thread detects the existence of a target queue in the current query cycle, it will read the corresponding number of task instructions with pending execution marks from the database according to the number of instructions written in the current target queue, thereby ensuring the normal execution of the task.

[0072] As an optional embodiment, the task scheduling method further includes:

[0073] Obtain the response information of the task instructions fed back by each working service node, and store the response information in the preset temporary storage space;

[0074] When the storage conditions are met, the various response information stored in the preset temporary storage space will be stored in the database.

[0075] Understandably, after the master service node sends a task instruction to the corresponding worker service node, the worker service node will send a response to the master service node. This response includes, but is not limited to, status information such as success or failure, as well as execution logs. The master service node stores the received feedback information in a pre-defined temporary storage space in Redis. When storage conditions are met, it then writes all the feedback information stored in the temporary storage space to the database for persistence, thus achieving read-write separation and ensuring stability during high-frequency read / write operations with a large number of tasks.

[0076] The storage conditions can be reaching a new storage cycle, the amount of stored data exceeding a preset amount of data, or other conditions. This embodiment does not impose specific limitations on these conditions.

[0077] As an optional embodiment, before storing each response information stored in the preset temporary storage space into the database when the storage conditions are met, the task scheduling method further includes:

[0078] Determine if the number of tasks in each queue is greater than the preset number;

[0079] If so, determine that the storage conditions are met;

[0080] If not, the storage conditions are not met.

[0081] Considering that the trigger condition for the monitoring thread to read task instructions from the database in this application is that there is a target queue in each queue with a number of tasks less than a preset number, this embodiment can use the fact that the number of tasks in each queue is greater than the preset number as the trigger condition for storing the response information in the database. Since the number of tasks in each queue is greater than the preset number, the monitoring thread will not read task instructions from the database. At this time, storing each response information stored in the preset temporary storage space into the database can further realize read-write separation and ensure the stability of high-frequency read and write when a large number of tasks exist.

[0082] As an optional embodiment, after obtaining the response information of the task instructions fed back by each working service node, the task scheduling method further includes:

[0083] Determine whether any of the response messages meet the alarm conditions;

[0084] If so, generate an alarm message based on the target response information and display a notification.

[0085] Specifically, after receiving the response information from each working service node, the main service node will determine whether there is target response information that meets the alarm conditions. If so, it will push the alarm information generated based on the target response information to the alarm thread so that the alarm thread can promptly control the page alarm prompt, thereby improving the security and reliability of the task scheduling platform.

[0086] In summary, the solution provided in this application addresses the read / write pressure on the database caused by a large number of tasks requiring frequent database reads and writes in a production environment, leading to data errors, loss, dirty reads, phantom reads, and other anomalies. Based on this solution, the stability of the dolphinscheduler platform in a production environment can be significantly improved. Batch processing can significantly reduce the number of concentrated processing requests, making it suitable for large-scale task read / write operations and monitoring information acquisition. It optimizes the topology architecture of the traditional dolphinscheduler platform in the persistence layer by using batch processing to write tasks in batches. A middle layer built with a queue data structure achieves stable and efficient continuous writing. Parameters such as write batches can be preset or dynamically set according to the specific production environment, thus improving the stability of the production environment. At the same time, compared with native relational databases, distributed storage can guarantee data integrity to a greater extent.

[0087] Secondly, please refer to Figure 4 , Figure 4 This application provides a schematic diagram of the structure of a task scheduling system, applied to the main service node of a task scheduling platform. The task scheduling platform also includes a database for storing task instructions and at least one queue for issuing task instructions. The task scheduling system includes:

[0088] Query module 1 is used to query the number of tasks corresponding to task instructions in each queue during the current query period;

[0089] The first judgment module 2 is used to determine whether there is a target queue with a number of tasks less than a preset number. If so, the scheduling module 3 is triggered.

[0090] The scheduling module 3 is used to read task instructions from the database and write the task instructions into the target queue, so that the target queue can send the task instructions to the corresponding work service nodes in the task scheduling platform.

[0091] Understandably, the UI layer initiates task commands to the API layer. The API layer parses the task commands from the UI layer and persistently stores them in the database. A monitoring thread is created in the main service node. This thread reads task commands in batches from the database, writes them to the corresponding queues, and monitors the distribution of task commands in the queues. The main service node can have one monitoring thread monitoring each queue, or multiple monitoring threads can be created, each monitoring a specific queue. In one optional implementation, each monitoring thread has the same query cycle; in another optional implementation, each monitoring thread has a different query cycle. After task instructions are written to the queue, they are sequentially sent to the corresponding worker service nodes in a first-in, first-out (FIFO) order. Therefore, the number of task instructions in the queue varies in each query cycle. However, considering that some tasks include many task instructions, the number of tasks in the queue may remain unchanged in adjacent query cycles. To avoid frequent database reads when there are too many batch-executed tasks, the monitoring thread first queries the number of tasks corresponding to the task instructions in each queue in each query cycle. This allows the task instructions to be read from the database only when the number of tasks in the queue is small. The query cycle can be determined based on the actual number of tasks.

[0092] A task includes at least one task instruction, and each task instruction carries task identification information corresponding to its task. It can be understood that all task instructions included in the same task carry the same task identification information. Considering the case that there are multiple tasks corresponding to task instructions in a certain queue, the number of tasks corresponding to the queue can be determined according to the number of different task identification information in the queue.

[0093] Understandably, to avoid problems caused by frequent database reads when there are too many tasks to execute in batches, after obtaining the number of tasks corresponding to each queue in the current query cycle, it is determined whether the number of tasks corresponding to each queue is less than a preset number. The queue with a number of tasks less than the preset number is determined as the target queue for the current query cycle. The monitoring thread reads task instructions from the database in batches and writes the task instructions into the target queue. If the number of tasks corresponding to each queue obtained in the current query cycle is greater than or equal to the preset number, the monitoring thread is suspended and does not perform any processing, waiting for the next query cycle. Therefore, in this embodiment, at least one queue for issuing task instructions is set in the task scheduling platform. The task instructions read from the database are first written to each queue in batches, and the queues issue the task instructions. When the number of tasks corresponding to a queue is less than a certain value, the database is read again. This avoids network communication failures and data loss caused by frequent database reads, ensuring the stability of high-frequency read and write operations when there are a large number of tasks.

[0094] As an optional embodiment, the task scheduling system further includes:

[0095] The first determining module is used to determine the current instruction write count corresponding to the target queue if it is determined that a target queue exists.

[0096] The process of reading task instructions from the database includes:

[0097] Read the task instruction from the database to record the current number of instructions to be written.

[0098] As an optional embodiment, the task scheduling system further includes:

[0099] The second determining module is used to determine the reading priority of each target queue if there are multiple target queues.

[0100] The process of reading task instructions from the database includes:

[0101] The task instructions corresponding to each target queue are read from the database in descending order of read priority.

[0102] As an optional embodiment, the process of reading task instructions from the database includes:

[0103] Read the task instructions with pending execution flags from the database.

[0104] As an optional embodiment, the task scheduling system further includes:

[0105] The storage processing module is used to obtain the response information of the task instructions fed back by each working service node, store the response information in a preset temporary storage space, and when the storage conditions are met, store each response information stored in the preset temporary storage space into the database.

[0106] As an optional embodiment, the task scheduling system further includes:

[0107] The second judgment module is used to determine whether the number of tasks in each queue is greater than the preset number. If so, it is determined that the storage condition is met; otherwise, it is determined that the storage condition is not met.

[0108] As an optional embodiment, the task scheduling system further includes:

[0109] The third judgment module is used to determine whether there is a target response information that meets the alarm conditions for each response information. If so, an alarm information is generated and displayed based on the target response information.

[0110] Thirdly, this application also provides an electronic device, including:

[0111] Memory, used to store computer programs;

[0112] A processor is used to implement the steps of the task scheduling method as described in any of the embodiments above when executing a computer program.

[0113] Specifically, the memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer-readable instructions, and the internal memory provides an environment for the operation of the operating system and computer-readable instructions stored in the non-volatile storage media. When the processor executes the computer program stored in the memory, it can perform the following steps: In the current query cycle, query the number of tasks corresponding to the task instructions in each queue; determine whether there is a target queue with a number of tasks less than a preset number; if so, read the task instructions from the database and write the task instructions into the target queue, so that the target queue can send the task instructions to the corresponding worker service node in the task scheduling platform.

[0114] As can be seen, in this embodiment, at least one queue for issuing task instructions is set in the task scheduling platform. The task instructions read from the database are first written into each queue in batches, and the queues issue the task instructions. When the number of tasks corresponding to the queue is less than a certain value, the database is read again. This avoids network communication failures and data loss caused by frequent database reads, and ensures the stability of high-frequency read and write when a large number of tasks exist.

[0115] As an optional embodiment, when the processor executes a computer subroutine stored in memory, it can perform the following steps: if a target queue is determined to exist, determine the current instruction write count corresponding to the target queue; and read the task instruction with the current instruction write count from the database.

[0116] As an optional embodiment, when the processor executes a computer subroutine stored in memory, it can perform the following steps: if there are multiple target queues, determine the reading priority of each target queue; and read the task instructions corresponding to each target queue from the database in descending order of reading priority.

[0117] As an optional embodiment, when the processor executes a computer subroutine stored in memory, it may perform the following steps: read a task instruction marked with a pending execution identifier from a database.

[0118] As an optional embodiment, when the processor executes the computer subroutine stored in the memory, it can perform the following steps: obtain the response information of the task instructions fed back by each working service node, and store the response information in a preset temporary storage space; when the storage conditions are met, store each response information stored in the preset temporary storage space into the database.

[0119] As an optional embodiment, when the processor executes the computer subroutine stored in the memory, it can perform the following steps: determine whether the number of tasks in each queue is greater than a preset number; if yes, determine that the storage condition is met; if no, determine that the storage condition is not met.

[0120] As an optional embodiment, when the processor executes the computer subroutine stored in the memory, it can perform the following steps: determine whether there is a target response information that meets the alarm conditions for each response information; if so, generate alarm information based on the target response information and display a notification.

[0121] Based on the above embodiments, as a preferred embodiment, the electronic device further includes:

[0122] An input interface, connected to the processor, is used to acquire externally imported computer programs, parameters, and instructions, which are then stored in memory under the processor's control. This input interface can be connected to an input device to receive parameters or instructions manually entered by the user. This input device can be a touch layer covering the display screen, or buttons, a trackball, or a touchpad located on the terminal casing.

[0123] The display unit, connected to the processor, is used to display the data sent by the processor. This display unit can be an LCD screen or an e-ink screen, etc.

[0124] The network port, connected to the processor, is used for communication with external terminal devices. The communication technology used for this connection can be wired or wireless, such as Mobile High Definition Link (MHL), Universal Serial Bus (USB), High Definition Multimedia Interface (HDMI), Wireless Fidelity (WiFi), Bluetooth, Bluetooth Low Energy, or IEEE 802.11s-based communication technologies.

[0125] Fourthly, this application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the task scheduling method described in any of the above embodiments.

[0126] The computer-readable storage medium may include various media capable of storing program code, such as USB flash drives, external hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks. The storage medium stores a computer program, which, when executed by a processor, performs the following steps: During the current query cycle, query the number of tasks corresponding to task instructions in each queue; determine if there is a target queue with a task count less than a preset number; if so, read the task instructions from the database and write them to the target queue, so that the target queue can distribute the task instructions to the corresponding worker service node in the task scheduling platform.

[0127] As can be seen, in this embodiment, at least one queue for issuing task instructions is set in the task scheduling platform. The task instructions read from the database are first written into each queue in batches, and the queues issue the task instructions. When the number of tasks corresponding to the queue is less than a certain value, the database is read again. This avoids network communication failures and data loss caused by frequent database reads, and ensures the stability of high-frequency read and write when a large number of tasks exist.

[0128] As an optional embodiment, when a computer subroutine stored in a computer-readable storage medium is executed by a processor, it can specifically implement the following steps: if it is determined that a target queue exists, determine the current instruction write count corresponding to the target queue; and read the task instruction with the current instruction write count from the database.

[0129] As an optional embodiment, when a computer subroutine stored in a computer-readable storage medium is executed by a processor, the following steps can be implemented: if there are multiple target queues, determine the reading priority of each target queue; read the task instructions corresponding to each target queue from the database in descending order of reading priority.

[0130] As an optional embodiment, when a computer subroutine stored in a computer-readable storage medium is executed by a processor, it may specifically perform the following steps: reading a task instruction marked with a pending execution identifier from a database.

[0131] As an optional embodiment, when a computer subroutine stored in a computer-readable storage medium is executed by a processor, it can specifically implement the following steps: obtaining response information of task instructions fed back by each working service node, storing the response information in a preset temporary storage space; and storing each response information stored in the preset temporary storage space into a database when the storage conditions are met.

[0132] As an optional embodiment, when a computer subroutine stored in a computer-readable storage medium is executed by a processor, it can specifically implement the following steps: determining whether the number of tasks in each queue is greater than a preset number; if yes, determining that the storage condition is met; if no, determining that the storage condition is not met.

[0133] It should also be noted that, in this specification, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0134] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A task scheduling method, characterized in that, The task scheduling method is applied to the main service node in a task scheduling platform, which further includes a database for storing task instructions and at least one queue for issuing the task instructions. In the current query period, query the number of tasks corresponding to the task instructions in each of the queues; Determine whether there exists a target queue with a task count less than a preset number; If so, the task instruction is read from the database and written into the target queue, so that the target queue can send the task instruction to the corresponding work service node in the task scheduling platform; The process of reading the task instructions from the database includes: Read task instructions with pending execution flags from the database; The task scheduling method further includes: Obtain the response information of the task instructions fed back by each of the work service nodes, and store the response information in a preset temporary storage space; When the storage conditions are met, each of the response information stored in the preset temporary storage space is stored in the database; Before storing each response information stored in the preset temporary storage space into the database when the storage conditions are met, the task scheduling method further includes: Determine whether the number of tasks in each queue is greater than a preset number; If so, determine that the storage conditions are met; If not, the storage conditions are not met.

2. The task scheduling method according to claim 1, characterized in that, After determining whether there exists a target queue with a task quantity less than a preset quantity, the task scheduling method further includes: If the target queue is determined to exist, the current instruction write count corresponding to the target queue is determined; The process of reading the task instructions from the database includes: Read the task instruction of the current instruction write quantity from the database.

3. The task scheduling method according to claim 1, characterized in that, After determining whether there exists a target queue with a task quantity less than a preset quantity, the task scheduling method further includes: If there are multiple target queues, determine the reading priority of each target queue; The process of reading the task instructions from the database includes: The task instructions corresponding to each target queue are read from the database in descending order of read priority.

4. The task scheduling method according to claim 1, characterized in that, After obtaining the response information of the task instructions fed back by each of the work service nodes, the task scheduling method further includes: Determine whether any of the aforementioned response messages contains a target response message that meets the alarm conditions; If so, generate an alarm message based on the target response information and issue a notification.

5. A task scheduling system, characterized in that, The task scheduling system includes a main service node applied in a task scheduling platform, the task scheduling platform further including a database for storing task instructions and at least one queue for issuing the task instructions, and the task scheduling system includes: The query module is used to query the number of tasks corresponding to the task instructions in each queue during the current query period. The first judgment module is used to determine whether there is a target queue with a number of tasks less than a preset number. If so, the scheduling module is triggered. The scheduling module is used to read the task instruction from the database and write the task instruction into the target queue, so that the target queue can send the task instruction to the corresponding work service node in the task scheduling platform. The process of reading the task instructions from the database includes: Read task instructions with pending execution flags from the database; The task scheduling system also includes: The storage processing module is used to obtain the response information of the task instructions fed back by each of the work service nodes, store the response information in a preset temporary storage space, and when the storage conditions are met, store each of the response information stored in the preset temporary storage space into the database. The second judgment module is used to determine whether the number of tasks in each queue is greater than a preset number. If so, it is determined that the storage condition is met; otherwise, it is determined that the storage condition is not met.

6. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor, configured to implement the task scheduling method as described in any one of claims 1-4 when executing the computer program.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the task scheduling method as described in any one of claims 1-4.

Images