A task scheduling method and device

Abstract

The present invention discloses a task scheduling method and device. The method is applied to a task scheduling system and includes: triggering each task in the target scheduling resources corresponding to each project at regular intervals, generating an execution record for the execution information corresponding to each task after the regular triggering, and recording the execution record in the local message list in the database; querying the execution status information corresponding to each task in the local message list to obtain a query result; when it is determined based on the query result that there are tasks to be dispatched in the local message list, publishing the tasks to be dispatched to the homonymous message queues corresponding to each project; when it is determined that the tasks to be dispatched have been published to the homonymous message queues, updating the execution status of the tasks to be dispatched in the execution record to dispatched, and when it is determined that the execution status of all tasks in the local message list has been updated to dispatched, determining that the task scheduling is completed.

Description

Technical Field

[0001] The present invention relates to the technical field of computer applications, and particularly to a task scheduling method and device. Background Art

[0002] In a big data platform, some services are often run regularly. For example, new data is regularly imported from a database into the big data platform, and the data processed by the big data platform is exported to a database or a file system. Specifically, task scheduling refers to automatically executing tasks based on a given time point, a given time interval, or a given number of executions. Therefore, task scheduling plays a crucial role in a big data platform.

[0003] Currently, when performing task scheduling for a project, since there is an upper limit to the scheduling resources for running tasks, that is, when the number of running tasks reaches a certain number, queuing needs to be performed through a resource queue. Before a task is sent to a message queue, once the task scheduling center crashes and restarts or the server network times out, the tasks that have not been sent may be lost. Summary of the Invention

[0004] Embodiments of the present invention provide a task scheduling method and device, which are used to solve the technical problem of easy loss of tasks in the prior art.

[0005] In a first aspect, a task scheduling method is provided, which is applied to a task scheduling system. The method at least includes:

[0006] Timed triggering is performed on each task in the target scheduling resources corresponding to each project, and execution information corresponding to each task after timed triggering is generated into an execution record, and the execution record is recorded in a local message list in a database; wherein, the execution record at least includes code backup information and task execution status information;

[0007] The execution status information corresponding to each task in the local message list is queried to obtain a query result; wherein, the execution status information is used to indicate whether any of the tasks has been sent;

[0008] When it is determined based on the query result that there are tasks to be sent in the local message list, the tasks to be sent are published to the same-name message queues corresponding to each project;

[0009] When it is determined that the tasks to be sent have been published to the same-name message queues, the execution status of the tasks to be sent in the execution record is updated to sent, and when it is determined that the execution status of all tasks in the local message list has been updated to sent, it is determined that task scheduling is completed.

[0010] In a possible implementation manner, before timing-triggering each task in the target scheduling resources corresponding to each project, the method further includes:

[0011] Receiving a first request for scheduling resources corresponding to a first project, where the first request carries the project type of the first project;

[0012] Determining the target scheduling resources corresponding to the first project based on the project type and a preset allocation rule.

[0013] In a possible implementation manner, querying the execution status information corresponding to each task in the local message list in the database includes:

[0014] Determining a preset query period;

[0015] When it is determined that the current moment is within the preset query period, querying the execution status information corresponding to each task in the local message list in the database;

[0016] Obtaining a query result based on the execution status information corresponding to each task; where the query result includes the number of tasks to be dispatched and the number of tasks that have been dispatched but not completed.

[0017] In a possible implementation manner, the method further includes:

[0018] When the execution status of any task in the execution record is updated to dispatched, deleting the record corresponding to the any task in the local message list.

[0019] In a possible implementation manner, after querying the number of tasks to be dispatched in the local message list in the database, the method further includes:

[0020] When it is determined that the query result has been empty for a preset duration, or the number of tasks to be dispatched in the query result is less than the number of tasks corresponding to the target scheduling resources within the preset duration, sending a first prompt message to the task scheduling system; the first prompt message is used to prompt the task scheduling system to perform scheduling processing on idle resources.

[0021] In a possible implementation manner, determining the target scheduling resources corresponding to the first project based on the project type and a preset allocation rule includes:

[0022] Determining whether the utilization rate of the scheduling resource pool of the task scheduling system reaches a preset threshold;

[0023] When it is determined that the utilization rate does not reach the preset threshold, determining the scheduling resource ratio corresponding to the project type based on the preset allocation rule;

[0024] Determine the target scheduling resources allocated to the first project based on the scheduling resource ratio and the scheduling resource pool.

[0025] In a possible implementation, before timing-triggering each task in the target scheduling resources corresponding to each project, the method further includes:

[0026] Receive a second request for scheduling resources for a newly applied project, where the number of tasks of the newly added project is carried in the second request;

[0027] Based on the request and the scheduling resource pool of the task scheduling system, allocate corresponding new scheduling resources for the newly added project.

[0028] In a second aspect, a task scheduling device is provided, which is applied to a task scheduling system. The device at least includes:

[0029] A processing unit, configured to perform timing triggering on each task in the target scheduling resources corresponding to each project, generate an execution record for the execution information corresponding to each task after the timing triggering, and record the execution record in the local message list of the database; wherein, the execution record at least includes code backup information and the execution status information of the task;

[0030] A query unit, configured to query the execution status information corresponding to each task in the local message list to obtain a query result; wherein, the execution status information is used to indicate whether any of the tasks has been dispatched;

[0031] A dispatching unit, configured to, when it is determined based on the query result that there are tasks to be dispatched in the local message list, publish the tasks to be dispatched to the same-name message queues corresponding to each project;

[0032] An update unit, configured to, when it is determined that the task to be dispatched has been published to the same-name message queue, update the execution status of the task to be dispatched in the execution record to dispatched, and, when it is determined that the execution status of all tasks in the local message list has been updated to dispatched, determine that the task scheduling is completed.

[0033] In a possible implementation, before timing-triggering each task in the target scheduling resources corresponding to each project, the processing unit is further configured to:

[0034] Receive a first request for scheduling resources corresponding to the first project, where the project type of the first project is carried in the first request;

[0035] Based on the project type and a preset allocation rule, determine the target scheduling resources corresponding to the first project.

[0036] In a possible implementation, the query unit is specifically configured to:

[0037] Determine a preset query period;

[0038] When it is determined that the current time is within the preset query period, query the execution status information corresponding to each task in the local message list in the database;

[0039] Obtain a query result based on the execution status information corresponding to each task; wherein, the query result includes the number of tasks to be dispatched and the number of tasks that have been dispatched but not completed.

[0040] In a possible implementation, the device further includes a deletion unit, configured to:

[0041] When the execution status of any task in the execution record is updated to dispatched, delete the record corresponding to the any task in the local message list.

[0042] In a possible implementation, after querying the number of tasks to be dispatched in the local message list in the database, the device further includes a prompting unit, configured to:

[0043] When it is determined that the query result has been empty within a preset duration, or the number of tasks to be dispatched in the query result is less than the number of tasks corresponding to the target scheduling resources within the preset duration, send a first prompt message to the task scheduling system; the first prompt message is used to prompt the task scheduling system to perform scheduling processing on idle resources.

[0044] In a possible implementation, the processing unit is specifically configured to:

[0045] Determine whether the utilization rate of the scheduling resource pool of the task scheduling system reaches a preset threshold;

[0046] When it is determined that the utilization rate does not reach the preset threshold, determine the scheduling resource ratio corresponding to the project type based on the preset allocation rule;

[0047] Determine the target scheduling resources allocated to the first project based on the scheduling resource ratio and the scheduling resource pool.

[0048] In a possible implementation, before performing a timing trigger on each task in the target scheduling resources corresponding to each project, the processing unit is further configured to:

[0049] Receive a second request for applying for scheduling resources for a newly added project, where the second request carries the number of tasks of the newly added project;

[0050] Based on the request and the scheduling resource pool of the task scheduling system, allocate corresponding new scheduling resources to the new project.

[0051] In a third aspect, a computer device is provided, and the computer device includes:

[0052] A memory for storing program instructions;

[0053] A processor for calling the program instructions stored in the memory and executing the steps included in any of the methods in the first aspect according to the obtained program instructions.

[0054] In a fourth aspect, a computer-readable storage medium is provided, and a computer program is stored on the computer-readable storage medium, and the computer program is executed by a processor to implement the steps included in any of the methods in the first aspect.

[0055] In a fifth aspect, a computer program product is provided, and when the computer program product runs on a computer device, the computer device can execute the steps included in any of the methods in the first aspect.

[0056] The technical solutions provided by the embodiments of the present invention at least bring the following beneficial effects:

[0057] In the embodiments of the present invention, each task in the target scheduling resources corresponding to each project can be triggered regularly, the execution information corresponding to each task after being triggered regularly is generated into an execution record, and the execution record is recorded in the local message list of the database. In this way, the execution status information corresponding to each task in the local message list can be queried to obtain a query result. Therefore, when it is determined based on the query result that there is a task to be issued in the local message list, the task to be issued is published to the homonymous message queues corresponding to each project; when it is determined that the task to be issued has been published to the homonymous message queue, the execution status of the task to be issued in the execution record is updated to issued.

[0058] That is to say, each time a query is made, since only the execution records of the tasks to be issued and the tasks that have been issued but not completed are stored in the local message list, and when the task scheduling system waits or is abnormal due to insufficient scheduling resources, the status of the task execution record will not be updated, so the task will be issued again, thus avoiding the loss of tasks.

[0059] Other features and advantages of the present invention will be described in the following specification, and, in part, will be obvious from the specification, or will be understood by implementing the present invention. The objectives and other advantages of the present invention can be achieved and obtained by the structures specifically pointed out in the written specification, claims, and drawings. Description of the Drawings

[0060] The accompanying drawings here are incorporated into the specification and form a part of this specification, showing embodiments consistent with the present invention, and are used together with the specification to explain the principles of the present invention, and do not constitute an improper limitation of the present invention.

[0061] Figure 1 It is a schematic diagram of an application scenario in an embodiment of the present invention;

[0062] Figure 2 It is a schematic block diagram of a task scheduling system in an embodiment of the present invention;

[0063] Figure 3 It is a schematic diagram of the process of applying for scheduling resources in an embodiment of the present invention;

[0064] Figure 4 It is a flowchart of a task scheduling method in an embodiment of the present invention;

[0065] Figure 5 It is a schematic diagram of issuing a task to be executed in an embodiment of the present invention;

[0066] Figure 6 It is a schematic diagram of executing a task in an embodiment of the present invention;

[0067] Figure 7 It is a structural block diagram of a task scheduling device in an embodiment of the present invention;

[0068] Figure 8 It is a schematic diagram of the structure of a computer device in an embodiment of the present invention. Detailed implementation manners

[0069] To make the objectives, technical solutions and advantages of the present invention clearer and more understandable, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some of the embodiments of the present invention, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without making creative efforts belong to the scope of protection of the present invention. Without conflict, the embodiments in the present invention and the features in the embodiments can be combined with each other arbitrarily. And although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than here.

[0070] In the description, claims and drawings of the present invention, the terms "first" and "second" are used to distinguish different objects, rather than to describe a specific order. In addition, the term "comprising" and any variations thereof are intended to cover non-exclusive protection. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally further include steps or units not listed, or may optionally further include other steps or units inherent to these processes, methods, products or devices.

[0071] As mentioned above, in the prior art, when performing task scheduling, there may be a situation where tasks are lost, resulting in the inability to execute subsequent projects.

[0072] In view of this, an embodiment of the present invention provides a task scheduling method and apparatus. In this method, a local message list for recording execution records is introduced. Specifically, each task is triggered at regular intervals, and the execution information corresponding to each task after being triggered at regular intervals is generated into an execution record, and then the execution record is persisted to a database. Thus, based on the execution status information corresponding to each task in the local message list, each task is dispatched and the execution status of each task is updated, so as to ensure that tasks are finally dispatched as much as possible in case of anomalies.

[0073] After introducing the design concept of the embodiment of the present invention, the following briefly introduces the application scenarios applicable to the technical solutions in the embodiment of the present invention. It should be noted that the description of the application scenarios in the embodiment of the present invention is to more clearly illustrate the technical solutions in the embodiment of the present invention, and does not constitute a limitation on the technical solutions provided in the embodiment of the present invention. Those skilled in the art know that with the emergence of new application scenarios, the technical solutions provided in the embodiment of the present invention are equally applicable to similar technical problems.

[0074] In the embodiment of the present invention, the technical solution can be applied to any scenario that requires task scheduling, and the embodiment of the present invention does not make any limitations in this regard.

[0075] In the embodiment of the present invention, please refer to Figure 1 As shown in a schematic diagram of an application scenario. In this scenario, it includes a first electronic device 101 used by a project administrator who applies for scheduling resources, a second electronic device 102 used by a system management that reviews project applications, a computer device 103 of a deployment task scheduling system that performs task scheduling on scheduling resources, and a database for storing data.

[0076] Among them, the computer device 103 may include one or more processors, a memory, an I / O interface for interacting with the first electronic device 101, and an I / O interface for interacting with the second electronic device 102, etc.

[0077] In a specific implementation process, multiple computer devices 103 can interact with multiple first electronic devices 101, or one computer device 103 can interact with multiple first electronic devices 101, or one computer device 103 can interact with one first electronic device 101. There is no limitation in the embodiments of the present invention. It should be noted that Figure 1 only the interaction between one first electronic device 101, the second electronic device 102, and the computer device 103 is shown.

[0078] In a specific implementation process, the first electronic device 101 and the computer device 103 can be communicatively connected through one or more networks. The second electronic device 102 and the computer device 103 can also be communicatively connected through one or more networks. The network can be a wired network or a wireless network. For example, the wireless network can be a mobile cellular network or a Wireless-Fidelity (WIFI) network. Of course, it can also be other possible networks, and there is no limitation in the embodiments of the present invention.

[0079] Please refer to Figure 2 shown, which is a schematic diagram of a system applicable to the embodiments of the invention. The figure includes a scheduling resource management module 201 and a task execution module 202. The task scheduling method of the embodiments of the present invention can be achieved through Figure 1 the cooperation of the scheduling resource management module 201 and the task execution module 202 in it.

[0080] In a specific implementation process, the scheduling resource management module 201 can receive information sent by a system administrator or a project administrator based on their corresponding electronic devices, and then process the information to achieve the management of scheduling resources.

[0081] In a specific implementation process, the task execution module 202 can trigger each task in the target scheduling resources corresponding to each project at a fixed time, generate an execution record for the execution information corresponding to each task after the fixed-time trigger, and record the execution record in the local message list in the database; query the execution status information corresponding to each task in the local message list to obtain a query result; when it is determined based on the query result that there are tasks to be dispatched in the local message list, publish the tasks to be dispatched to the same-name message queues corresponding to each project; when it is determined that the tasks to be dispatched have been published to the same-name message queues, update the execution status of the tasks to be dispatched in the execution record to dispatched.

[0082] Please refer to Figure 3 shown, which is a schematic diagram of the process of applying for scheduling resources shown in the embodiments of the present invention. Specifically, a project administrator can, based on their corresponding first electronic device, and the first electronic device is, for example, Figure 2The first client shown submits a scheduling resource application approval request to the task scheduling system. Then, the task scheduling system can receive the application approval request for the corresponding project and create a corresponding resource approval record in the database.

[0083] Further, after the task scheduling system creates the corresponding project approval record, it can also send a prompt message to the system administrator. Thus, the system administrator can, based on its corresponding second electronic device, that is, Figure 2 the first client shown in [], send a request to the task scheduling system to query the newly created resource approval record. Thus, the task scheduling system can query the details of the resource approval record from the database, determine the query result, and feedback the query result to the second client. Specifically, after the system administrator approves, an indication message of the approval of the application resource can be sent to the task scheduling system based on the second client. Thus, based on this indication message, the task scheduling system allocates the scheduling resources for the project, such as updating the project scheduling resources and binding the execution nodes. After the task scheduling system determines that the scheduling resource allocation is completed, it can feedback corresponding information to the second client and the first client respectively to prompt that the scheduling resource application has been approved.

[0084] It should be noted that in the actual implementation process, when the project scheduling resources are insufficient, the project administrator can continue to apply for scheduling resources to the task scheduling system based on the first client. After the system administrator approves based on its corresponding second client, the scheduling resources of the project will take effect automatically.

[0085] To further illustrate the solution of the task scheduling method provided by the embodiments of the present invention, the following will be described in detail in combination with the accompanying drawings and specific implementation manners. Although the embodiments of the present invention provide the method operation steps as shown in the following embodiments or drawings, in the described method, there may be more or fewer operation steps based on routine or non-creative labor. In steps where there is no necessary causal relationship logically, the execution order of these steps is not limited to the execution order provided by the embodiments of the present invention. In the actual processing process of the method or when the device executes, it can be executed in the method order shown in the embodiments or drawings or executed in parallel (for example, in an application environment of a parallel processor or multi-threaded processing).

[0086] The following will be combined with Figure 4 the following shown method flowchart to illustrate the task scheduling method in the embodiments of the present invention. Figure 4 Each step shown can be executed by a computer device such as Figure 1 shown. In the specific implementation process, the computer device can be a server, such as a personal computer, a large or medium-sized computer, a computer cluster, etc.

[0087] Step 401: Trigger each task in the target scheduling resources corresponding to each project at regular intervals, generate an execution record for the execution information corresponding to each task after the regular trigger, and record the execution record in the local message list of the database.

[0088] In the embodiment of the present invention, before triggering each task in the target scheduling resources corresponding to each project at regular intervals, the task scheduling system may receive a first request for scheduling resources corresponding to the first project, where the first request carries the project type of the first project, and then may determine the target scheduling resources corresponding to the first project based on the project type and the preset allocation rule.

[0089] In the embodiment of the present invention, the first project may be any project, and the task scheduling system may simultaneously receive first requests for scheduling resources of multiple projects. Then, the task scheduling system may divide the node resources based on the project types of the projects, and automatically register the task executor under the specified project. Specifically, the task executor may allocate scheduling resources for each project based on the preset allocation rule or the information generated by the corresponding operation of the system administrator, that is, the tasks in the target scheduling resources corresponding to each project are executed by the task executors under their respective projects, so as to ensure that the execution of tasks between projects does not affect each other. It can be seen that in the embodiment of the present invention, the scheduling of tasks in the target scheduling resources corresponding to each project can be efficiently and quickly implemented, and the scheduling processing of tasks in the target scheduling resources corresponding to each project is independently executed.

[0090] In the embodiment of the present invention, the task scheduling system may adopt but is not limited to the following steps to determine the target scheduling resources corresponding to the first project:

[0091] Step a: Determine whether the utilization rate of the scheduling resource pool of the task scheduling system reaches a preset threshold.

[0092] In the embodiment of the present application, the utilization rate of the scheduling resource pool of the task scheduling system may be judged. Specifically, it may be determined whether the utilization rate of the scheduling resource pool of the task scheduling system reaches a preset threshold. Among them, the preset threshold may be determined according to the actual implementation, and the embodiment of the present invention does not limit this. For example, the preset threshold is 90%.

[0093] Specifically, when the task scheduling system determines that the utilization rate of its corresponding scheduling resource pool reaches the preset threshold, it may send a warning message to the first electronic device corresponding to the project administrator, prompting that tasks exceeding the number of scheduling resources will queue up and enter the waiting queue, and wait to run in turn after the previous tasks are executed.

[0094] Specifically, when the task scheduling system determines that the utilization rate of its corresponding scheduling resource pool has not reached the preset threshold, step b is executed: Based on the preset allocation rule, determine the scheduling resource ratio corresponding to the project type.

[0095] In the embodiments of the present invention, the project type can be, for example, a production environment type, a development and debugging type, a test and acceptance type, and so on;

[0096] The preset allocation rule is, for example, the first rule. Specifically, for the production environment type, the development and debugging type, and the test and acceptance type, the respective allocated resource numbers are 10%, 1%, and 1% of the total number of scheduling resources in the scheduling resource pool of the task scheduling system. It should be noted that the preset allocation rule can be determined corresponding to the type of project in actual implementation, and the embodiments of the present invention do not limit this.

[0097] Step c: Based on the scheduling resource ratio and the scheduling resource pool, determine the target scheduling resources allocated to the first project.

[0098] For example, assume that there are 1000 scheduling resources in the scheduling resource pool of the task scheduling system and none of them are applied for use. If the project type of the first project is the production environment type, 100 scheduling resources are determined to be allocated to the first project according to the aforementioned first rule; if the project type of the first project is the development and debugging type, 10 scheduling resources are allocated to the first project; if the project type of the first project is the test and acceptance type, 10 scheduling resources are allocated to the first project.

[0099] Another example, assume that there are 1000 scheduling resources in the scheduling resource pool of the task scheduling system, 700 of which have been applied for and 300 are not applied for. If the project type of the first project is the production environment type, then 30 scheduling resources are determined to be allocated to the first project; if the project type of the first project is the development and debugging type, then 3 scheduling resources are determined to be allocated to the first project; if the project type of the first project is the test and acceptance type, then 3 scheduling resources are determined to be allocated to the first project.

[0100] In the embodiments of the present invention, when the utilization rate of the scheduling resource pool of the task scheduling system reaches the first threshold, which is 70% for example, the task scheduling system can send a prompt message to the second electronic device used by the system administrator, prompting that the scheduling resource pool of the task scheduling system needs to be expanded.

[0101] In the embodiments of the present invention, the task scheduling system can also receive a second request for applying for scheduling resources for a newly added project. The second request carries the number of tasks of the newly added project, and then corresponding newly added scheduling resources can be allocated to the newly added project based on the request and the scheduling resource pool of the task scheduling system.

[0102] It can be seen that in the embodiments of the present invention, a method for allocating scheduling resources to projects is further provided, thereby enriching the implementation methods for determining the scheduling resources corresponding to projects and enhancing the applicability of the solutions provided in the embodiments of the present invention.

[0103] In the embodiments of the present invention, after the task scheduling system determines the target scheduling resources corresponding to each project, it can trigger each task in the target scheduling resources corresponding to each project at regular intervals. Specifically, when a task is triggered, an execution record can be generated based on the execution information of the task and persisted in the database, that is, the execution record is recorded in the local message list in the database. In addition, the status information in the execution records corresponding to each task can be initialized to the to-be-issued status.

[0104] Specifically, the execution record at least includes code backup information and the execution status information of the task; among them, the code backup information is used for task execution in subsequent steps, and the execution status information of the task is used to determine whether to issue and manage the number of running tasks in subsequent steps.

[0105] Step 402: Query the execution status information corresponding to each task in the local message list to obtain a query result; where the execution status information is used to indicate whether any task has been issued.

[0106] Step 403: When it is determined based on the query result that there are tasks to be issued in the local message list, publish the tasks to be issued to the message queues with the same name corresponding to each project.

[0107] In the embodiments of the present invention, the task scheduling system can create respective message queues for each project, and the name of the message queue is the same as the project name of each project. For the convenience of description, hereinafter, the message queue created by the task scheduling system for any project is referred to as a message queue with the same name.

[0108] Step 404: When it is determined that the task to be issued has been published to the message queue with the same name, update the execution status of the task to be issued in the execution record to issued, and when it is determined that the execution status of all tasks in the local message list has been updated to issued, it is determined that the task scheduling is completed.

[0109] In the embodiments of the present invention, the control of scheduling resources can also be realized based on the polling trigger mechanism for querying a specified number of tasks. In this way, when the scheduling resources applied for by a project change, it can take effect immediately, and the tasks that were queued for execution before the previously applied scheduling resources of the project were modified will be executed immediately.

[0110] In an embodiment of the present invention, the task scheduling system can determine a preset query period, and then periodically query each project with scheduled resource allocation based on the preset query period. Among them, each project queries the same number of tasks in the database according to the number of tasks corresponding to the target scheduling resource. If the execution status of the task is to be dispatched, the task is dispatched to the message queue with the same name as the project. Specifically, the preset query period can be determined according to the actual implementation, and the embodiment of the present invention does not limit this.

[0111] Specifically, when it is determined that the current moment is within the preset query period, query the execution status information corresponding to each task in the local message list in the database; based on the execution status information corresponding to each task, obtain the query result; among them, the query result includes the number of tasks to be dispatched and the number of tasks that have been dispatched but not completed.

[0112] For example, the preset query period is 1 minute. When it is determined that the current moment, that is, 9:27:30, belongs to the query period of 27-28 minutes, the execution status information corresponding to each task in the local message list in the database can be queried. Assuming that it is determined that there are 3 tasks, and the execution status information of task 1 is to be dispatched, the execution status information of task 2 is to be dispatched, and the execution status information of task 3 is dispatched but not completed, then it can be determined that the query result is that there are 2 tasks to be dispatched and 1 task that has been dispatched but not completed.

[0113] Specifically, if during a certain query, based on the queried local message list, it is determined that the task list contains tasks to be dispatched, it means that the number of dispatched tasks has not reached the upper limit at this time. Therefore, the tasks to be dispatched will be dispatched, and after the dispatch, the execution status of the task to be dispatched is updated to the dispatched state; if the execution status information of the tasks recorded in the queried local message list is all dispatched, it means that the number of dispatched tasks has reached the upper limit at this time, and the tasks are still running. Therefore, no new tasks will be dispatched.

[0114] In an embodiment of the present invention, when it is determined that the query result has been empty for a preset duration, or the number of tasks to be dispatched in the query result is less than the number of tasks corresponding to the target scheduling resource within the preset duration, a first prompt message is sent to the task scheduling system; the first prompt message is used to prompt the task scheduling system to schedule and process the idle resources. Among them, the preset duration can be determined according to the actual implementation situation, for example, it is 2 times the duration corresponding to the preset query period, etc. The embodiment of the present invention does not limit this.

[0115] In an embodiment of the present invention, after the execution status of any task in the execution record is updated to dispatched, the record corresponding to any task in the local message list is deleted. In this way, it can be ensured that the dispatched tasks will not be dispatched again.

[0116] Please refer toFigure 5 , Figure 5 This is a schematic diagram of the task to be executed and issued in the embodiment of the present invention. It can be seen that each time a timed task is queried, since only the execution records of the tasks to be issued and the issued but uncompleted tasks are stored in the local message list, the query result only includes the number of tasks to be issued and the issued but uncompleted tasks, and does not include the tasks that have been issued and completed, so as to ensure that each project can run tasks according to the allocated number of tasks.

[0117] Obviously, in the case of abnormal situations such as service downtime and restart or network timeout between services, the execution record of the task will not be updated to issued. Therefore, when the timed task is queried again, the execution record of the task can be queried from the database and the task can be re-issued.

[0118] Please refer to Figure 6 , Figure 6 This is a schematic diagram of task execution in the embodiment of the present invention. Specifically, the task scheduling system may include message queues with the same name corresponding to multiple projects, and each queue includes multiple task execution nodes. In the actual implementation process, the project administrator can apply according to actual needs, and create any number of projects based on the first electronic device. Thus, when the task scheduling system allocates target scheduling resources to the newly created projects, it can automatically allocate a specified number of task execution nodes for each project, and the allocated execution nodes are subscribed to the queue with the same name as the project, so as to process the tasks corresponding to the target scheduling resources corresponding to the project. It can be seen that the creation and flexible adjustment of the project-level node group can be realized based on this.

[0119] It should be noted that in the embodiment of the present invention, the message queue in the local message list is in the point-to-point consumption mode. For example, taking kafka as an example, the consumers of the execution nodes of the same project need to specify a unique GroupId. That is to say, after a message is published, it will only be consumed by one consumer, so as to ensure that after a task snapshot is published, it can only be executed by one execution node.

[0120] Based on the same inventive concept, the embodiment of the present invention provides a task scheduling device, and this task scheduling device can implement the functions corresponding to the foregoing task scheduling method. This task scheduling device can be a hardware structure, a software module, or a combination of a hardware structure and a software module. This task scheduling device can be implemented by a chip system, and the chip system can be composed of chips, or can include chips and other discrete devices. Please refer to Figure 7 As shown in the figure, this task scheduling device includes: a processing unit 701, a query unit 702, a distribution unit 703, and an update unit 704, where:

[0121] The processing unit 701 is configured to trigger each task in the target scheduling resources corresponding to each project at regular intervals, generate an execution record for the execution information corresponding to each task after the regular triggering, and record the execution record in the local message list in the database; wherein, the execution record at least includes code backup information and the execution status information of the task.

[0122] The query unit 702 is configured to query the execution status information corresponding to each task in the local message list to obtain a query result; wherein, the execution status information is used to indicate whether any of the tasks has been dispatched.

[0123] The dispatch unit 703 is configured to, when it is determined based on the query result that there are tasks to be dispatched in the local message list, publish the tasks to be dispatched to the homonymous message queues corresponding to each project.

[0124] The update unit 704 is configured to, when it is determined that the task to be dispatched has been published to the homonymous message queue, update the execution status of the task to be dispatched in the execution record to dispatched, and when it is determined that the execution status of all tasks in the local message list has been updated to dispatched, determine that the task scheduling is completed.

[0125] In a possible implementation manner, before triggering each task in the target scheduling resources corresponding to each project at regular intervals, the processing unit 701 is further configured to:

[0126] Receive a first request for scheduling resources corresponding to a first project, where the first request carries the project type of the first project.

[0127] Based on the project type and a preset allocation rule, determine the target scheduling resources corresponding to the first project.

[0128] In a possible implementation manner, the query unit 702 is specifically configured to:

[0129] Determine a preset query period.

[0130] When it is determined that the current moment is within the preset query period, query the execution status information corresponding to each task in the local message list in the database.

[0131] Based on the execution status information corresponding to each task, obtain a query result; wherein, the query result includes the number of tasks to be dispatched and the number of tasks that have been dispatched but not completed.

[0132] In a possible implementation manner, the apparatus further includes a deletion unit, configured to:

[0133] After the execution status of any task in the execution record is updated to dispatched, delete the record corresponding to the any task in the local message list.

[0134] In a possible implementation, after querying the number of tasks to be dispatched in the local message list in the database, the device further includes a prompting unit for:

[0135] When it is determined that the query result has been empty within a preset time period, or the number of tasks to be dispatched in the query result is less than the number of tasks corresponding to the target scheduling resources within the preset time period, send a first prompting message to the task scheduling system; the first prompting message is used to prompt the task scheduling system to perform scheduling processing on idle resources.

[0136] In a possible implementation, the processing unit 701 is specifically configured to:

[0137] Determine whether the usage rate of the scheduling resource pool of the task scheduling system reaches a preset threshold;

[0138] When it is determined that the usage rate does not reach the preset threshold, determine the scheduling resource ratio corresponding to the project type based on the preset allocation rule;

[0139] Based on the scheduling resource ratio and the scheduling resource pool, determine the target scheduling resources allocated to the first project.

[0140] In a possible implementation, before timing triggering each task in the target scheduling resources corresponding to each project, the processing unit 701 is further configured to:

[0141] Receive a second request for applying for scheduling resources for a newly added project, where the second request carries the number of tasks of the newly added project;

[0142] Based on the request and the scheduling resource pool of the task scheduling system, allocate corresponding newly added scheduling resources to the newly added project.

[0143] All relevant contents of each step involved in the embodiment of the foregoing task scheduling method can be cited in the function description of the corresponding functional module of the task scheduling device in the embodiment of the present invention, and will not be repeated here.

[0144] The division of modules in the embodiments of the present invention is illustrative, and is only a logical function division. In actual implementation, there may be other division methods. In addition, in each embodiment of the present invention, each functional module may be integrated in a controller, may exist separately physically, or two or more modules may be integrated in one module. The above integrated modules may be implemented in the form of hardware or in the form of software functional modules.

[0145] Based on the same inventive concept, an embodiment of the present invention provides a computer device. Please refer to Figure 8 As shown, the computer device includes at least one processor 801 and a memory 802 connected to the at least one processor. In the embodiment of the present invention, the specific connection medium between the processor 801 and the memory 802 is not limited. Figure 8 Taking the connection between the processor 801 and the memory 802 through the bus 800 as an example, the bus 800 is Figure 8 represented by a thick line in Figure 8 The connection manners between other components are only for illustrative purposes and are not to be construed as limiting. The bus 800 can be divided into an address bus, a data bus, a control bus, etc. For the sake of convenience of representation,

[0146] in

[0147] it is only represented by a thick line, but it does not mean that there is only one bus or one type of bus. In addition, the task scheduling device further includes a communication interface 803 for receiving or sending data.

[0148] Optionally, the processor 801 may include one or more processing units. The processor 801 may integrate an application processor and a modem processor. Among them, the application processor mainly processes the operating system, user interface, application programs, etc., and the modem processor mainly processes wireless communication. It can be understood that the above-mentioned modem processor may not be integrated into the processor 801. In some embodiments, the processor 801 and the memory 802 may be implemented on the same chip, and in some embodiments, they may also be separately implemented on independent chips.

[0149] The processor 801 may be a general-purpose processor, such as a central processing unit (CPU), a digital signal processor, an application specific integrated circuit, a field programmable gate array, or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, and can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of the present invention. The general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of the method disclosed in combination with the embodiments of the present invention may be directly embodied as being executed by a hardware processor, or executed by a combination of hardware and software modules in the processor.

[0150] The memory 802, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules. The memory 802 may include at least one type of storage medium, for example, it may include flash memory, hard disk, multimedia card, card-type memory, random access memory (RAM), static random access memory (SRAM), programmable read-only memory (PROM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), magnetic memory, magnetic disk, optical disk, and so on. The memory 802 is any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 802 in the embodiments of the present invention may also be a circuit or any other device capable of implementing a storage function, for storing program instructions and / or data.

[0151] By designing and programming the processor 801, the code corresponding to the task scheduling method introduced in the foregoing embodiments can be solidified into the chip, so that the chip can execute the steps of the foregoing task scheduling method when running. How to design and program the processor 801 is a well-known technology to those skilled in the art and will not be elaborated here.

[0152] Based on the same inventive concept, the embodiments of the present invention also provide a computer-readable storage medium, which stores a computer program, and when the computer program is executed by a processor, it implements the steps of the task scheduling method as described above.

[0153] In some possible embodiments, various aspects of the task scheduling method provided by the present invention may also be implemented in the form of a program product, which includes program code. When the program product runs on a control computer device, the program code is used to cause the control computer device to execute the steps in the task scheduling method according to various exemplary embodiments of the present invention described above in this specification.

[0154] Those skilled in the art should understand that the embodiments of the present invention can be provided as a method, a system, or a computer program product. Therefore, the present invention can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk memories and optical memories, etc.) containing computer-usable program code.

[0155] The present invention is described with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems), and computer program products according to embodiments of the present invention. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, as well as the combination of flows and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate means for implementing the functions specified in Figure 1 one or more of the flows Figure 1 or blocks or combinations of blocks.

[0156] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory generate a manufactured article including instruction means, and the instruction means implements the functions specified in Figure 1 one or more of the flows Figure 1 or blocks or combinations of blocks.

[0157] These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process, and thus the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in Figure 1 one or more of the flows Figure 1 or blocks or combinations of blocks.

[0158] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalent technologies, the present invention also intends to include these modifications and variations.

Claims

1. A task scheduling method, characterized in that, Applied to a task scheduling system, the method includes: Timely trigger each task in the target scheduling resources corresponding to each project, generate an execution record for the execution information corresponding to each task after the timely trigger, and record the execution record in the local message list in the database; wherein, the execution record at least includes code backup information and the execution status information of the task. Query the execution status information corresponding to each task in the local message list to obtain a query result; wherein, the execution status information is used to indicate whether any of the tasks has been dispatched. When it is determined based on the query result that there are tasks to be dispatched in the local message list, publish the tasks to be dispatched to the homonymous message queues corresponding to each project. When it is determined that the tasks to be dispatched have been published to the homonymous message queues, update the execution status of the tasks to be dispatched in the execution record to dispatched, and when it is determined that the execution status of all tasks in the local message list has been updated to dispatched, determine that the task scheduling is completed.

2. The method according to claim 1, characterized in that, Before timely triggering each task in the target scheduling resources corresponding to each project, the method further includes: Receive a first request for scheduling resources corresponding to the first project, where the first request carries the project type of the first project. Based on the project type and a preset allocation rule, determine the target scheduling resources corresponding to the first project.

3. The method according to claim 1 or 2, characterized in that, Query the execution status information corresponding to each task in the local message list in the database, including: Determine a preset query period. When it is determined that the current moment is within the preset query period, query the execution status information corresponding to each task in the local message list in the database. Based on the execution status information corresponding to each task, obtain a query result; wherein, the query result includes the number of tasks to be dispatched and the number of tasks that have been dispatched but not completed.

4. The method according to claim 1 or 2, characterized in that, The method further includes: When the execution status of any task in the execution record is updated to dispatched, delete the record corresponding to the any task in the local message list.

5. The method according to claim 1 or 2, characterized in that, After querying the number of tasks to be dispatched in the local message list in the database, the method further includes: When it is determined that the query result has been empty for a preset duration, or the number of tasks to be dispatched in the query result is less than the number of tasks corresponding to the target scheduling resources within the preset duration, send a first prompt message to the task scheduling system; the first prompt message is used to prompt the task scheduling system to perform scheduling processing on idle resources.

6. The method according to claim 2, characterized in that, Based on the project type and a preset allocation rule, determining the target scheduling resources corresponding to the first project includes: Determine whether the utilization rate of the scheduling resource pool of the task scheduling system reaches a preset threshold. When it is determined that the utilization rate does not reach the preset threshold, then based on the preset allocation rule, determine the scheduling resource ratio corresponding to the project type. Based on the scheduling resource ratio and the scheduling resource pool, determine the target scheduling resources allocated to the first project.

7. The method according to claim 1, characterized in that, Before timely triggering each task in the target scheduling resources corresponding to each project, the method further includes: Receive a second request for scheduling resources for a newly added project, where the number of tasks of the newly added project is carried in the second request; Based on the request and the scheduling resource pool of the task scheduling system, allocate corresponding newly added scheduling resources for the newly added project.

8. A task scheduling device, characterized in that, Applied to a task scheduling system, the device includes: A processing unit, configured to perform timed triggering on each task in the target scheduling resources corresponding to each project, generate an execution record for the execution information corresponding to each task after the timed triggering, and record the execution record in the local message list in the database; wherein, the execution record at least includes code backup information and the execution status information of the task; A query unit, configured to query the execution status information corresponding to each task in the local message list to obtain a query result; wherein, the execution status information is used to indicate whether any of the tasks has been dispatched; A dispatching unit, configured to, when it is determined based on the query result that there are tasks to be dispatched in the local message list, publish the tasks to be dispatched to the same-name message queues corresponding to each project; An updating unit, configured to, when it is determined that the tasks to be dispatched have been published to the same-name message queues, update the execution status of the tasks to be dispatched in the execution record to dispatched, and, when it is determined that the execution status of all tasks in the local message list has been updated to dispatched, determine that the task scheduling is completed.

9. A computer device, characterized in that, The computer device includes: a memory, a processor, and a computer program stored on the memory and executable on the processor, and when the computer program is executed by the processor, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that, A computer program is stored on the computer-readable storage medium, and when the computer program is executed by the processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images