Task scheduling control method, device and system and computer readable storage medium
By splitting the risk control tasks into subtasks and executing them in parallel in the message queue of the scheduling server, the problem of low scheduling efficiency caused by asynchronous task blocking is solved, and the real-time and throughput of task scheduling in scenarios such as risk control platforms are improved.
Patent Information
- Application Number
- CN202411322260.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-20
- Publication Date
- 2024-12-20
AI Technical Summary
In the prior art, the blocking demand of asynchronous tasks leads to low task scheduling efficiency, making it difficult to meet the real-time requirements of decision-making scenarios such as risk control platforms. The tasks that could have been executed synchronously are separated into front and back orders.
By splitting the risk control tasks into a set of subtasks and adding the target subtasks that meet the preset push conditions to the message queue of the scheduling server, the parallel execution of subtasks is realized to avoid blocking scheduling of asynchronous tasks in the message queue.
It improves the task execution efficiency and throughput of the scheduling system, meets the real-time requirements of scenario scheduling, avoids the blockage of tasks in the queue, and realizes parallel execution of tasks.
Smart Images
Figure CN119166309A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of distributed data processing technology, and in particular to a task scheduling control method, device, system and computer-readable storage medium. Background Art
[0002] With the continuous discovery of science and technology, people have higher and higher requirements for data processing. In order to meet the processing needs of multiple tasks, the scheduling system application was born; the scheduling system is a system used to manage and allocate computing resources to perform tasks. The demand for multi-tasking has promoted the development of the scheduling system to ensure that the system can efficiently process multiple tasks without resource contention. For example, in the financial field, the risk control platform receives multiple risk indicator calculation tasks for business scenarios, and manages and allocates computing resources through the scheduling system to perform multiple risk indicator calculation tasks of the risk control platform.
[0003] However, during the actual research and development process, the inventors of the present application discovered that in the related technology, in order to meet the blocking requirements of asynchronous tasks, the next task scheduling is initiated only after the current task scheduling is completed, resulting in two scheduled tasks that could have been executed synchronously being separated into a front-to-back sequence, and the task execution efficiency is low, making it difficult to meet the real-time scheduling requirements of decision-making scenarios such as risk control platforms. Summary of the invention
[0004] The present application provides a task scheduling control method, device, system and computer-readable storage medium, which can improve the task execution efficiency of the scheduling system and meet the real-time requirements of scene scheduling to a certain extent.
[0005] In a first aspect, the present application provides a task scheduling control method, which is applied to a scheduling server in a task scheduling system, wherein the task scheduling system includes a service interface layer, a scheduling server, and a plurality of task executors, and the method includes:
[0006] When the service interface layer receives the risk control task from the business platform, it receives a scheduling request for the risk control task initiated by the service interface layer;
[0007] In response to the scheduling request, split the risk control task to obtain a subtask set of the risk control task;
[0008] Adding the target subtask that meets the preset push condition in the subtask set to the message queue of the scheduling server;
[0009] The target subtask in the message queue is distributed to a target task executor, so as to call the target task executor to execute the target subtask.
[0010] In a second aspect, the present application provides a task scheduling control device, the task scheduling control device comprising:
[0011] A receiving unit, configured to receive a scheduling request for the risk control task initiated by the service interface layer when the service interface layer receives the risk control task of the business platform;
[0012] A response unit, configured to respond to the scheduling request and split the risk control task to obtain a subtask set of the risk control task;
[0013] A push unit, used to add a target subtask in the subtask set that meets a preset push condition to a message queue of the scheduling server;
[0014] The distribution unit is used to distribute the target subtask in the message queue to the target task executor to call the target task executor to execute the target subtask.
[0015] In the third aspect, the present application also provides a task scheduling system, which includes a service interface layer, a state machine, a scheduling server and multiple task executors, and the scheduling server includes a memory and a processor; the memory is used to store computer programs; the processor is used to execute the computer program and implement the task scheduling control method when executing the computer program.
[0016] In a fourth aspect, the present application also provides a computer-readable storage medium on which a computer program is stored, and the computer program is loaded by a processor to execute the task scheduling control method.
[0017] In this application, on the one hand, by adding the target subtasks that meet the preset push conditions in the subtask set to the message queue of the scheduling server, the scheduling tasks that do not have serial dependencies can be executed in parallel, thereby improving the throughput of the scheduling system. On the other hand, by splitting the risk control task into multiple subtasks, the long task can be split into different executor threads for processing, thereby reducing the impact of the long task on the throughput of the scheduling system to a certain extent. On the third hand, by adding the target subtasks that meet the preset push conditions to the message queue of the scheduling server when the subtask meets the preset push conditions, the asynchronous tasks can be blocked outside the message queue, avoiding the blocking scheduling when the asynchronous tasks enter the message queue and are distributed, thereby avoiding the problem that the message queue needs to wait for the first task to complete and return the result before distributing the next task, thereby avoiding the problem that two tasks that could have been executed synchronously are asynchronously executed due to blocking requirements, so that each scheduling task in the message queue can be executed in parallel, improving the throughput of the scheduling system per unit time, and improving the throughput of the scheduling system under the condition of meeting the blocking scheduling of asynchronous tasks, thereby improving the real-time performance of task scheduling. Thus, the task execution efficiency of the scheduling system can be improved, and the real-time performance requirements of scene scheduling can be met to a certain extent. BRIEF DESCRIPTION OF THE DRAWINGS
[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without paying any creative work.
[0019] Figure 1 It is a structural schematic block diagram of a task scheduling system provided in an embodiment of the present application;
[0020] Figure 2 is a schematic block diagram of the structure of a scheduling server provided in an embodiment of the present application;
[0021] Figure 3 This is a flowchart of a task scheduling control method provided by an embodiment of the present application;
[0022] Figure 4 It is a schematic diagram for explaining the serial dependency relationship between tasks in a subtask set in an embodiment of the present application;
[0023] Figure 5 is another schematic diagram illustrating the serial dependency relationship between tasks in a subtask set in an embodiment of the present application;
[0024] Figure 6 It is a schematic diagram illustrating the overall structural framework of the task scheduling system in the embodiment of the present application;
[0025] Figure 7 It is a schematic diagram of the structure of an embodiment of the task scheduling control device provided in the embodiment of the present application. DETAILED DESCRIPTION
[0026] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.
[0027] The flowcharts shown in the accompanying drawings are only examples and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may also be decomposed, combined or partially merged, so the actual execution order may change according to actual conditions.
[0028] In the description of the embodiments of the present application, it should be understood that the terms "first" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of the present application, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined.
[0029] In order to enable any person skilled in the art to implement and use the present application, the following description is provided. In the following description, details are listed for the purpose of explanation. It should be understood that those of ordinary skill in the art can recognize that the present application can also be implemented without using these specific details. In other examples, the known process will not be elaborated in detail to avoid unnecessary details that make the description of the present application embodiment obscure. Therefore, the present application is not intended to be limited to the embodiments shown, but is consistent with the widest range of principles and features disclosed in accordance with the embodiments of the present application.
[0030] Embodiments of the present application provide a task scheduling control method, device, system, and computer-readable storage medium.
[0031] The executor of the task scheduling control method of the embodiment of the present application may be the task scheduling control device provided in the embodiment of the present application, or the scheduling server provided in the embodiment of the present application, wherein the task scheduling control device may be implemented in hardware or software.
[0032] Some embodiments of the present application are described in detail below in conjunction with the accompanying drawings. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.
[0033] Figure 1 It is a structural schematic block diagram of a task scheduling system provided in an embodiment of the present application.
[0034] like Figure 1 As shown, the task scheduling system 100 includes a service interface layer 101 , a scheduling server 102 and a plurality of task executors 103 .
[0035] The service interface layer 101 is used to provide a task scheduling control entry for the business platform, including but not limited to: initiating scheduling, suspending scheduling, stopping scheduling, and querying the execution status of scheduling tasks (such as the entire risk control task and each subtask of the risk control task). The business platform is the source platform of the risk control task. For example, in the scenario of calculating the risk index of the risk control platform, the business platform can be the risk control platform.
[0036] The scheduling server 102 is responsible for splitting the risk control task into multiple subtasks, pushing each subtask that meets the execution condition (i.e., meets the preset push condition) at the current time point as a scheduling task to the message queue, and distributing each scheduling task in the message queue (including but not limited to: such as the entire risk control task, each subtask of the risk control task) to the corresponding task executor 103, refer to Figure 6 It is understandable that when the risk control task needs to be split, the scheduling server 102 splits the risk control task into multiple subtasks and pushes each subtask to the message queue as a scheduling task; when the risk control task does not need to be split, the scheduling server 102 can also push the entire risk control task to the message queue as a scheduling task.
[0037] Among them, the task executor 103 is responsible for the execution of scheduling tasks (including but not limited to: the entire risk control task, the various subtasks of the risk control task). The task executor 103 can read and execute scheduling tasks (including but not limited to: the entire risk control task, the various subtasks of the risk control task) from the message queue.
[0038] Furthermore, the task scheduling system 100 may also include a state machine 104. The state machine 104 is used to maintain the state of the scheduling task (such as the entire risk control task, each subtask of the risk control task).
[0039] Further, refer to Figure 6 ,Scheduling tasks (such as risk control tasks, subtasks of risk control tasks, ,the execution status of risk control tasks, and the execution status of ,subtasks) can be maintained in the database in a list format.,For example, the execution status of the scheduling tasks can be ,recorded in the database.
[0040] See also Figure 2 , Figure 2 A schematic block diagram of the structure of a scheduling server provided in an embodiment of the present application. The scheduling server can be in the form of a computer device or a software system.
[0041] like Figure 2 As shown, the scheduling server includes a processor, a memory and a network interface connected through a system bus, wherein the memory may include a non-volatile storage medium and an internal memory.
[0042] The non-volatile storage medium can store an operating system and a computer program. The computer program includes program instructions, and when the program instructions are executed, the processor can execute any task scheduling control method.
[0043] The processor is used to provide computing and control capabilities to support the operation of the entire scheduling server.
[0044] The internal memory provides an environment for the operation of the computer program in the non-volatile storage medium. When the computer program is executed by the processor, the processor can execute any task scheduling control method.
[0045] The network interface is used for network communication, such as sending assigned tasks, etc. Those skilled in the art will understand that Figure 2 The structure shown in the figure is only a block diagram of a partial structure related to the scheme of the present application, and does not constitute a limitation on the scheduling server to which the scheme of the present application is applied. The specific scheduling server may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.
[0046] It should be understood that the processor may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.
[0047] In one embodiment, the processor is used to run a computer program stored in the memory to implement the following steps:
[0048] When the service interface layer receives a risk control task from the business platform, it receives a scheduling request for the risk control task initiated by the service interface layer; in response to the scheduling request, it splits the risk control task to obtain a subtask set of the risk control task; adds the target subtask in the subtask set that meets the preset push conditions to the message queue of the scheduling server; distributes the target subtask in the message queue to the target task executor, so as to call the target task executor to execute the target subtask.
[0049] Those skilled in the art will understand that Figure 1 or Figure 2 The structure shown in is merely a block diagram of a portion of the structure related to the embodiment of the present application, and does not constitute a limitation on the task scheduling system and scheduling server in the embodiment of the present application.
[0050] The following will Figure 1Taking the scheduling server in the task scheduling system shown in as an example as the execution subject of the task scheduling control method, the task scheduling control method provided by the embodiment of the present application is introduced in detail. It should be noted that, Figure 1 The scenario is only used to explain the task scheduling control method provided in the embodiment of the present application, but does not constitute a limitation on the application scenario of the task scheduling control method provided in the embodiment of the present application.
[0051] See also Figure 3 , Figure 3 This is a flow chart of a task scheduling control method provided by an embodiment of the present application. The task scheduling control method is applied to a scheduling server in a task scheduling system. The task scheduling system includes a service interface layer, a scheduling server, and multiple task executors. The task scheduling control method includes steps 301 to 304, wherein:
[0052] 301. When the service interface layer receives a risk control task from a business platform, it receives a scheduling request for the risk control task initiated by the service interface layer.
[0053] Among them, the business platform is the source platform of the risk control task. For example, in a scenario where the task scheduling control method is used to calculate the risk index of the risk control platform, the business platform can be the risk control platform.
[0054] Among them, the risk control task is a task to be distributed. For example, in a scenario where the task scheduling control method is used to calculate risk indicators of a risk control platform, the risk control task may be a risk indicator calculation task.
[0055] In order to better understand the embodiments of the present application, some names involved in the embodiments of the present application are first introduced below:
[0056] 1. Task scheduling process: The task scheduling process is the process of arranging, allocating and managing some tasks to ensure that these tasks are completed within the specified time and achieve the expected goals.
[0057] 2. Scheduling tasks: Scheduling tasks are specific activities or operations that need to be completed during the scheduling process. In some embodiments, the scheduling task may be the entire risk control task; in some embodiments, the scheduling task may also be the subtasks of the risk control task.
[0058] 3. Serial dependency: Serial dependency means that two scheduled tasks have a sequential execution relationship. For example, if task b can be executed only after task a is completed, then task a and task b are considered to have a serial dependency.
[0059] 4. Predecessor task: A predecessor task is a task that needs to be executed first between two scheduled tasks with a serial dependency relationship. For example, if task a and task b have a serial dependency relationship, and task b can only be executed after task a is completed, then task a is the predecessor task in this serial dependency relationship.
[0060] 5. Post-task: A post-task is a task that needs to be executed later than another task in a series dependency relationship. For example, if task a and task b have a series dependency relationship, and task b can only be executed after task a is completed, then task b is the post-task in the series dependency relationship.
[0061] Exemplarily, in order to improve the real-time processing of risk control tasks, when the business platform generates a new risk control task, it sends the risk control task to the task scheduling system; the task scheduling system receives the risk control task of the business platform through the service interface layer. When the service interface layer receives the risk control task of the business platform, the service interface layer initiates a scheduling request for the risk control task. At this time, the scheduling server can receive the scheduling request for the risk control task initiated by the service interface layer, and enter step 302 to respond to the scheduling request to complete the processing of the risk control task.
[0062] 302. In response to the scheduling request, split the risk control task to obtain a subtask set of the risk control task.
[0063] Exemplarily, step 302 may specifically include the following steps: in response to the scheduling request, obtaining the scenario type and scenario data carried by the risk control task; obtaining each target evaluation task associated with the scenario type from the preset evaluation tasks; generating task data for each target evaluation task based on the scenario data; taking the task data of each target evaluation task as a subtask to obtain a subtask set of the risk control task.
[0064] The scenario type refers to the scenario category of the risk control task. For example, if the risk control indicator calculation is performed in the small and micro pre-lease scenario, the scenario type of the risk control task is small and micro pre-lease.
[0065] Among them, scenario data is used to implement risk control tasks, for example, it may include credit data, market data, financial institution asset data, financial institution operation data, etc.
[0066] For example, taking the scenario type of risk control task as small and micro pre-rental, the preset assessment tasks include credit assessment subtasks, market risk assessment subtasks, liquidity risk assessment subtasks, operational risk assessment subtasks, and comprehensive risk assessment subtasks. The target assessment tasks associated with the scenario type of risk control task (small and micro pre-rental) are: credit assessment subtasks, market risk assessment subtasks, liquidity risk assessment subtasks, and operational risk assessment subtasks; at this time, task data of credit assessment subtasks can be generated according to scenario data (such as credit investigation data), task data of market risk assessment subtasks can be generated according to scenario data (such as market data), task data of liquidity risk assessment subtasks can be generated according to scenario data (such as financial institution asset data), and task data of operational risk assessment subtasks can be generated according to scenario data (such as financial institution operation data), so as to obtain the subtask set of risk control task.
[0067] 303. Add the target subtasks in the subtask set that meet the preset push conditions to the message queue of the scheduling server.
[0068] In this embodiment, the preset push conditions mainly include two: ① the unpushed subtask belongs to the post-task in the serial dependency relationship, and the target predecessor subtask of the unpushed subtask has been completed; ② the unpushed subtask does not belong to the post-task in the serial dependency relationship.
[0069] The fact that the unpushed subtask is a post-task in a serial dependency relationship means that the unpushed subtask has a serial dependency relationship with other scheduled tasks, and the unpushed subtask is a post-task in at least one serial dependency relationship. Figure 4 As shown, for subtask D, subtask D and other scheduled tasks (such as Figure 4 As shown, subtask D has a serial dependency relationship with subtask A, and subtask D has a serial dependency relationship with subtask G). Since subtask D has a serial dependency relationship with subtask A, and subtask D is a post-task, subtask D belongs to the post-task in the serial dependency relationship.
[0070] In some embodiments, the fact that the unpushed subtask is not a successor task in a serial dependency relationship means that the unpushed subtask has a serial dependency relationship with other scheduled tasks, and the unpushed subtask is a predecessor task in all serial dependencies. Figure 4 As shown, for subtask A, subtask A and other scheduled tasks (such as Figure 4As shown, subtask A has a serial dependency relationship with subtask D, and subtask A has a serial dependency relationship with subtask E). Since subtask A is a predecessor task in the serial dependency relationship between subtask A and subtask D, and subtask A is also a predecessor task in the serial dependency relationship between subtask A and subtask E, subtask A does not belong to the successor task in the serial dependency relationship.
[0071] Alternatively, in some embodiments, the fact that the unpushed subtask does not belong to a subsequent task in a serial dependency relationship means that the unpushed subtask does not have a serial dependency relationship with other scheduled tasks.
[0072] Exemplarily, the subtask set is traversed, and for each unpushed subtask in the subtask set, first, it is detected whether the unpushed subtask belongs to a post-task in the serial dependency relationship; when the traversed unpushed subtask does not belong to a post-task in the serial dependency relationship, at this time, referring to the method in steps 3031B to 3032B, the unpushed subtask is directly pushed to the message queue of the scheduling server to complete the execution of the unpushed subtask as soon as possible and improve the real-time performance of task scheduling. When the traversed unpushed subtask is a post-task in a serial dependency relationship, since the unpushed subtask has a serial dependency relationship with other scheduled tasks, at this time, referring to the method in steps 3031A to 3033A, it is necessary to further detect whether the predecessor task in the serial dependency relationship of the unpushed subtask is completed. When the predecessor subtask in the serial dependency relationship of the unpushed subtask is completed, the serial dependency relationship of the unpushed subtask is pushed to the message queue of the scheduling server to block the asynchronous scheduling task, avoid the problem of difficult blocking of asynchronous scheduling tasks, and realize that when the two scheduling tasks are asynchronous, the scheduling of the next scheduling task is initiated only after the current scheduling task is completed.
[0073] There are multiple ways to push the subtask to the message queue in step 303, illustratively including the following ways (1) and (2):
[0074] (1) In some embodiments, the unpushed subtask belongs to a post-task in a serial dependency relationship. In this case, step 303 may specifically include the following steps 3031A to 3034A:
[0075] 3031A. Obtain unpushed subtasks from the subtask set.
[0076] 3032A. Detect whether the unpushed subtask belongs to a post-task in the serial dependency relationship.
[0077] There are multiple ways to “detect whether the unpushed subtask belongs to a post-task in a serial dependency relationship”, illustratively including the following ways ① and ②:
[0078] Method ①: In some embodiments, detection is performed based on the node processing strategy of the unpushed subtask. In this case, "detecting whether the unpushed subtask belongs to a post-task in the serial dependency relationship" can be performed in the following steps A1 to A3:
[0079] A1. According to the business process strategy of the risk control task, a series dependency check is performed on the subtasks that have not been pushed.
[0080] A2. If the unpushed subtask has a target serial dependency relationship, and the unpushed subtask is a post-task in the target serial dependency relationship, it is determined that the unpushed subtask belongs to a post-task in the serial dependency relationship.
[0081] A3. If the unpushed subtask does not have a target serial dependency relationship, it is determined that the unpushed subtask does not belong to a post-task in the serial dependency relationship.
[0082] A4. If the unpushed subtask has a target serial dependency relationship, and the unpushed subtasks are all predecessor tasks in the target serial dependency relationship, it is determined that the unpushed subtask does not belong to a successor task in the serial dependency relationship.
[0083] For example, Figure 4 As shown, assuming that the business process strategy of the risk control task is as follows Figure 4 The task flow shown in the figure takes subtask F as an example. According to the business process strategy of the risk control task, the subtask F that has not been pushed is tested for serial dependency. Specifically, it is tested whether the subtask F that has not been pushed has a serial dependency. If the subtask F that has not been pushed has a serial dependency (such as Figure 4 As shown in the figure, subtask F has a serial dependency relationship with subtask B, and subtask F has a serial dependency relationship with subtask H), then further detect whether the unpushed subtask (subtask F) is a predecessor task in the serial dependency relationship in which the unpushed subtask (subtask F) is located. If the unpushed subtask (subtask F) is a successor task in at least one serial dependency relationship (such as Figure 4 As shown, in the serial dependency relationship between subtask F and subtask B, subtask F is a post-task), then it is determined that the unpushed subtask (subtask F) belongs to the post-task in the serial dependency relationship.
[0084] Another example: Figure 4 As shown, assuming that the business process strategy of the risk control task is as follows Figure 4 The task flow shown in the figure takes subtask A as an example. According to the business process strategy of the risk control task, the subtask (subtask A) that has not been pushed is tested for serial dependency. Specifically, it is tested whether the subtask (subtask A) that has not been pushed has a serial dependency. If the subtask (subtask A) that has not been pushed has a serial dependency (such as Figure 4 As shown in the figure, subtask A has a serial dependency relationship with subtask D, and subtask A has a serial dependency relationship with subtask E), then further check whether the unpushed subtask (subtask A) is all a predecessor task in the serial dependency relationship where the unpushed subtask (subtask A) is located. If the unpushed subtask (subtask A) has a predecessor task in all the serial dependency relationships (such as Figure 4 As shown, in the serial dependency relationship between subtask A and subtask D, subtask A is a predecessor task, and in the serial dependency relationship between subtask A and subtask E, subtask A is also a predecessor task), then it is determined that the unpushed subtask (subtask A) does not belong to the successor task in the serial dependency relationship.
[0085] For example, Figure 5 As shown, assuming that the business process strategy of the risk control task is as follows Figure 5 The task flow shown takes the unpushed subtask as subtask C as an example. According to the business process strategy of the risk control task, a serial dependency detection is performed on the unpushed subtask (subtask C). Specifically, it is detected whether there is a serial dependency between the unpushed subtask (subtask C). If there is no serial dependency between the unpushed subtask (subtask C), it is determined that the unpushed subtask (subtask C) does not belong to the post-task in the serial dependency.
[0086] Method ②: In some embodiments, each subtask in the subtask set carries a category label, and the detection is based on the category label carried by the unpushed subtask. In this case, "detecting whether the unpushed subtask belongs to a post-task in the serial dependency relationship" can be as follows: Steps B1 to B2:
[0087] B1. Obtain the category label carried by the unpushed subtask.
[0088] B2. Based on the category label carried by the unpushed subtask, determine whether the unpushed subtask belongs to a post-task in the serial dependency relationship or does not belong to a post-task in the serial dependency relationship.
[0089] The category label carried by the unpushed subtask is used to indicate whether the unpushed subtask belongs to the post-task in the serial dependency relationship.
[0090] Exemplarily, when splitting the risk control task, it can be determined whether each subtask belongs to a post-task in the serial dependency relationship, and when subtask i belongs to a post-task in the serial dependency relationship, a first label is added to subtask i, and when task i does not belong to a post-task in the serial dependency relationship, a second label is added to subtask i; in step B2, when it is detected that the category label carried by the unpushed subtask is the first label, it is determined that the unpushed subtask belongs to a post-task in the serial dependency relationship. When it is detected that the category label carried by the unpushed subtask is the second label, it is determined that the unpushed subtask does not belong to a post-task in the serial dependency relationship.
[0091] 3033A. If the unpushed subtask is a post-task in a serial dependency relationship, after the target pre-task of the unpushed subtask is distributed, the execution status of the target pre-task is detected.
[0092] The target predecessor subtask refers to the predecessor task in the serial dependency relationship of the unpushed subtask.
[0093] There are many ways to detect the execution status of the target predecessor subtask in step 3033A, illustratively including:
[0094] In some embodiments, the execution status of the scheduled task is maintained in a database. In step 3033A, if the unpushed subtask belongs to a subsequent task in a serial dependency relationship, then after the target predecessor subtask of the unpushed subtask is distributed, the task status of the target predecessor subtask is queried from the database.
[0095] In some embodiments, reference Figure 6In order to avoid the huge database pressure caused by the system frequently checking the database and modifying the task status of the table, and to improve the real-time update of the task status, a state machine is used to report the execution status of the scheduled task to the scheduling server in real time. In step 3033A, if the unpushed subtask belongs to a post-task in the serial dependency relationship, then after the target predecessor subtask of the unpushed subtask is distributed, after the target predecessor subtask of the unpushed subtask is distributed, the execution status of the target predecessor subtask is obtained through the state machine. Specifically, after the target predecessor subtask of the unpushed subtask is distributed to the executor, the state machine can monitor the execution status of the corresponding executor on the target predecessor subtask in real time, and when the execution of the target predecessor subtask is completed, the state machine will feedback the execution status of the target predecessor subtask to the scheduling server in real time, so that the scheduling server can obtain the task status of the target predecessor subtask without repeatedly querying the database, thereby avoiding the huge database pressure caused by the system frequently checking the database and modifying the task status of the table, improving the real-time update of the task status, and thereby increasing the speed at which the unpushed subtask is added to the message queue, thereby improving the scheduling real-time of the scheduling task; in addition, when the execution of the target predecessor subtask is completed, the state machine can also update the execution status of the target predecessor subtask in the database.
[0096] 3034A When the target predecessor subtask is executed, the unpushed subtask is pushed to the message queue of the scheduling server as a target subtask that meets the preset push conditions.
[0097] For example, refer to Figure 4 , taking the unpushed subtask as subtask F as an example, the implementation of steps 3031A to 3034A is as follows: In steps 3031A to 3032A, the unpushed subtask (subtask F) is obtained from the subtask set, and a serial dependency check is performed on the unpushed subtask (subtask F), specifically to check whether the unpushed subtask (subtask A) has a serial dependency. If the unpushed subtask (subtask F) has a serial dependency (such as Figure 4 As shown in the figure, subtask F has a serial dependency relationship with subtask B, and subtask F has a serial dependency relationship with subtask H), then further detect whether the unpushed subtask (subtask F) is a predecessor task in the serial dependency relationship in which the unpushed subtask (subtask F) is located. If the unpushed subtask (subtask F) is a successor task in at least one serial dependency relationship (such as Figure 4As shown, in the serial dependency relationship between subtask F and subtask B, subtask F is a post-task), then it is determined that the unpushed subtask (subtask F) belongs to the post-task in the serial dependency relationship. In step 3033A, after the target predecessor subtask (i.e., subtask B) of the unpushed subtask (subtask F) is distributed to the executor, the state machine can monitor the execution status of the corresponding executor on the target predecessor subtask (i.e., subtask B) in real time, and when the target predecessor subtask (i.e., subtask B) is executed, the state machine will feed back the execution status of the target predecessor subtask (i.e., subtask B) to the scheduling server in real time, so that the scheduling server can obtain the task status of the target predecessor subtask without repeatedly querying the database. In step 3034A, when the target predecessor subtask (i.e., subtask B) is executed, the unpushed subtask (subtask F) is pushed to the message queue of the scheduling server as the target subtask that meets the preset push conditions.
[0098] Therefore, through steps 3031A to 3034A, when the unpushed subtask belongs to the post-task in the serial dependency relationship, the execution status of the target predecessor subtask is detected, and when the execution of the target predecessor subtask is completed, the unpushed subtask is pushed into the message queue as the target subtask that meets the preset push conditions. When two scheduled tasks need to be executed asynchronously, the subtask that needs to be executed later can be blocked (that is, when the unpushed subtask belongs to the post-task in the serial dependency relationship, the unpushed subtask is blocked), so as to realize the blocking of asynchronous tasks outside the message queue, avoid blocking scheduling when the asynchronous tasks enter the message queue and are distributed, thereby avoiding the problem that the message queue needs to wait for the first task to complete and return the result before distributing the next task, and further avoid the problem that two tasks that could have been executed synchronously are executed asynchronously due to blocking requirements, so that each scheduled task in the message queue can be executed in parallel, thereby improving the throughput of the scheduling system per unit time, and improving the throughput of the scheduling system while satisfying the blocking scheduling of asynchronous tasks.
[0099] (2) In some embodiments, the unpushed subtask does not belong to the post-task in the serial dependency relationship. In this case, step 303 may specifically include the following steps 3031B to 3033B:
[0100] 3031B. Obtain unpushed subtasks from the subtask set.
[0101] 3032B. Detect whether the unpushed subtask belongs to a post-task in the serial dependency relationship.
[0102] The implementation method of step 3032B "detecting whether the unpushed subtask does not belong to the subsequent task in the serial dependency relationship" is similar to that of step 2031A. For details, please refer to the relevant description above and will not be repeated here.
[0103] 3033B. If the unpushed subtask does not belong to a post-task in the serial dependency relationship, the unpushed subtask is pushed to the message queue of the scheduling server as a target subtask that meets the preset push condition.
[0104] For example, refer to Figure 4 , taking the unpushed subtask as subtask A as an example, the implementation of steps 3031B to 3033B is as follows: In steps 3031B to 3032B, the unpushed subtask (subtask A) is obtained from the subtask set, and a serial dependency check is performed on the unpushed subtask (subtask A), specifically to check whether the unpushed subtask (subtask A) has a serial dependency. If the unpushed subtask (subtask A) has a serial dependency (such as Figure 4 As shown in the figure, subtask A has a serial dependency relationship with subtask D, and subtask A has a serial dependency relationship with subtask E), then further check whether the unpushed subtask (subtask A) is all a predecessor task in the serial dependency relationship where the unpushed subtask (subtask A) is located. If the unpushed subtask (subtask A) has a predecessor task in all the serial dependency relationships (such as Figure 4 As shown, in the serial dependency relationship between subtask A and subtask D, subtask A is a predecessor task, and in the serial dependency relationship between subtask A and subtask E, subtask A is also a predecessor task), then it is determined that the unpushed subtask (subtask A) does not belong to the successor task in the serial dependency relationship. In step 3033B, the unpushed subtask (subtask A) is directly pushed to the message queue of the scheduling server as a target subtask that meets the preset push conditions.
[0105] Therefore, through steps 3031B to 3033B, when the unpushed subtask does not belong to the post-task in the serial dependency relationship, the unpushed subtask is directly pushed to the message queue of the scheduling server as a target subtask that meets the preset push conditions. Non-asynchronous tasks can be added to the message queue for execution as early as possible, so that two tasks that could have been executed synchronously can be executed synchronously normally, thereby improving the throughput of the scheduling system per unit time.
[0106] 304. Distribute the target subtask in the message queue to a target task executor to call the target task executor to execute the target subtask.
[0107] In some embodiments, when the target subtask is at the head of the message queue, the target executor of the target subtask is determined from the multiple task executors; the target subtask in the message queue is distributed to the target task executor to call the target task executor to execute the target subtask.
[0108] In some embodiments, in order to reduce the task maintenance business data cost, the business parameters of the risk control task can be passed in the scheduling chain, and the scheduling server passes the business parameters of the risk control task to each subtask. At this time, the task scheduling control method can also include obtaining the business parameters of the risk control task; in step 304, when the target subtask is at the head of the message queue, the target executor of the target subtask is determined from the multiple task executors; the target subtask and the business parameters in the message queue are distributed to the target task executor to call the target task executor to execute the target subtask according to the business parameters. Exemplarily, the business parameters can be shown in Table 1 below:
[0109] Table 1
[0110]
[0111]
[0112] From the above content, it can be seen that, on the one hand, by adding the target subtasks that meet the preset push conditions in the subtask set to the message queue of the scheduling server, the scheduling tasks that do not have serial dependencies can be executed in parallel, thereby improving the throughput of the scheduling system. On the other hand, by splitting the risk control task into multiple subtasks, long tasks can be split into different executor threads for processing, which reduces the impact of long tasks on the throughput of the scheduling system to a certain extent. On the third hand, by adding the target subtasks that meet the preset push conditions to the message queue of the scheduling server when the subtask meets the preset push conditions, asynchronous tasks can be blocked outside the message queue to avoid blocking scheduling when asynchronous tasks enter the message queue and are distributed, thereby avoiding the problem that the message queue needs to wait for the first task to complete and return the result before distributing the next task, thereby avoiding the problem that two tasks that could have been executed synchronously are executed asynchronously due to blocking requirements, so that each scheduling task in the message queue can be executed in parallel, improving the throughput of the scheduling system per unit time, and improving the throughput of the scheduling system under the condition of meeting the blocking scheduling of asynchronous tasks, thereby improving the real-time performance of task scheduling. In this way, the task execution efficiency of the scheduling system can be improved, and the problem of tasks being unable to be executed in parallel due to blocking in the queue can be avoided, thus meeting the real-time requirements of scene scheduling to a certain extent.
[0113] In addition, in order to better implement the task scheduling control method in the embodiment of the present application, based on the task scheduling control method, the embodiment of the present application also provides a task scheduling control device, such as Figure 7 FIG. 1 is a schematic diagram of a structure of a task scheduling control device according to an embodiment of the present application. The task scheduling control device 700 includes:
[0114] The receiving unit 701 is configured to receive a scheduling request for the risk control task initiated by the service interface layer when the service interface layer receives the risk control task of the business platform;
[0115] A response unit 702 is used to split the risk control task in response to the scheduling request to obtain a subtask set of the risk control task;
[0116] A push unit 703, configured to add a target subtask in the subtask set that meets a preset push condition to a message queue of the scheduling server;
[0117] The distribution unit 704 is used to distribute the target subtask in the message queue to the target task executor to call the target task executor to execute the target subtask.
[0118] In some embodiments, the push unit 703 is specifically used for:
[0119] Obtaining unpushed subtasks from the subtask collection;
[0120] Detect whether the unpushed subtask belongs to a post-task in the serial dependency relationship;
[0121] If the unpushed subtask belongs to a post-task in a serial dependency relationship, after the target pre-task of the unpushed subtask is distributed, detecting the execution status of the target pre-task;
[0122] When the target preceding subtask is executed, the unpushed subtask is pushed to the message queue of the scheduling server as a target subtask that meets the preset pushing conditions.
[0123] In some embodiments, the task scheduling system further includes a state machine, and the push unit 703 is specifically used to:
[0124] After the target predecessor subtask of the unpushed subtask is distributed, the execution status of the target predecessor subtask is obtained through the state machine.
[0125] In some embodiments, the push unit 703 is specifically used for:
[0126] According to the business process strategy of the risk control task, a series dependency detection is performed on the subtasks that have not been pushed;
[0127] If the unpushed subtask has a target serial dependency, and the unpushed subtask is a post-task in the target serial dependency, then determining that the unpushed subtask belongs to a post-task in the serial dependency;
[0128] Alternatively, if the unpushed subtask does not have a target serial dependency, it is determined that the unpushed subtask does not belong to a post-task in the serial dependency;
[0129] Alternatively, if the unpushed subtask has a target serial dependency, and the unpushed subtasks are all predecessor tasks in the target serial dependency, it is determined that the unpushed subtask does not belong to a successor task in the serial dependency.
[0130] In some embodiments, the push unit 703 is specifically used for:
[0131] Obtaining unpushed subtasks from the subtask collection;
[0132] Detect whether the unpushed subtask belongs to a post-task in the serial dependency relationship;
[0133] If the unpushed subtask does not belong to a post-task in the serial dependency relationship, the unpushed subtask is pushed to the message queue of the scheduling server as a target subtask that meets the preset push conditions.
[0134] In some embodiments, the response unit 702 is specifically used to:
[0135] In response to the scheduling request, obtaining the scenario type and scenario data carried by the risk control task;
[0136] Obtaining each target evaluation task associated with the scenario type from preset evaluation tasks;
[0137] Generating task data for each target assessment task according to the scenario data;
[0138] The task data of each target assessment task is taken as a subtask to obtain a subtask set of the risk control task.
[0139] In some embodiments, the receiving unit 701 is specifically used for:
[0140] Obtaining business parameters of the risk control task;
[0141] In some embodiments, the distribution unit 704 is specifically used to:
[0142] When the target subtask is at the head of the message queue, determining a target executor of the target subtask from the multiple task executors;
[0143] The target subtask and the business parameters in the message queue are distributed to the target task executor, so as to call the target task executor to execute the target subtask according to the business parameters.
[0144] In specific implementation, the above units can be implemented as independent entities, or can be arbitrarily combined and implemented as the same or several entities. The specific implementation of the above units can refer to the previous task scheduling control method embodiment, which will not be repeated here.
[0145] A person skilled in the art will appreciate that all or part of the steps in the above-mentioned task scheduling control method may be completed by instructions, or by controlling related hardware through instructions. The instructions may be stored in a computer-readable storage medium and loaded and executed by a processor.
[0146] To this end, an embodiment of the present application provides a computer-readable storage medium, in which multiple computer programs are stored. The computer programs can be loaded by a processor to execute any task scheduling control method provided by the embodiment of the present application.
[0147] For example, in some embodiments, the computer program can be loaded by a processor to execute the following steps in any of the task scheduling control methods provided in the embodiments of the present application:
[0148] When the service interface layer receives a risk control task from the business platform, it receives a scheduling request for the risk control task initiated by the service interface layer; in response to the scheduling request, it splits the risk control task to obtain a subtask set of the risk control task; adds the target subtask in the subtask set that meets the preset push conditions to the message queue of the scheduling server; distributes the target subtask in the message queue to the target task executor, so as to call the target task executor to execute the target subtask.
[0149] The computer-readable storage medium may include: a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.
[0150] In the above-mentioned task scheduling control device and computer-readable storage medium embodiments, the description of each embodiment has its own emphasis. For parts not described in detail in a certain embodiment, reference can be made to the relevant description of other embodiments. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working process and beneficial effects of the task scheduling control device, computer-readable storage medium and its corresponding units described above can refer to the description of the task scheduling control method in the above embodiment, and will not be repeated here.
[0151] The above describes in detail a task scheduling control method, device, system and computer-readable storage medium provided in the embodiments of the present application. Specific examples are used herein to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method and its core idea of the present application. At the same time, for those skilled in the art, according to the idea of the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application. The non-company software tools or components that appear in the embodiments of the present application are only examples and do not represent actual use.
Claims
1. A task scheduling control method, It is characterized in that A scheduling server applied to a task scheduling system, wherein the task scheduling system includes a service interface layer, a scheduling server and a plurality of task executors, and the method includes: When the service interface layer receives the risk control task from the business platform, it receives a scheduling request for the risk control task initiated by the service interface layer; In response to the scheduling request, split the risk control task to obtain a subtask set of the risk control task; Adding the target subtask that meets the preset push condition in the subtask set to the message queue of the scheduling server; The target subtask in the message queue is distributed to a target task executor, so as to call the target task executor to execute the target subtask.
2. The task scheduling control method according to claim 1, It is characterized in that The step of adding the target subtask in the subtask set that meets the preset push condition to the message queue of the scheduling server includes: Obtaining unpushed subtasks from the subtask collection; Detect whether the unpushed subtask belongs to a post-task in the serial dependency relationship; If the unpushed subtask belongs to a post-task in a serial dependency relationship, after the target pre-task of the unpushed subtask is distributed, detecting the execution status of the target pre-task; When the target preceding subtask is executed, the unpushed subtask is pushed to the message queue of the scheduling server as a target subtask that meets the preset pushing conditions.
3. The task scheduling control method according to claim 2, It is characterized in that The task scheduling system further includes a state machine, which detects the execution state of the target predecessor subtask after the target predecessor subtask of the unpushed subtask is distributed, including: After the target predecessor subtask of the unpushed subtask is distributed, the execution status of the target predecessor subtask is obtained through the state machine.
4. The task scheduling control method according to claim 2, It is characterized in that The detecting whether the unpushed subtask belongs to a post-task in a serial dependency relationship includes: According to the business process strategy of the risk control task, a series dependency detection is performed on the subtasks that have not been pushed; If the unpushed subtask has a target serial dependency, and the unpushed subtask is a post-task in the target serial dependency, then determining that the unpushed subtask belongs to a post-task in the serial dependency; Alternatively, if the unpushed subtask does not have a target serial dependency, it is determined that the unpushed subtask does not belong to a post-task in the serial dependency; Alternatively, if the unpushed subtask has a target serial dependency, and the unpushed subtasks are all predecessor tasks in the target serial dependency, it is determined that the unpushed subtask does not belong to a successor task in the serial dependency.
5. The task scheduling control method according to claim 1, It is characterized in that The step of adding the target subtask in the subtask set that meets the preset push condition to the message queue of the scheduling server includes: Obtaining unpushed subtasks from the subtask collection; Detect whether the unpushed subtask belongs to a post-task in the serial dependency relationship; If the unpushed subtask does not belong to a post-task in the serial dependency relationship, the unpushed subtask is pushed to the message queue of the scheduling server as a target subtask that meets the preset push conditions.
6. The task scheduling control method according to claim 1, It is characterized in that The step of splitting the risk control task in response to the scheduling request to obtain a subtask set of the risk control task includes: In response to the scheduling request, obtaining the scenario type and scenario data carried by the risk control task; Obtaining each target evaluation task associated with the scenario type from preset evaluation tasks; Generating task data for each target assessment task according to the scenario data; The task data of each target assessment task is taken as a subtask to obtain a subtask set of the risk control task.
7. The task scheduling control method according to claim 1, It is characterized in that The method further comprises: Obtaining business parameters of the risk control task; The step of distributing the target subtask in the message queue to a target task executor to call the target task executor to execute the target subtask includes: When the target subtask is at the head of the message queue, determining a target executor of the target subtask from the multiple task executors; The target subtask and the business parameters in the message queue are distributed to the target task executor, so as to call the target task executor to execute the target subtask according to the business parameters.
8. A task scheduling control device, It is characterized in that The task scheduling control device comprises: A receiving unit, configured to receive a scheduling request for the risk control task initiated by the service interface layer when the service interface layer receives the risk control task of the business platform; A response unit, configured to split the risk control task in response to the scheduling request to obtain a subtask set of the risk control task; A push unit, used to add a target subtask in the subtask set that meets a preset push condition to a message queue of the scheduling server; The distribution unit is used to distribute the target subtask in the message queue to the target task executor to call the target task executor to execute the target subtask.
9. A task scheduling system, It is characterized in that The task scheduling system includes a service interface layer, a state machine, a scheduling server and a plurality of task executors, wherein the scheduling server includes a memory and a processor; The memory is used to store computer programs; The processor is used to execute the computer program and implement the task scheduling control method according to any one of claims 1 to 7 when executing the computer program.
10. A computer-readable storage medium, It is characterized in that A computer program is stored thereon, and the computer program is loaded by a processor to execute the task scheduling control method according to any one of claims 1 to 7.