A task scheduling method, device, server and storage medium
Through the task operation interface detection of trigger instructions, shard processing and dynamic scheduling, the problem of poor task scheduling flexibility in the existing technology is solved, and the flexibility and reliability of task scheduling are improved, ensuring the efficiency and reliability of task processing.
Patent Information
- Application Number
- CN202111565419.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-12-20
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2041-12-20
AI Technical Summary
The prior art has poor flexibility in task scheduling, it is difficult to effectively handle large batches or long logical tasks, and lacks a flexible scheduling and transfer mechanism.
Provide a task scheduling method, detects trigger instructions through the task operation interface, determines the target task collection and slices the processing, pushes the sub-task collection to the device cluster according to preset rules, monitors the execution status, and dynamically adjusts the execution device in abnormal situations.
It improves the flexibility and reliability of task scheduling, ensures the efficiency and reliability of task processing, and can perform scheduling and transfer in abnormal situations to adapt to different task needs.
Smart Images

Figure CN114237852B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of data processing, and in particular to a task scheduling method, device, server and storage medium. Background Art
[0002] With the rapid development of computer technology, the amount of business data is also increasing rapidly. For the processing of large amounts of data, distributed task processing can usually be used. For tasks that do not require real-time processing, or have a large number of tasks, or have a long processing logic time, asynchronous scheduling can be used for task processing. However, the current asynchronous scheduling scheme for implementing tasks in the industry usually uses a timed start scheduling unit to schedule tasks, and the flexibility of task scheduling is poor. Summary of the invention
[0003] The embodiments of the present application provide a task scheduling method, device, server and storage medium, which can provide a task operation interface for users to trigger task scheduling, so as to trigger the execution of task scheduling under the trigger instruction issued by the task operation interface, thereby improving the flexibility of triggering task scheduling.
[0004] In a first aspect, an embodiment of the present application discloses a task scheduling method, the method comprising:
[0005] When a trigger instruction associated with a trigger task is detected on the task operation interface, a target task set is determined based on the trigger instruction, the target task set includes a plurality of target tasks;
[0006] Slice each target task in the target task set to obtain multiple subtask sets, each subtask set including one or more target tasks;
[0007] Pushing each subtask set to multiple execution devices included in the device cluster according to a preset push rule, so that each execution device executes the target task in the received subtask set, and determining the respective task status according to the execution status of each execution device;
[0008] The task status of each execution device is obtained. When an abnormal execution device is detected in the device cluster, a target execution device is selected from the device cluster according to the task status, and a subtask set corresponding to the abnormal execution device is pushed to the target execution device.
[0009] In a second aspect, an embodiment of the present application discloses a task scheduling device, the device comprising:
[0010] a detection unit, configured to, when a trigger instruction associated with a trigger task is detected on the task operation interface, determine a target task set based on the trigger instruction, wherein the target task set includes a plurality of target tasks;
[0011] A determination unit, configured to perform slicing processing on each target task in the target task set to obtain a plurality of subtask sets, each of which includes one or more target tasks;
[0012] A push unit, used to push each subtask set to multiple execution devices included in the device cluster according to a preset push rule, so that each execution device executes the target task in the received subtask set, and determines the task status of each execution device according to the execution status of each execution device;
[0013] A selection unit, configured to obtain the task status of each execution device, and when an abnormal execution device is detected in the device cluster, select a target execution device from the device cluster according to the task status;
[0014] The pushing unit is further used to push the subtask set corresponding to the abnormal execution device to the target execution device.
[0015] In a third aspect, an embodiment of the present application discloses a server, comprising a processor and a memory, wherein the memory is used to store a computer program, the computer program comprises program instructions, and the processor is configured to call the program instructions to execute the method of the first aspect above.
[0016] In a fourth aspect, an embodiment of the present application discloses a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a processor, the processor executes the method of the first aspect.
[0017] In an embodiment of the present application, when a trigger instruction associated with a trigger task is detected on a task operation interface, the server can determine a target task set including multiple target tasks based on the trigger instruction, and perform slicing processing on each target task in the target task set to obtain multiple subtask sets. Then, each subtask set is pushed to multiple execution devices included in the device cluster according to a preset push rule, so that each execution device executes the target task in the received subtask set, and determines the respective task status according to the execution status of each execution device. The server can also obtain the task status of each execution device, and when an abnormal execution device with an abnormality is detected in the device cluster, the target execution device can be selected from the device cluster according to the task status, and the subtask set corresponding to the abnormal execution device can be pushed to the target execution device. By implementing the above method, a task operation interface for users to trigger task scheduling can be provided, so that the execution of task scheduling can be triggered by the trigger instruction issued by the task operation interface, thereby improving the flexibility of triggering task scheduling; after the task slice processing is completed, each slice can be packaged into a specific scheduling unit, and then distributed to specific execution devices, so that each execution device can execute the corresponding task; at the same time, when each task execution device executes the corresponding task, it can also send a task status indicating the task execution status to the server, so that the server can monitor the task status of each task execution device, and can realize the scheduling transfer after the task execution is abnormal according to the task status, thereby improving the reliability and efficiency of task processing. BRIEF DESCRIPTION OF THE DRAWINGS
[0018] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
[0019] Figure 1 It is a flowchart of a task scheduling system provided by an embodiment of the present application;
[0020] Figure 2 It is a flowchart of a task scheduling method provided in an embodiment of the present application;
[0021] Figure 3 This is a schematic diagram of a structure for determining a target task set provided by an embodiment of the present application;
[0022] Figure 4 It is a flowchart of a task scheduling method provided in an embodiment of the present application;
[0023] Figure 5It is a structural diagram of a task scheduling device provided in an embodiment of the present application;
[0024] Figure 6 It is a structural diagram of a server provided in an embodiment of the present application. DETAILED DESCRIPTION
[0025] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0026] See also Figure 1 , Figure 1 1 is a schematic diagram of the architecture of a task scheduling system provided in an embodiment of the present application. The present application involves a terminal 101, a server 102 and a device cluster, wherein the device cluster includes a plurality of execution devices 103.
[0027] Taking the terminal 101 as an example, the terminal 101 can receive a trigger instruction associated with a trigger task initiated by a user on the task operation interface, and send the trigger instruction to the server 102. The server 102 can determine a target task set including multiple target tasks based on the trigger instruction, and perform slicing processing on each target task in the target task set to obtain multiple subtask sets; the server 102 can also push each subtask set to multiple execution devices 103 included in the device cluster, so that each execution device 103 executes the target task in the received subtask set, and determines the task status of each execution device according to the execution status of each execution device.
[0028] Subsequently, the server 102 may obtain the task status of each execution device 103. When an abnormal execution device is detected in the device cluster, a target execution device may be selected from the device cluster according to the task status, and the subtask set corresponding to the abnormal execution device may be pushed to the target execution device.
[0029] Figure 1The terminal 101 shown can be a device such as a smart phone, a tablet computer, a laptop computer, a desktop computer, or a peripheral device such as a handle or a touch screen; the server 102 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (Content Delivery Network, CDN), and basic cloud computing services such as big data and artificial intelligence platforms, etc. The execution device 103 can be a terminal or a server. The terminal 101 and the server 102 can be directly or indirectly connected via wired or wireless communication, and the server 102 and the execution device 103 can also be directly or indirectly connected via wired or wireless communication, and this application does not limit this.
[0030] The implementation details of the technical solution of the embodiment of the present application are described in detail below:
[0031] See also Figure 2 , Figure 2 : is a flowchart of a task scheduling method provided in an embodiment of the present application. The task scheduling method can be executed by the server mentioned above; the task scheduling method includes the following steps:
[0032] S201: When a trigger instruction associated with a trigger task is detected on a task operation interface, a target task set is determined based on the trigger instruction.
[0033] In one implementation, a task operation interface can be output on the terminal screen used by the user, and the user can perform relevant operations on the task operation interface to realize the processing of the task. Among them, the target task in this application can be a task in the task library, that is, a task uploaded in advance; it can also be a task uploaded in real time, that is, a task that needs to be processed when the task scheduling method is executed. The target task can be various types of tasks, such as loan collection tasks, insurance sales tasks, information entry tasks, etc. The tasks to be processed can be tasks of the same type or different types, which is not limited in this application.
[0034] In one implementation, when a trigger instruction associated with a trigger task is detected on a task operation interface, a target task set including multiple target tasks can be determined based on the trigger instruction, so that operations such as slicing and scheduling of multiple target tasks can be implemented later. Optionally, the trigger instruction can be used to trigger the execution of a task scheduling method, and the trigger instruction can also carry task trigger information, so that the corresponding target task set can be determined based on the carried task trigger information. The task trigger information can include task trigger attributes, and the task trigger attributes can be obtained based on relevant operations of the user on the task operation interface.
[0035] For example, see Figure 3 As shown, the task operation interface may include a task trigger setting area 31 and a confirmation control 32, wherein the task trigger setting area 31 may include one or more task trigger setting items, and the task trigger setting items may be used for the user to input information related to the task attributes of the task. For example, the task trigger attribute setting item may include a task type setting item, a task time setting item, a task area setting item, etc. The user may enter the corresponding information in the one or more task trigger setting items, such as entering loan collection in the task type setting item, entering 2020.1.1-2020.12.30 in the task time setting item, and then triggering the confirmation control 32 (such as clicking or pressing). When the terminal detects that the confirmation control 32 is triggered, the terminal may generate a trigger instruction, and may obtain the user's input information in the task trigger setting item, and carry the input information in the trigger instruction, which is the task trigger information described above, and the input information corresponding to a task trigger setting item is a task trigger attribute. Then the terminal may send the trigger instruction carrying the task trigger information to the server.
[0036] Then, after the server obtains the trigger instruction, the task trigger attribute carried in the trigger instruction can be matched with the task attribute of each pending task in the pending task set. Among them, each pending task in the pending task set can be understood as a task in the task library described above. For any pending task in the pending task set, if the task trigger attribute is included in the task attribute of any pending task, then any pending task can be determined as a target task.
[0037] For example, assuming that there are three task trigger attributes, if the three task trigger attributes are all present in all the task attributes of a pending task, the pending task can be determined as the target task. Assuming that the user enters loan collection in the task type setting item and enters 2020.1.1-2020.12.30 in the task time setting item, that is, the task trigger attributes include loan collection (task type) and 2020.1.1-2020.12.30 (task time), then the pending tasks with the task type of loan collection and the task time between 2020.1.1-2020.12.30 can be filtered out from the pending task set, and the filtered pending tasks are the target tasks.
[0038] S202: Slice each target task in the target task set to obtain multiple subtask sets.
[0039] In one implementation, the task attributes of each target task in the target task set can be obtained, so as to perform sharding processing on each target task in the target task set based on the task attributes to obtain multiple subtask sets. Each subtask set can include one or more target tasks; the task attributes can include task type, task time, task data volume, task processing area, etc.
[0040] Optionally, assuming that the task attributes include the task processing area, sharding can be performed based on the task processing area, such as dividing the target tasks in the same task processing area into one shard (a shard can be understood as a set of subtasks), where the same task processing area can refer to the same city, or the same street, or the same community, etc., and its specific definition can be determined based on actual conditions. For example, for loan collection tasks, assuming that the task processing areas corresponding to the collection tasks are scattered in various cities, the collection tasks in the same city can be divided into one shard. Assuming that the task processing areas corresponding to the collection tasks are in the same city, but scattered in different streets, the collection tasks on the same street can be divided into one shard.
[0041] Optionally, assuming that the task attributes include task time, slicing can be performed based on task time, such as dividing the target tasks in the same time period into one slicing. In a specific implementation, multiple time periods can be predefined, and the corresponding target tasks in each time period can be divided into one slicing, that is, the time period in which the task time of each target task is located can be determined, so as to determine the corresponding slicing according to the time period. In a specific implementation, the task time of each target task can be obtained, and then the latest task time can be determined from all the task times, and then multiple time periods can be divided based on the current time (or the earliest task time) and the latest task time. For example, for a collection task of a loan collection class, the task time can be the collection deadline, where the latest collection deadline is 2020.10.10, and the current time is 2020.2.2, then multiple time periods can be divided based on 2020.2.2 and 2020.10.10. Further, the time period corresponding to each target task is determined to divide multiple target tasks into multiple subtask sets.
[0042] Optionally, assuming that the task attributes include the amount of task data, sharding may be performed based on the amount of task data. For example, even distribution may be performed based on the amount of task data, that is, ensuring that the amount of task data corresponding to each shard is the same.
[0043] Optionally, the slicing process can be performed based on one task attribute, or based on multiple task attributes, which is not limited in this application. For example, multiple target tasks can be slicingly processed based on the task type and the task processing area. In a specific implementation, the task type and the task processing area of each target task can be determined first, and then the multiple target tasks can be slicingly processed according to the task type to obtain a first subtask set, wherein the task type of the target tasks in any first subtask set is the same. Then, each first subtask set is slicingly processed again based on the task processing area to obtain multiple subtask sets.
[0044] S203: Pushing each subtask set to multiple execution devices included in the device cluster according to preset push rules, so that each execution device executes the target task in the received subtask set, and determines the task status of each execution device according to the execution status of each execution device.
[0045] In one implementation, the target tasks included in each subtask set can be pushed to the corresponding execution device so that the execution device can execute the corresponding target tasks. For example, the execution device can perform data processing on the data to be processed indicated by the target task according to the data processing method indicated by the target task. Assuming that the data to be processed indicated by the target task is a one million car insurance policy, and the processing method is to record the user information of the one million car insurance policy, then after the execution device obtains the target task, it can perform data processing on the data to be processed indicated in the target task, that is, the user information in the one million car insurance policy, that is, record the user information of the one million car insurance policy.
[0046] In one implementation, after the target task set is divided into multiple subtask sets, each subtask set can be pushed to the execution device according to the preset push rules, so that the execution device can execute the corresponding target task. The preset push rules can be specifically described as follows. For example, different situations can be divided based on the number of execution devices and the number of subtask sets in the device cluster to push tasks. Assume that the number of execution devices in the device cluster is M, and the number of subtask sets is N. When the size relationship between M and N is uncertain, the corresponding push method may be different.
[0047] When N (the number of subtask sets) is less than or equal to M (the number of devices), in order to quickly complete the task, each subtask set can be pushed to an execution device, that is, to ensure that one execution device executes all target tasks in a subtask set. Then, an execution device can be selected from the device cluster for each subtask set, and each subtask set can be pushed to the corresponding execution device, that is, the execution device corresponding to each subtask set is different.
[0048] In one implementation, when selecting an execution device for each subtask set, the push can be performed based on the total task data volume of the subtask set and the idle load volume of the execution device. In the process of pushing, the idle load volume of each execution device can be ensured to be greater than or equal to the total task data volume of the corresponding subtask set as much as possible to ensure the execution efficiency of the execution device. In a specific implementation, the total task data volume of each subtask set and the idle load volume of each execution device can be determined first. Then, the subtask sets are sorted in order from large to small according to the total task data volume to obtain a first sorting result for the subtask sets, and the execution devices are sorted in order from large to small according to the idle load volume to obtain a second sorting result for the execution devices. Then, the subtask sets are pushed according to the first sorting result of the subtask set and the second sorting result of the execution devices.
[0049] Assuming that the number of subtask sets is equal to the number of execution devices, the subtask sets and execution devices can be matched one-to-one according to the order corresponding to the first sorting result of the subtask sets and the order corresponding to the second sorting result of the execution devices, that is, the subtask set ranked first in the first sorting result can be pushed to the execution device ranked first in the second sorting result, and the subtask set ranked second in the first sorting result can be pushed to the execution device ranked second in the second sorting result, and so on. Then, an execution device can be selected for each subtask set. In the case where the number of subtask sets is less than the number of execution devices, the push can also be performed in the above manner. The subsequent push of each subtask set in the M subtask sets to an execution device in the device cluster can also be performed in the above manner.
[0050] When N (the number of subtask sets) is greater than M (the number of devices), M subtask sets can be selected from multiple subtask sets, and each of the M subtask sets can be pushed to an execution device in the device cluster. Then, when the task status of the execution device is obtained to be the target task status, one of the NM subtask sets is pushed to the execution device whose task status is the target task status.
[0051] In one implementation, when selecting M subtask sets from multiple subtask sets, the selection may be made in combination with relevant features, for example, the selection may be made based on features such as the urgency of the subtask set, the total task data volume of the subtask set, and the idle load volume of the execution device. Among them, for a subtask set, the total task data volume of the subtask set may refer to the sum of the task data volumes corresponding to the various target tasks included in the subtask set.
[0052] Optionally, the urgency of a subtask set is used as an example for explanation, and the urgency can be determined based on the task time. In a specific implementation, assuming that the task attributes corresponding to each target task include the task time, the reference task time corresponding to each subtask set can be obtained, wherein any reference task time can include: the earliest task time or the latest task time among the task times corresponding to all target tasks of any subtask set. After determining the reference task time corresponding to each subtask set, M subtask sets are determined from the N subtask sets based on the reference task time. For example, multiple subtask sets can be sorted in descending order according to the reference task time to obtain a task set sorting result. Furthermore, the subtask sets in the first M positions can be selected from the task set sorting result. The M subtask sets are the more urgent subtask sets among the current multiple subtask sets, and these subtask sets can be processed with priority.
[0053] In one implementation, before sending the target tasks in each subtask set to the corresponding execution device, it is also possible to detect whether there is an abnormal execution device in the device cluster. If an abnormal execution device is detected, the abnormal execution device can be filtered from the device cluster, that is, the abnormal execution device does not need to be used for data processing, wherein the determination of the abnormal execution device can refer to the relevant description in step S204.
[0054] In one implementation, when executing the target task in the corresponding subtask set, the execution device can determine the corresponding task status based on the execution status of its own task, and send the task status to the server. Optionally, when the execution device completes the execution of all target tasks in a subtask set, it can send the task status to the server, and the task status can be used to indicate that the task has been completed. Optionally, when the execution device executes all target tasks in a subtask set, it can periodically send the task status to the server, and the task status can include: indicating that the task is in progress or indicating that the task has been completed. Among them, the periodic sending of the task status can specifically be sending the task status according to a preset sending time interval. For example, the preset sending time interval can be 5 seconds, or 7 seconds, etc., and the specific value is not limited in this application.
[0055] S204: Acquire the task status of each execution device. When an abnormal execution device is detected in the device cluster, select a target execution device from the device cluster according to the task status, and push the subtask set corresponding to the abnormal execution device to the target execution device.
[0056] In one implementation, the server can receive the task status sent by each execution device to monitor the task execution status of each execution device through the task status, which is conducive to the subsequent scheduling transfer after the task execution is abnormal, and is also conducive to the task scheduling of the remaining subtask sets. The remaining subtask set may refer to the subtask set remaining after one task scheduling when the number of subtask sets is greater than the number of execution devices.
[0057] In one implementation, when the execution device executes the corresponding target task, the server can also monitor in real time whether there are abnormal execution devices in the device cluster to avoid these abnormal execution devices affecting the execution of the task due to the inability to process the target task normally. In the specific implementation, taking any execution device in the device cluster as an example, for any execution device in the device cluster, a signal detection request can be sent to the any execution device at a preset time interval to request the any execution device to return the target data. Among them, the preset time interval can be preset, such as the preset time interval can be 1 second, or 3 seconds. After the server sends the signal detection request, it can be further determined whether the target data sent by the any execution device can be received within the preset time period. The preset time period can also be preset, such as 2 seconds, or 4 seconds. The preset time period can be determined based on the historical response time period of the execution device. The response time period can be understood as: the difference between the time when the server sends the request and the time when the server receives the response data (the data obtained in response to the request). The historical response time period can be understood as the response time period involved before the current time. For example, multiple historical response time periods can be obtained, and the average of the multiple historical response time periods can be used as the preset time period; for example, any of the obtained historical response time periods can be used as the preset time period. If the server can receive the target data within the preset time period, it indicates that the execution device is normal; if the server does not receive the target data within the preset time period, it indicates that any of the execution devices is abnormal, that is, any of the execution devices can be determined as abnormal execution devices. In this application, an execution device that has an abnormality can be referred to as an abnormal execution device.
[0058] In one implementation, after determining that there is an abnormal execution device in the device cluster, it is necessary to push the subtask set corresponding to the abnormal execution device to other normal execution devices to ensure the normal execution of the target task. Then, a normal execution device can be selected from other execution devices first, and the execution device can be referred to as the target execution device. It should be noted that there may be one or more abnormal execution devices in the device cluster, and this application is mainly described by taking an abnormal execution device as an example. Among them, the specific implementation method of selecting a target execution device due to the existence of an abnormal execution device can be to select the target execution device from the device cluster based on the task status of each execution device. For example, an execution device with a task status of a target task state can be determined from the device cluster to determine the target execution device according to the execution device in the target task state, wherein the target task state can refer to a task state used to indicate that the task has been completed. If the number of execution devices in the target task state is one, the execution device can be determined as the target execution device, and if the number of execution devices in the target task state is multiple, an execution device can be randomly selected from the multiple execution devices to be determined as the target execution device.
[0059] In an embodiment of the present application, when a trigger instruction associated with a trigger task is detected on a task operation interface, the server can determine a target task set including multiple target tasks based on the trigger instruction, and perform slicing processing on each target task in the target task set to obtain multiple subtask sets. Then, each subtask set is pushed to multiple execution devices included in the device cluster according to a preset push rule, so that each execution device executes the target task in the received subtask set, and determines the respective task status according to the execution status of each execution device. The server can also obtain the task status of each execution device, and when an abnormal execution device with an abnormality is detected in the device cluster, the target execution device can be selected from the device cluster according to the task status, and the subtask set corresponding to the abnormal execution device can be pushed to the target execution device. By implementing the above method, a task operation interface can be provided for users to trigger task scheduling, so that the execution of task scheduling can be triggered by the trigger instruction issued by the task operation interface, so that flexible triggering of task scheduling can be achieved based on user needs; after the task slice processing is completed, each slice can be packaged into a specific scheduling unit and then distributed to specific execution devices, so that each execution device can execute the corresponding task; at the same time, when each task execution device executes the corresponding task, it can also send a task status indicating the task execution status to the server, so that the server can monitor the task status of each task execution device, and can realize the scheduling transfer after the task execution is abnormal according to the task status, thereby improving the reliability and efficiency of task processing.
[0060] See also Figure 4 , Figure 4 : is a flowchart of a task scheduling method provided in an embodiment of the present application. The task scheduling method can be executed by the server mentioned above. The task scheduling method includes the following steps:
[0061] S401: When a trigger instruction associated with a trigger task is detected on a task operation interface, a target task set is determined based on the trigger instruction.
[0062] In one implementation, a task operation interface can be output on the terminal screen used by the user, and the user can perform relevant operations on the task trigger interface so that the terminal detects the trigger instruction associated with the trigger task and sends the trigger instruction to the server. For example, the task operation interface may include a task trigger control. When the user performs a trigger operation (such as clicking or pressing) on the task trigger control, the terminal can detect the trigger instruction associated with the trigger task. Among them, the trigger instruction can be associated with a task that needs to be scheduled, and the task can be a pending task in the pending task set described above. Then, the server can determine the target task set based on the trigger instruction, and the target task in the target task set is the task that needs to be scheduled.
[0063] In one implementation, the task that needs to be scheduled and is associated with the trigger instruction can be predicted in advance. After the prediction is completed, a reminder message can be sent to the terminal used by the user. The reminder message can be used to instruct the user to perform relevant operations in the task trigger interface so that the currently predicted task can be scheduled. The predicted task can be set at a fixed time, such as automatically starting the predicted task at 9 o'clock every morning to determine the task that needs to be executed every day. The predicted task can also be triggered by the user.
[0064] In one implementation, the specific implementation of the prediction task can be: the server can obtain the task attributes of each task to be processed in the set of tasks to be processed, and call the pre-trained task recognition model to recognize the task attributes to obtain the recognition result of each task to be processed. The task attributes can refer to the above description; the recognition result can include whether the task to be processed needs to be scheduled; the task recognition model can be a machine learning model that can be used for recognition, for example, the task recognition model can be a decision tree (Decision Tree, DT) model, a support vector machine (Support Vector Machine, SVM) model, a random forest (Random Forest, RF) model, a logistic regression (Logistic Regression, LR) model, etc. Other machine learning models that can be used for recognition can also be included, and the specific model is not limited in the application. Optionally, before using the task recognition model for recognition processing, the task recognition model can be trained first, wherein the training sample set for training can include multiple training samples, each training sample can include the task attributes of a task and the training recognition result corresponding to the task, and the training recognition result can be manually annotated or annotated in other ways.
[0065] Then, after determining the recognition result of each pending task based on the task recognition model, it is possible to further determine the pending task whose recognition result is the target recognition result in the pending task set, and the target recognition result can be used to indicate that the pending task needs to be scheduled. Then, after determining the pending task whose recognition result is the target recognition result, a trigger instruction associated with the trigger task can be generated, and the trigger instruction can be associated with the pending task whose recognition result is the target recognition result. And after the trigger instruction is generated, it is also possible to further perform relevant operations on the task trigger interface based on the user so that the trigger instruction is triggered (or understood as being awakened). For example, it can be triggered by the task trigger control on the task trigger interface described above. Optionally, when the prediction task is not completed (i.e., the pending task whose recognition result is the target recognition result in the pending task set has not yet been determined), the task trigger control can be in a first state, and when the prediction task is completed (i.e., the pending task whose recognition result is the target recognition result in the pending task set has been determined), the task trigger control can be in a second state. The first state may be a non-triggerable state, i.e., the task trigger control cannot be triggered, such as the user cannot click the task trigger control; the second state may be a triggerable state, i.e., the task trigger control can be triggered, such as the user can click the task trigger control. Then, when the task trigger control is in the second state, the user can trigger the task trigger control, so that the server can receive the trigger instruction associated with the trigger task sent by the terminal, and further determine the target task set based on the trigger instruction.
[0066] Through the above method, it can be known that task trigger prediction can be performed in advance for multiple pending tasks in the pending task set, so as to determine the pending tasks that need to be scheduled, that is, the target tasks, from the pending task set, so as to realize intelligent task triggering. Compared with the above description that the user sets relevant information on the task operation interface, the user's related operations on the task operation interface can also be simplified. For example, the user only needs to trigger the task trigger control of the trigger task on the task operation interface, thereby improving the task processing speed and the user experience.
[0067] S402: Slice each target task in the target task set to obtain multiple subtask sets.
[0068] S403: Push each subtask set to multiple execution devices included in the device cluster according to preset push rules, so that each execution device executes the target task in the received subtask set, and determines the task status of each execution device according to the execution status of each execution device.
[0069] S404: Acquire the task status of each execution device. When an abnormal execution device is detected in the device cluster, determine an initial execution device whose task status is the target task status from the device cluster to obtain an initial execution device set.
[0070] S405: Obtain the idle load of each initial execution device respectively, determine the target execution device from the initial execution device set according to the idle load, and push the subtask set corresponding to the abnormal execution device to the target execution device.
[0071] In steps S404 and S405, when an abnormal execution device is detected in the device cluster, a target execution device can be selected from multiple execution devices based on characteristics of multiple dimensions. For example, the target execution device can be selected based on two dimensions: task status and idle load. The specific implementation method can be described as follows.
[0072] In one embodiment, an execution device whose task state is the target task state can be determined from the device cluster, and the execution device in the target task state can be called an initial execution device, so that an initial execution device set can be obtained, and the initial execution device set can include one or more initial execution devices. If the initial execution device set includes one initial execution device, the initial execution device can be determined as the target execution device. If the initial execution device set includes multiple initial execution devices, the following operation can be performed: the idle load of each initial execution device can be obtained respectively, so as to determine the target execution device from the initial execution device set according to the idle load. For example, the total task data volume of the subtask set corresponding to the abnormal execution device can be determined, and then, the candidate execution device whose idle load is greater than or equal to the total task data volume can be determined from the initial execution device set to obtain a candidate execution device set, and the candidate execution device set can include one or more candidate execution devices. Then, the target execution device can be selected from the candidate execution device set. Among them, if the candidate execution device set includes one candidate execution device, the candidate execution device can be determined as the target execution device. If the candidate execution device set includes multiple candidate execution devices, a candidate execution device can be randomly selected from the multiple candidate execution devices as the target execution device.
[0073] The specific implementation of steps S402-S403 can refer to the specific description of steps S202-S203 in the above embodiment, which will not be repeated here.
[0074] In an embodiment of the present application, task trigger prediction can be performed in advance for multiple pending tasks in a pending task set to determine the pending tasks that need to be scheduled, i.e., the target tasks, from the pending task set, so as to realize intelligent task triggering, simplify the user's related operations on the task operation interface, increase the task processing speed, and improve the user experience. When abnormal execution devices are detected in the device cluster, the target execution device can be selected based on the features of multiple dimensions, which can improve the rationality of the scheduling transfer after the abnormal task execution, thereby improving the reliability of task processing.
[0075] See also Figure 5 , Figure 5 1 is a schematic diagram of a task scheduling device provided in an embodiment of the present application. The task scheduling device comprises:
[0076] A detection unit 501 is configured to determine a target task set based on a trigger instruction associated with a trigger task when a trigger instruction is detected on a task operation interface, wherein the target task set includes a plurality of target tasks;
[0077] A determination unit 502 is used to perform slicing processing on each target task in the target task set to obtain multiple subtask sets, each subtask set including one or more target tasks;
[0078] The push unit 503 is used to push each subtask set to multiple execution devices included in the device cluster according to a preset push rule, so that each execution device executes the target task in the received subtask set, and determines the task status of each execution device according to the execution status of each execution device;
[0079] A selection unit 504 is configured to obtain the task status of each execution device, and when an abnormal execution device is detected in the device cluster, select a target execution device from the device cluster according to the task status;
[0080] The pushing unit 503 is further configured to push the subtask set corresponding to the abnormal execution device to the target execution device.
[0081] In one implementation, the determining unit 502 is further configured to:
[0082] For any execution device in the device cluster, sending a signal detection request to the any execution device at a preset time interval to request the any execution device to return target data;
[0083] Determining whether the target data sent by any execution device is received within a preset time period;
[0084] If the target data is not received, any of the execution devices is determined as an abnormal execution device.
[0085] In one implementation, the selection unit 504 is specifically configured to:
[0086] Determine an initial execution device whose task status is a target task status from the device cluster, and obtain an initial execution device set, wherein the initial execution device set includes one or more initial execution devices;
[0087] The idle load amount of each initial execution device is obtained respectively, and a target execution device is determined from the initial execution device set according to the idle load amount.
[0088] In one implementation, the selection unit 504 is specifically configured to:
[0089] Determine the total task data volume of the subtask set corresponding to the abnormal execution device;
[0090] Determine, from the initial execution device set, candidate execution devices whose idle load is greater than or equal to the total task data volume, to obtain a candidate execution device set, wherein the candidate execution device set includes one or more candidate execution devices;
[0091] A target execution device is selected from the candidate execution device set.
[0092] In one implementation, the number of devices executing devices in the device cluster is M, and the number of subtask sets is N; the pushing unit 503 is specifically configured to:
[0093] When N is less than or equal to M, selecting an execution device from the device cluster for each subtask set, and pushing each subtask set to the corresponding execution device, wherein each subtask set corresponds to a different execution device;
[0094] When N is greater than M, M subtask sets are selected from the multiple subtask sets, each subtask set in the M subtask sets is pushed to an execution device in the device cluster, and when the task state of the execution device is obtained to be the target task state, one subtask set in the NM subtask sets is pushed to the execution device whose task state is the target task state.
[0095] In one implementation, the push unit 503 is specifically configured to:
[0096] Obtaining reference task times corresponding to each subtask set, where any reference task time includes: the earliest task time or the latest task time among the task times corresponding to all target tasks in any subtask set;
[0097] Sorting the plurality of subtask sets in descending order of reference task time to obtain a task set sorting result;
[0098] Select the subtask sets that are in the top M positions from the task set sorting result.
[0099] In one implementation, the detection unit 501 is further configured to:
[0100] Get the task attributes of each pending task in the pending task set;
[0101] Calling a pre-trained task recognition model to perform recognition processing on the task attributes to obtain recognition results for each task to be processed;
[0102] Determine the pending tasks whose recognition results are target recognition results in the pending task set, and after determining the pending tasks whose recognition results are target recognition results, generate a trigger instruction associated with the trigger task, wherein the trigger instruction is associated with the pending tasks whose recognition results are target recognition results.
[0103] It is understandable that the functions of the various functional units of the task scheduling device described in the embodiment of the present application can be based on Figure 2 or Figure 4 The specific implementation process of the method in the method embodiment can be referred to Figure 2 or Figure 4 The relevant description of the method embodiment will not be repeated here.
[0104] In the embodiment of the present application, when the detection unit 501 detects a trigger instruction associated with a trigger task on the task operation interface, the target task set is determined based on the trigger instruction, and the target task set includes multiple target tasks; the determination unit 502 performs slicing processing on each target task in the target task set to obtain multiple subtask sets, each subtask set including one or more target tasks; the push unit 503 pushes each subtask set to multiple execution devices included in the device cluster according to the preset push rule, so that each execution device executes the target task in the received subtask set, and determines the respective task status according to the execution status of each execution device; the selection unit 504 obtains the task status of each execution device, and when an abnormal execution device with an abnormality is detected in the device cluster, the target execution device is selected from the device cluster according to the task status, and the push unit 503 pushes the subtask set corresponding to the abnormal execution device to the target execution device. By implementing the above method, the flexibility of trigger task scheduling can be improved.
[0105] See also Figure 6 , Figure 66 is a schematic diagram of a server structure provided in an embodiment of the present application. The server includes: a processor 601, a memory 602 and a network interface 603. The processor 601, the memory 602 and the network interface 603 can exchange data.
[0106] The processor 601 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.
[0107] The memory 602 may include a read-only memory and a random access memory, and provide program instructions and data to the processor 601. A portion of the memory 602 may also include a random access memory. When the processor 601 calls the program instructions, it is used to execute:
[0108] When a trigger instruction associated with a trigger task is detected on the task operation interface, a target task set is determined based on the trigger instruction, the target task set includes a plurality of target tasks;
[0109] Slice each target task in the target task set to obtain multiple subtask sets, each subtask set including one or more target tasks;
[0110] Pushing each subtask set to multiple execution devices included in the device cluster according to a preset push rule, so that each execution device executes the target task in the received subtask set, and determining the respective task status according to the execution status of each execution device;
[0111] The task status of each execution device is obtained. When an abnormal execution device is detected in the device cluster, a target execution device is selected from the device cluster according to the task status, and a subtask set corresponding to the abnormal execution device is pushed to the target execution device.
[0112] In one implementation, the processor 601 is further configured to:
[0113] For any execution device in the device cluster, sending a signal detection request to the any execution device at a preset time interval to request the any execution device to return target data;
[0114] Determining whether the target data sent by any execution device is received within a preset time period;
[0115] If the target data is not received, any of the execution devices is determined as an abnormal execution device.
[0116] In one implementation, the processor 601 is specifically configured to:
[0117] Determine an initial execution device whose task status is a target task status from the device cluster, and obtain an initial execution device set, wherein the initial execution device set includes one or more initial execution devices;
[0118] The idle load amount of each initial execution device is obtained respectively, and a target execution device is determined from the initial execution device set according to the idle load amount.
[0119] In one implementation, the processor 601 is specifically configured to:
[0120] Determine the total task data volume of the subtask set corresponding to the abnormal execution device;
[0121] Determine, from the initial execution device set, candidate execution devices whose idle load is greater than or equal to the total task data volume, to obtain a candidate execution device set, wherein the candidate execution device set includes one or more candidate execution devices;
[0122] A target execution device is selected from the candidate execution device set.
[0123] In one implementation, the number of devices executing devices in the device cluster is M, and the number of subtask sets is N; the processor 601 is specifically configured to:
[0124] When N is less than or equal to M, selecting an execution device from the device cluster for each subtask set, and pushing each subtask set to the corresponding execution device, wherein each subtask set corresponds to a different execution device;
[0125] When N is greater than M, M subtask sets are selected from the multiple subtask sets, each subtask set in the M subtask sets is pushed to an execution device in the device cluster, and when the task state of the execution device is obtained to be the target task state, one subtask set in the NM subtask sets is pushed to the execution device whose task state is the target task state.
[0126] In one implementation, the processor 601 is specifically configured to:
[0127] Obtaining reference task times corresponding to each subtask set, where any reference task time includes: the earliest task time or the latest task time among the task times corresponding to all target tasks in any subtask set;
[0128] Sorting the plurality of subtask sets in descending order of reference task time to obtain a task set sorting result;
[0129] Select the subtask sets that are in the top M positions from the task set sorting result.
[0130] In one implementation, the processor 601 is further configured to:
[0131] Get the task attributes of each pending task in the pending task set;
[0132] Calling a pre-trained task recognition model to perform recognition processing on the task attributes to obtain recognition results for each task to be processed;
[0133] Determine the pending tasks whose recognition results are target recognition results in the pending task set, and after determining the pending tasks whose recognition results are target recognition results, generate a trigger instruction associated with the trigger task, wherein the trigger instruction is associated with the pending tasks whose recognition results are target recognition results.
[0134] In a specific implementation, the processor 601 and the memory 602 described in the embodiment of the present application can execute the embodiment of the present application. Figure 2 or Figure 4 The implementation method described in the task scheduling method provided can also be used to implement the embodiments of this application. Figure 5 The implementation method of the described task scheduling device will not be repeated here.
[0135] In an embodiment of the present application, when a trigger instruction associated with a trigger task is detected on a task operation interface, the processor 601 can determine a target task set based on the trigger instruction, the target task set including multiple target tasks; perform slicing processing on each target task in the target task set to obtain multiple subtask sets, each subtask set including one or more target tasks; push each subtask set to multiple execution devices included in the device cluster according to a preset push rule, so that each execution device executes the target task in the received subtask set, and determines the respective task status according to the execution status of each execution device; obtain the task status of each execution device, and when an abnormal execution device with an abnormality is detected in the device cluster, select a target execution device from the device cluster according to the task status, and push the subtask set corresponding to the abnormal execution device to the target execution device. By implementing the above method, the flexibility of trigger task scheduling can be improved.
[0136] The present application also provides a computer-readable storage medium in which program instructions are stored. When the program is executed, it may include: Figure 2 or Figure 4 Part or all of the steps of the task scheduling method in the corresponding embodiment.
[0137] It should be noted that, for the above-mentioned various method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that this application is not limited by the order of the actions described, because according to this application, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by this application.
[0138] A person skilled in the art may understand that all or part of the steps in the various methods of the above embodiments may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the storage medium may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.
[0139] It should be emphasized that in order to further ensure the privacy and security of the above data, the above data can also be stored in a node of a blockchain. Among them, the blockchain referred to in this application is a new application mode of computer technologies such as distributed data storage, point-to-point transmission, consensus mechanism, encryption algorithm, etc. Blockchain is essentially a decentralized database, a string of data blocks generated by cryptographic methods. Each data block contains a batch of network transaction information, which is used to verify the validity of its information (anti-counterfeiting) and generate the next block. The blockchain can include the blockchain underlying platform, the platform product service layer, and the application service layer.
[0140] The above is a detailed introduction to a task scheduling method, device, server and storage medium provided in the embodiments of the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for general technical personnel in this field, according to the ideas of the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.
Claims
1. A task scheduling method, characterized in that: include: Get the task attributes of each pending task in the pending task set; Calling a pre-trained task recognition model to perform recognition processing on the task attributes to obtain recognition results of each task to be processed; determining the task to be processed whose recognition result is a target recognition result in the set of tasks to be processed, and after determining the task to be processed whose recognition result is a target recognition result, generating a trigger instruction associated with the trigger task, wherein the trigger instruction is associated with the task to be processed whose recognition result is the target recognition result; When a trigger instruction associated with a trigger task is detected on the task operation interface, a target task set is determined based on the trigger instruction, the target task set includes a plurality of target tasks; Slice each target task in the target task set to obtain multiple subtask sets, each subtask set including one or more target tasks; Pushing each subtask set to multiple execution devices included in the device cluster according to preset push rules, so that each execution device executes the target task in the received subtask set, and determines the respective task status according to the execution status of each execution device; wherein the number of execution devices in the device cluster is M, and the number of subtask sets is N; pushing each subtask set to multiple execution devices included in the device cluster according to preset push rules includes: when N is less than or equal to M, selecting an execution device from the device cluster for each subtask set, and pushing each subtask set to the corresponding execution device, and the execution devices corresponding to each subtask set are different ; When N is greater than M, obtain the reference task time corresponding to each subtask set, and any reference task time includes: the earliest task time or the latest task time among the task times corresponding to all target tasks of any subtask set; sort the multiple subtask sets in descending order according to the reference task time to obtain a task set sorting result; select the subtask set in the top M positions from the task set sorting result; push each subtask set in the M subtask sets to an execution device in the device cluster, and when it is obtained that the task state of the execution device is the target task state, push a subtask set in the NM subtask sets to the execution device whose task state is the target task state; Obtaining the task status of each execution device, and for any execution device in the device cluster, sending a signal detection request to any execution device at a preset time interval to request any execution device to return target data; determining whether the target data sent by any execution device is received within a preset time period; if the target data is not received, determining any execution device as an abnormal execution device; When an abnormal execution device is detected in the device cluster, a target execution device is selected from the device cluster according to the task status, and a subtask set corresponding to the abnormal execution device is pushed to the target execution device.
2. The method according to claim 1, characterized in that The selecting a target execution device from the device cluster according to the task status includes: Determine an initial execution device whose task status is a target task status from the device cluster, and obtain an initial execution device set, wherein the initial execution device set includes one or more initial execution devices; The idle load amount of each initial execution device is obtained respectively, and a target execution device is determined from the initial execution device set according to the idle load amount.
3. The method according to claim 2, characterized in that The step of determining a target execution device from the initial execution device set according to the idle load comprises: Determine the total task data volume of the subtask set corresponding to the abnormal execution device; Determine, from the initial execution device set, candidate execution devices whose idle load is greater than or equal to the total task data volume, to obtain a candidate execution device set, wherein the candidate execution device set includes one or more candidate execution devices; A target execution device is selected from the candidate execution device set.
4. A task scheduling device, characterized in that: include: A generating unit, used for obtaining the task attribute of each task to be processed in the set of tasks to be processed; Calling a pre-trained task recognition model to perform recognition processing on the task attributes to obtain recognition results of each task to be processed; determining the task to be processed whose recognition result is a target recognition result in the set of tasks to be processed, and after determining the task to be processed whose recognition result is a target recognition result, generating a trigger instruction associated with the trigger task, wherein the trigger instruction is associated with the task to be processed whose recognition result is the target recognition result; a detection unit, configured to, when a trigger instruction associated with a trigger task is detected on the task operation interface, determine a target task set based on the trigger instruction, wherein the target task set includes a plurality of target tasks; A determination unit, configured to perform slicing processing on each target task in the target task set to obtain a plurality of subtask sets, each of which includes one or more target tasks; A push unit, for pushing each subtask set to a plurality of execution devices included in a device cluster according to a preset push rule, so that each execution device executes a target task in the received subtask set, and determines the respective task status according to the execution status of each execution device; wherein the number of execution devices in the device cluster is M, and the number of subtask sets is N; pushing each subtask set to a plurality of execution devices included in the device cluster according to the preset push rule comprises: when N is less than or equal to M, selecting an execution device from the device cluster for each subtask set, and pushing each subtask set to the corresponding execution device, and the execution device corresponding to each subtask set different; when N is greater than M, obtain the reference task time corresponding to each subtask set, and any reference task time includes: the earliest task time or the latest task time among the task times corresponding to all target tasks of any subtask set; sort the multiple subtask sets in descending order according to the reference task time to obtain a task set sorting result; select the subtask set in the top M positions from the task set sorting result; push each subtask set in the M subtask sets to an execution device in the device cluster, and when the task state of the execution device is obtained to be the target task state, push a subtask set in the NM subtask sets to the execution device whose task state is the target task state; A selection unit is used to obtain the task status of each execution device, and for any execution device in the device cluster, send a signal detection request to any execution device at a preset time interval to request any execution device to return target data; determine whether the target data sent by any execution device is received within a preset time period; if the target data is not received, determine any execution device as an abnormal execution device; when an abnormal execution device is detected in the device cluster, select a target execution device from the device cluster according to the task status; The pushing unit is further used to push the subtask set corresponding to the abnormal execution device to the target execution device.
5. A server, characterized in that: The method comprises a processor and a memory, wherein the memory is used to store a computer program, the computer program comprises program instructions, and the processor is configured to call the program instructions to execute the method according to any one of claims 1 to 3.
6. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a processor, the processor is caused to perform the method according to any one of claims 1 to 3.
