Task scheduling method and device, electronic equipment and readable storage medium

Abstract

The embodiment of the invention provides a task scheduling method and device, electronic equipment and a readable storage medium. The method comprises the following steps: acquiring a task of which the task state is a to-be-scheduled state from a target task as a to-be-scheduled task; the task state of the target task is set as a to-be-scheduled state under the condition that the duration from the starting time of the target task to the current time is not greater than the preset duration. And adding the to-be-scheduled task into a local task queue based on the starting time and the ending time of the to-be-scheduled task. Detecting the to-be-scheduled task in the local task queue, starting to execute the detected first task, and ending to execute the detected second task; the first task is a to-be-scheduled task with the starting time reaching the current time, and the second task is a to-be-scheduled task with the ending time reaching the current time. Therefore, task scheduling of the to-be-scheduled task can be realized, the task starting delay is reduced, and the task scheduling efficiency can be improved.

Description

Technical Field

[0001] The present invention belongs to the field of computer technology, and in particular relates to a task scheduling method, device, electronic equipment and readable storage medium. Background Art

[0002] With the continuous development of computer technology, there are more and more scenarios for processing data based on data processing algorithms. The task processing platform can provide algorithm processing capabilities. Users can submit tasks to the task processing platform, and the task processing platform uses data processing algorithms to process the data indicated by the task.

[0003] Accordingly, in order to process the data indicated by the task, how to schedule the task becomes a technical problem that needs to be solved urgently. Summary of the invention

[0004] The present invention provides a task scheduling method, device, electronic device and readable storage medium to solve the technical problem of how to perform task scheduling.

[0005] In a first aspect, the present invention provides a task scheduling method, the method comprising:

[0006] Obtaining a task whose task status is to be scheduled from the target task as the task to be scheduled; the task status of the target task is set to the to-be-scheduled state when the time between the start time of the target task and the current time is not greater than a preset time;

[0007] Based on the start time and end time of the task to be scheduled, add the task to be scheduled to the local task queue;

[0008] Detect the tasks to be scheduled in the local task queue, start executing the first task detected, and end executing the second task detected; the first task is a task to be scheduled whose start time reaches the current time, and the second task is a task to be scheduled whose end time reaches the current time.

[0009] In a second aspect, the present invention provides a task scheduling device, the device comprising:

[0010] The first acquisition module is used to acquire a task whose task status is a to-be-scheduled state from the target task as the to-be-scheduled task; the task status of the target task is set to the to-be-scheduled state when the time length between the start time of the target task and the current time is not greater than a preset time length;

[0011] A adding module, used for adding the task to be scheduled to a local task queue based on the start time and end time of the task to be scheduled;

[0012] A detection module is used to detect the tasks to be scheduled in the local task queue, and a first execution module is used to start executing the detected first task, and an end execution module is used to end executing the detected second task; the first task is a task to be scheduled whose start time reaches the current time, and the second task is a task to be scheduled whose end time reaches the current time.

[0013] In a third aspect, the present invention provides an electronic device comprising: a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned task scheduling method when executing the program.

[0014] In a fourth aspect, the present invention provides a readable storage medium, which, when instructions in the readable storage medium are executed by a processor of an electronic device, enables the electronic device to execute the above-mentioned task scheduling method.

[0015] In an embodiment of the present invention, a task whose task status is a to-be-scheduled state is obtained from a target task as a to-be-scheduled task; the task status of the target task is set to a to-be-scheduled state when the time length from the start time of the target task to the current time is not greater than a preset time length. Based on the start time and end time of the to-be-scheduled task, the to-be-scheduled task is added to a local task queue. The to-be-scheduled tasks in the local task queue are detected, and the first task detected is started to be executed, and the second task detected is finished to be executed; the first task is a to-be-scheduled task whose start time reaches the current time, and the second task is a to-be-scheduled task whose end time reaches the current time. In this way, task scheduling of the to-be-scheduled tasks can be realized, and the task status of the target task is set to a to-be-scheduled state when the time length from the start time of the target task to the current time is less than a preset time length, that is, the target task is set to a to-be-scheduled state in advance by a preset time length, so that the target task can be pulled in advance as a to-be-scheduled task, and then the target task can be executed more timely to a certain extent, reducing the task startup delay.

[0016] Furthermore, task scheduling can be achieved by adding the tasks to be scheduled to the local task queue, detecting the first task whose start time reaches the current time and the second task whose end time reaches the current time in the local task queue, and starting and ending the first task respectively, thereby improving the efficiency of task scheduling. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 It is an overall architecture diagram provided by an embodiment of the present invention;

[0019] Figure 2 is a flowchart of a task scheduling method provided by an embodiment of the present invention;

[0020] Figure 3 is a schematic diagram of a task data structure provided by an embodiment of the present invention;

[0021] Figure 4 It is a schematic diagram of a processing flow provided by an embodiment of the present invention;

[0022] Figure 5 is another processing flow diagram provided by an embodiment of the present invention;

[0023] Figure 6 is a schematic diagram of the most recent execution period information and the current time provided by an embodiment of the present invention;

[0024] Figure 7 is a schematic diagram of a scheduling task state machine provided by an embodiment of the present invention;

[0025] Figure 8 It is a partial schematic diagram of a local memory queue provided by an embodiment of the present invention;

[0026] Fig. 9 It is a schematic diagram of an interaction process provided by an embodiment of the present invention;

[0027] Fig.10 is a structural diagram of a task scheduling device provided by an embodiment of the present invention;

[0028] Fig.11 It is a structural diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0029] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0030] First, an application scenario involved in an embodiment of the present invention is described. The task scheduling method provided in an embodiment of the present invention can be applied to an artificial intelligence (AI) algorithm task scenario. The task processing platform can schedule AI algorithm tasks based on the task scheduling method, and the task processing platform can provide various types of algorithms for users to use. Among them, task scheduling includes starting to execute a task and ending to execute a task. Starting to execute a task is to start the task, and ending to execute a task is to complete the execution. Figure 1 This is an overall architecture diagram provided by an embodiment of the present invention, such as Figure 1 As shown, the task processing platform may include an application programming interface (API) service, a scheduling service, an algorithm service, and a streaming media service. Specifically, an AI algorithm task may be a video stream algorithm task. In the video stream algorithm task scenario, the data processing algorithm may be a video stream algorithm, and the video stream algorithm may refer to an AI algorithm for processing a video stream. Users can submit user tasks to the API service of the task processing platform based on the business layer. Among them, the user task may be a video stream algorithm task, and the video stream algorithm task is used to indicate which video stream algorithm is used to process the video streams provided by which devices, and the video stream algorithm task may be configured with a data source device identifier for indicating the device providing the video stream, and a data processing algorithm identifier for indicating the video stream algorithm. Among them, one data source device corresponds to one data source device identifier, and one video stream algorithm corresponds to one data processing algorithm identifier. At least one data source device identifier and one data processing algorithm identifier may be configured in a user task.

[0031] Exemplarily, the device may be a surveillance camera, and correspondingly, the video stream may be a video stream captured by the camera. The video stream algorithm has the capabilities of detection, classification, rendering, and analysis. Exemplarily, the video stream algorithm may include a video rendering algorithm, a video detection algorithm (e.g., a liveness detection algorithm, a flame detection algorithm), a video classification algorithm, a video analysis algorithm, and the like. An algorithm service may provide a video stream algorithm, and a streaming media service may provide video stream access capabilities. Furthermore, the API service may receive tasks issued by the business layer, and provide functions such as parameter verification and path verification. For example, if the parameters of the user task meet the preset parameter rules, and the number of paths of the video stream device defined in the user task does not exceed the preset path threshold, it may be determined that the user task has passed the verification. Then, through the task distribution function, the user task that has passed the verification is sent to the scheduling service.

[0032] The scheduling service can maintain the task status and provide the algorithm service with the task acquisition capability. Specifically, the algorithm service can send heartbeats to the scheduling service at regular intervals, obtain tasks from the scheduling service, pull video streams from the streaming media service, and execute tasks based on the video stream algorithm. The scheduling service can maintain a heartbeat connection with the algorithm service, and the scheduling service can also maintain the configuration management of the algorithm instance of the algorithm service and perform resource management. Specifically, the scheduling service can perform resource configuration for the algorithm instance and manage the algorithm instance. For example, the scheduling service can configure the maximum number of tasks to be acquired when the algorithm instance of the algorithm service acquires a task. The task scheduling method provided in an embodiment of the present invention is described in detail below.

[0033] Figure 2 is a flowchart of a task scheduling method provided by an embodiment of the present invention. Figure 2 As shown, the method may include:

[0034] Step 101, obtaining a task whose task status is a to-be-scheduled state from a target task as a to-be-scheduled task; the task status of the target task is set to the to-be-scheduled state when the duration between the start time of the target task and the current time is not greater than a preset duration.

[0035] Step 102: Add the task to be scheduled to a local task queue based on the start time and end time of the task to be scheduled.

[0036] Step 103, detect the tasks to be scheduled in the local task queue, start executing the first task detected, and end executing the second task detected; the first task is a task to be scheduled whose start time reaches the current time, and the second task is a task to be scheduled whose end time reaches the current time.

[0037] Among them, the target task can be a task obtained based on the user task and actually executed by the task processing platform. Among them, a target task can be used to indicate which data processing algorithm is used to process the data provided by which device, that is, a target task is configured with a data source device identifier and a data processing algorithm identifier. Exemplarily, a target task can indicate pulling a video stream from a fixed video stream device and executing a specific video stream algorithm. The target task can also be called a scheduling task, which can be obtained by splitting and merging operations based on user tasks through the scheduling service. Among them, each scheduling task can be configured with a start time and an end time. The start time of the scheduling task indicates the time point when the task starts to be executed, and the end time of the scheduling task indicates the time point when the task ends to be executed. The start time and the end time can be obtained according to the operation cycle configuration set by the user for the task.

[0038] Furthermore, the algorithm service can be used to obtain tasks in the to-be-scheduled state from the scheduling service as to-be-scheduled tasks. The to-be-scheduled tasks are subsequently added to the local task queue, and the to-be-scheduled tasks in the local task queue are detected to detect the to-be-scheduled tasks whose start time reaches the current time in the local memory queue as the first task, and whose end time reaches the current time as the second task, and to start executing the detected first task, and to end executing the detected second task, thereby implementing a series of operations to schedule tasks. Specifically, when the first task is detected, the operation of starting to execute the detected first task is performed, and when the second task is detected, the operation of ending the execution of the detected second task is performed.

[0039] Among them, the local task queue can be a queue in the local memory of the algorithm service, and a pull-based scheduling scheme can be adopted according to a timing cycle to obtain scheduled tasks in batches at regular intervals. Specifically, the algorithm service can actively pull all target tasks in the to-be-scheduled state from the scheduling service as the to-be-scheduled tasks according to a preset acquisition cycle. The pulled tasks to be scheduled are maintained in the local task queue of the local memory. Since the operating performance of a single machine memory is higher, the method of maintaining the tasks to be scheduled in the local task queue can improve the detection efficiency of the tasks to be scheduled and improve the task scheduling performance to a certain extent.

[0040] Among them, the preset acquisition cycle can be set as needed. For example, the preset acquisition cycle can be 5 seconds, which is not limited in the embodiment of the present invention. Alternatively, the scheduling service can also distribute tasks and actively send tasks in the state to be scheduled to the algorithm service. Accordingly, the algorithm service can receive tasks in the state to be scheduled as tasks to be scheduled, which is not limited in the embodiment of the present invention. Further, the task processing platform actually performs task scheduling on tasks in the state to be scheduled. Accordingly, compared with the method of setting the task state of the target task to the state to be scheduled only when the start time reaches the current time, in the embodiment of the present application, when the duration of the start time of the target task from the current time is not greater than the preset duration, the task state of the target task is set to the state to be scheduled. That is, the target task is set to the state to be scheduled in advance by the preset duration, so that the target task can be pulled in advance as a task to be scheduled, and the scheduling task can be started more timely to a certain extent, reducing the task startup delay, thereby improving the timeliness, time accuracy and reliability of task scheduling.

[0041] For example, assuming that the preset acquisition cycle is 5 seconds, the state setting cycle is 10 seconds, the start time of target task A is 10:01, and the scheduling service performs a state setting operation at 10:59. Since the current time has not reached the start time of target task A at this time, the state of target service A is not in the state to be scheduled, and target service A will not be pulled by the algorithm service. When the scheduling service starts the next state setting 10 seconds later, the task state of target service A is set to the state to be scheduled, but the current time is 9 seconds delayed from the start time of target task A. Further, if the most recent preset acquisition cycle is reached after 5 seconds, target task A is delayed by 14 seconds before being pulled by the algorithm service, resulting in a startup delay problem for the target task. If the task state of target task A is set to the state to be scheduled in advance, for example, the task state of target task A is set to the state to be scheduled 10 seconds in advance, then the delay can be reduced to 4 seconds, reducing the task startup delay caused by the periodic operation of the scheduling service and the algorithm service, thereby making the subsequent task scheduling more accurate. In the case where the target task A is executed for the specified task execution time, for example, the target task A needs to be executed for 60 minutes, then the end time of the task will be delayed accordingly after the target task A is executed for 60 minutes. Therefore, setting the task status to the waiting state by presetting the time in advance can also reduce the task end delay to a certain extent.

[0042] The preset duration can be set in advance as needed, and the preset duration can be a positive integer, for example, the preset duration can be 5 seconds. Exemplarily, the preset duration can be no less than the preset acquisition cycle, so that the task scheduling delay caused by the algorithm service acquiring tasks according to the preset acquisition cycle can be reduced to a greater extent, and the embodiment of the present invention does not limit this.

[0043] In summary, the task scheduling method provided by the embodiment of the present invention obtains a task whose task status is a to-be-scheduled state from a target task as a to-be-scheduled task; the task status of the target task is set to a to-be-scheduled state when the time length from the start time of the target task to the current time is not greater than the preset time length. Based on the start time and end time of the to-be-scheduled task, the to-be-scheduled task is added to the local task queue. The to-be-scheduled tasks in the local task queue are detected, and the first task detected is started to be executed, and the second task detected is finished to be executed; the first task is a to-be-scheduled task whose start time reaches the current time, and the second task is a to-be-scheduled task whose end time reaches the current time. In this way, task scheduling of the to-be-scheduled tasks can be realized, and the task status of the target task is set to a to-be-scheduled state when the time length from the start time of the target task to the current time is less than the preset time length, that is, the target task is set to a to-be-scheduled state in advance by the preset time length, so that the target task can be pulled in advance as a to-be-scheduled task, and then the target task can be started to be executed more timely to a certain extent, reducing the task startup delay.

[0044] Furthermore, task scheduling can be achieved by adding the tasks to be scheduled to the local task queue, detecting the first task whose start time reaches the current time and the second task whose end time reaches the current time in the local task queue, and starting and ending the first task respectively, thereby improving the efficiency of task scheduling.

[0045] Optionally, the embodiment of the present invention may further include the following steps to obtain the target task:

[0046] Step S21, submitting the user task to a preset scheduling service, and splitting the user task into at least one device subtask through the preset scheduling service; the user task is configured with at least one data source device identifier and a data processing algorithm identifier, and one device subtask is configured with a data source device identifier, a data processing algorithm identifier and a merge flag;

[0047] Step S22: when the preset merging condition is met, the device subtask whose merging flag is the first flag value is obtained as the task to be merged, and the tasks to be merged whose data source device identifier and data processing algorithm identifier are consistent are merged to obtain the target task.

[0048] Among them, the preset scheduling service is the above-mentioned scheduling service. Specifically, the user can provide a business request to the API service to issue user tasks to the API service. The API service can save user tasks and synchronize user tasks to the scheduling service, which will batch split the user tasks and asynchronously merge the device subtasks obtained by splitting. That is, after all the currently received user tasks are split, they are merged only when the preset merging conditions are met. Specifically, the user task can be split into device subtasks according to the device dimension. Among them, the device subtask can be regarded as the task details obtained by splitting. And according to the device and algorithm dimensions, the device subtasks that are relative to the data source device and use the same data processing algorithm are merged to obtain the target task. In actual application scenarios, there may be at least two user tasks with the same data processing algorithm identifier configured, and the configured data source device identifiers overlap. Then, among the device subtasks obtained by splitting these user tasks, there are device subtasks with the same data source device identifier and data processing algorithm identifier.

[0049] Taking device_id as a data source device identifier, event_code as a data processing algorithm identifier, and s_task_id as a scheduling task identifier as an example, in an embodiment of the present invention, s_task_id can be generated based on the device_id and event_code of the device subtask. For example, the data source device identifier and the data processing algorithm identifier can be concatenated to obtain the scheduling task identifier. The scheduling task identifier is used as a unique identifier to distinguish device subtasks, and the device subtasks with the same scheduling task identifier in the task to be merged are merged. Accordingly, the merged target task can be uniquely distinguished based on the scheduling task identifier, that is, different target tasks have different scheduling task identifiers.

[0050] Figure 3 is a schematic diagram of a task data structure provided by an embodiment of the present invention, such as Figure 3 As shown, t_user_task, t_schedule_task_detail and t_schedule_task represent user tasks, device subtasks and target tasks respectively. Figure 3The main data of the task are shown in the figure. In actual application, the task may also include other data, which is not limited by the embodiment of the present invention. The main task data included in the user task: task_id, device_ids, schedule_config, status, event_code, event_config, sync_status respectively represent the user task identifier, device identifier list, operation cycle configuration information, startup status, data processing algorithm identifier, algorithm configuration, synchronization identifier. Among them, the device identifier list includes all data source device identifiers configured for the user task, the algorithm configuration includes the configuration of the algorithm parameters of the data processing algorithm used, and the startup status includes two states: started and paused. The synchronization identifier is a newly added identifier used to indicate whether the user task has been successfully submitted to the scheduling service. The main task data of the device subtask include: id, task_id, s_task_id, event_config, status, schedule_config and task_merged. Wherein, id represents the detail identifier, which is used to sequentially identify all device subtasks, and task_merged represents the merge flag. When the merge flag is the first flag value, it indicates that the device subtask has not been merged. When the merge flag is the second flag value, it indicates that the device subtask has been merged. Wherein, the first flag value and the second flag value can be set as needed. For example, the first flag value and the second flag value can be 0 and 1 respectively. The main task data of the target task include: s_task_id, device_id, event_code, status, start_time and end_time. Wherein, start_time and end_time respectively represent the start time and end time of the target task. Wherein, Figure 3 The "1:*" in the table indicates a one-to-many relationship.

[0051] For a received user task, the scheduling service can split a data source device identifier in the device identifier list, a data processing algorithm identifier in the user task, and other task information into a device subtask, wherein the device subtasks split from a user task include the same data processing algorithm identifier and different data source device identifiers. Furthermore, each device subtask split from a user task can include other task information in the user task. For example, for a device subtask, task details information configured for the data source device identifier included in the user task, such as operation cycle configuration information and algorithm configuration information, can be included.

[0052] For the split device subtasks, the merge flag can be set to the first flag value, that is, task_merged=0. Then the split device subtasks are batched into the warehouse. After all the currently received user tasks are split and all the device subtasks of this split are obtained, all the device subtasks are submitted to the database in the scheduling service. For example, the split device subtasks can be written into a subtask table (for example, a detail table), and the subtask table is submitted to the database. The database can be a relational database.

[0053] Furthermore, when the preset merging conditions are met, the scheduling service can retrieve the device subtask whose merge flag is the first flag value as the task to be merged, and perform an asynchronous merging process. Among them, the preset merging conditions can be: completing task splitting or reaching the preset asynchronous compensation cycle. Specifically, after submitting the device subtask to the database, it can be determined that the preset merging conditions are met, thereby triggering the task merge. At the same time, when the preset asynchronous compensation cycle is reached, it will also be determined that the preset merging conditions are met, thereby triggering the task merge. Taking the preset asynchronous compensation cycle of 10 seconds as an example, in an embodiment of the present invention, the scheduling service can perform asynchronous compensation, and can perform a task merge once after submitting the device subtask to the database. At the same time, an asynchronous compensation task is executed every 10 seconds: retrieve the device subtask whose merge flag is the first flag value as the task to be merged, and perform task merging. For the retrieved tasks to be merged, the tasks to be merged in which the data source device identifier and the data processing algorithm identifier are consistent (that is, the scheduling task identifier is consistent) can be merged.

[0054] In actual application scenarios, a user task is often configured with a large number of data source device identifiers. Accordingly, if the user task is split according to the device dimension, a user task can be split into a large number of device subtasks. In the face of large-scale task scenarios, synchronous merging is adopted, that is, when a device subtask is split, other subtasks whose data source device identifier and data processing algorithm identifier are consistent with the device subtask are detected once, and then merged. This synchronous split-and-merge method has low task splitting performance.

[0055] In the embodiment of the present invention, after the user task is split into device subtasks and the split is completed, the merging operation is performed only under the preset merging conditions, that is, the task splitting and merging are performed asynchronously, so the task splitting performance is better. And the tasks to be merged with the same data source device identifier and data processing algorithm identifier are merged, and then the tasks are scheduled from the dimension of the target task, which can save the resources required for task scheduling, improve resource utilization and improve the throughput of task scheduling.

[0056] Optionally, in some embodiments, the submitting the user task to the preset scheduling service may specifically include: step S211, after setting the synchronization flag of the user task to a first specified value, submitting the user task to the preset scheduling service.

[0057] Correspondingly, the embodiment of the present invention may further include: Step S31, when the user task is submitted successfully, updating the synchronization flag of the user task to a second specified value.

[0058] Step S32: acquiring a user task whose synchronization identifier is the first specified value according to a first preset period, and resubmitting the acquired user task to the preset scheduling service.

[0059] Specifically, the API service may first set a synchronization flag of a first specified value for the user task. For example, after recording the user task in the database of the API service, the sync_status of the user task may be set to 0. Then, through the exception capture mechanism, the user task is synchronously submitted to the scheduling service through the first preset interface. When the user task is successfully submitted to the scheduling service, for example, when the first preset interface returns a success message, the synchronization flag of the user task is updated to the second specified value, for example, the sync_status of the user task is set to 1.

[0060] Correspondingly, after step S211, if sync_status=0, it means that the user task has not been successfully submitted to the scheduling service, and therefore, asynchronous compensation can be performed according to the first preset period. The first preset period can be set as needed, and illustratively, the first preset period can be 10 seconds. Specifically, the API service can retrieve user tasks with sync_status=0 every 10 seconds, and the time difference between the task submission time and the current time exceeds the first preset period, and resubmit these user tasks. Specifically, the user tasks can be re-submitted to the scheduling service synchronously through the first preset interface.

[0061] In an embodiment of the present invention, by setting a synchronization identifier, when the user task is successfully submitted, the synchronization identifier of the user task is updated to a second specified value, and the user task with the synchronization identifier as the first specified value is resubmitted to the preset scheduling service according to the first preset period. In this way, the omission of user tasks can be avoided and the distributed consistency between the API service and the scheduling service can be ensured. It should be noted that since other operations will be performed on the scheduling tasks in the scheduling task table, for example, the algorithm service will pull the target task in the scheduling task table, accordingly, the scheduling service can modify the task status of the taken task to scheduling based on the compensation task. For the target task obtained by merging, the scheduling task can use an optimistic locking mechanism to write the target task into the internal scheduling task table. Among them, the scheduling task table can be an info table. Specifically, the scheduling service can detect whether the current version information of the scheduling task table is consistent with the version information before the task is merged before writing the target task into the internal scheduling task table. If consistent, the target task is written into the internal scheduling task table to ensure data consistency.

[0062] Among them, the tasks to be merged with the same data source device identifier and data processing algorithm identifier can be regarded as a group of tasks to be merged, and any group of tasks to be merged can be merged to obtain a target task by adopting the following steps S221-S222. The tasks to be merged with the acquired merge flag being the first flag value can include multiple groups of tasks to be merged.

[0063] Optionally, in some embodiments, the step of merging the tasks to be merged based on the consistent data source device identifier and data processing algorithm identifier to obtain the target task may specifically include:

[0064] Step S221: Generate the latest execution period information based on the operation cycle configuration information of each task to be merged; the latest execution period information includes the start time and the end time.

[0065] Step S222: Generate the target task based on the task detail information of each of the tasks to be merged and the most recent execution period information.

[0066] Specifically, the start time and end time of the target task / task to be scheduled refer to the start time and end time included in the current most recent execution period information of the target task / task to be scheduled. For the tasks to be merged that have consistent data source device identifiers and data processing algorithm identifiers, the target task whose scheduling task identifier is consistent with the scheduling task identifiers of these tasks to be merged can be obtained from the scheduling task table. Then the task details information of the target task is obtained. If the target task with the consistent scheduling task identifier is an existing task obtained by merging before, then the task details information of the target task can be actually obtained. Accordingly, the operation cycle configuration information in the task details information of each task to be merged among these tasks to be merged and the task details information of the obtained target task are the operation cycle configurations involved in the calculation. The most recent execution period information can be calculated based on these operation cycle configuration information.

[0067] Accordingly, the task details information involved in the merger includes: the task details information of each task to be merged and the task details information of the acquired target task. The task details information of each task to be merged, the task details information of the acquired target task, and the most recent execution period information can be merged to obtain a target task. If the target task with the same task identifier is a newly created scheduling task based on the scheduling task identifier of these tasks to be merged (that is, the target task with the same task identifier is an empty task), then the task details information of the acquired target task is empty. Accordingly, the operation cycle configuration information in the task details information of each task to be merged among these tasks to be merged is the operation cycle configuration involved in the calculation, and the most recent execution period information can be calculated based on these. Then, the task details information and the most recent execution period information of each task to be merged are merged to achieve the merger and obtain a target task. Accordingly, the target task can be used to replace the existing target task in the scheduling task table.

[0068] Specifically, the task details information involved in the merge can be merged, and then the union of the details information can be combined with the most recent execution period information to obtain a target task. The task details information involved in the merge can be merged, and specifically can include: merging the parameters in each task details information (for example, alarm parameters, threshold parameters, parameters in the algorithm configuration information, etc.) to update the parameter configuration. Exemplarily, the parameters can be taken as a union. The startup status in each task details information is integrated. Specifically, if there is a startup status of started, the startup status is set to started. On the contrary, if there is no startup status of started, that is, the startup status in all task details information involved in the merge is paused, then the startup status is set to paused.

[0069] Among them, the operation cycle configuration information can also be called the periodic task configuration, and the operation cycle configuration information can include the periodic configuration, the time configuration and the week configuration. Among them, the periodic configuration, the time configuration and the week configuration can be multiple segments. Exemplarily, the periodic configuration can indicate the date of the running task, the time configuration can indicate the time period of the daily running task, and the week configuration can indicate the day of the week when the task is run. For example, you can configure multiple periodic configurations: {[2024-1-1~2024-2-1], [2024-3-1~2024-3-15]} to indicate that the task can be run from January 1, 2024 to February 1, 2024, and from March 1, 2024 to March 15, 2024. Configure multiple time configurations: {[6:00~8:00, 12:30~14:30]} to indicate that the task can be run from 6 to 8, and from 12:30 to 14:30. You can configure the week configuration: {Monday, Friday} to indicate that the task can be run every Monday and Friday.

[0070] The most recent execution period information can also be called the next running time period, the next execution time period, the most recent running cycle, etc. A target task can have multiple execution periods, and the execution period closest to the current time is the most recent execution period. Each execution period will be used as a most recent execution period, and a most recent execution period corresponds to a task execution process. The execution period information may include a start time and an end time, and the start time and the end time can be used as a retrieval index for starting / ending the execution of the task. The start time and end time of different execution period information for the same target task are different. Furthermore, the scheduling service can add an index corresponding to the start time and end time of the most recent execution period information in the database storing the target task to facilitate the query of the start time and end time of the most recent execution period information of the target task.

[0071] Further, when calculating the most recent execution period information, all the operation cycle configuration information involved in the calculation can be analyzed to obtain all the execution periods that meet these operation cycle configurations. Then the execution period whose end time is not greater than the current time and is closest to the current time is taken as the most recent execution period information. It can be understood that as time passes, the current time changes continuously, and the most recent execution period of the task will also change. Exemplarily, it is assumed that all the operation cycle configuration information involved in the calculation includes: {[2024-10-1~2024-10-2], [6:00~8:00], [10:00~12:00], Wednesday}. Then, by taking the union of these operation cycle configuration information, all execution periods can be obtained: 6 to 8 on October 1, 2024, and 10 to 12 on October 1, 2024. In this way, even if there are many tasks and the operation cycle configuration information is complex, the most recent execution period information can be calculated. It should be noted that since the week configuration is used to indicate that the operation is performed only on a certain day of the week, the week configuration will restrict the period configuration, that is, there may be dates in the period configuration that are excluded by the week configuration, such as October 2, 2024 in the above example. In this way, the excluded dates can be ignored when taking the union.

[0072] Assuming that the current time is 5:00 on October 1, 2024, the most recent execution period information is: [6:00 on October 1, 2024, 8:00 on October 1, 2024], when the current time exceeds 8:00 on October 1, 2024, the most recent execution period information is: [10:00 on October 1, 2024, 12:00 on October 1, 2024]. When the current time exceeds 12:00 on October 1, 2024, it can be determined that there is no most recent execution period information for the task. That is, if there is still an execution period whose end time is not greater than the current time, it can be determined that there is still most recent execution period information for the target task, otherwise it can be determined that there is no most recent execution period information. It should be noted that the merge can also be achieved by directly taking the union based on the task details information involved in the merge, and then after the merge, the most recent execution period information is calculated based on all the operation cycle configuration information involved in the calculation, and the most recent execution period information is added to the merged task. The embodiment of the present invention does not limit this.

[0073] In the embodiment of the present invention, the most recent execution period information is calculated when merging and generating the target task, and the target task is generated based on the task details information and the most recent execution period information of each task to be merged. In this way, the most recent execution period information of the target task can be easily obtained later, thereby facilitating subsequent processing.

[0074] Figure 4 is a schematic diagram of a processing flow provided by an embodiment of the present invention, such as Figure 4As shown, the API service can receive the issued business request, wherein the business request can be specifically used to add (i.e., create in the figure), delete, update, start or pause user tasks. Taking the business request for adding a new user task as an example, for the newly added user task (i.e., the incoming user task), the operation of adding a task_id lock can be performed, that is, the task_id of the incoming user task is locked. In this way, the processing of the user task is controlled based on the lock mechanism to avoid the problem of data inconsistency in a concurrent environment. Then, the user task is recorded in the database, and sync_status=0 is set for the user task, so that the transaction submission can be completed, the lock is released, and the lock is ended. Exemplarily, the task information (taskinfo) of the user task can be recorded in the structured query language (My Structured Query Language, mysql) database. Exemplarily, taskinfo may include: task_id, status, timeConfig, sync_status, updatetime and other information. Among them, timeConfig represents time configuration information, such as the aforementioned operation cycle configuration information, and updatetime represents the update time of the user task.

[0075] Afterwards, the exception capture mechanism can be triggered to start the synchronous deployment (deploy) process. Specifically, in the synchronous deployment process, the task_id of the user task can be locked first, and then the first preset interface (for example, the deploy interface) can be called to synchronize the user task to the HyperText Transfer Protocol (HTTP) interface in the scheduling service. When the user task is successfully submitted to the scheduling service, sync_status=1 and updatetime=now can be set. By setting updatetime=now, the synchronization time of the user task can be recorded. Finally, the submit and unlock operation can be performed, that is, submitting sync_status and updatetime, and unlocking the task_id of the user task. It should be noted that the API service can also send user tasks that need to be updated or deleted to the scheduling service through the first preset interface.

[0076] Furthermore, the API service may also include asynchronous compensation tasks. Specifically, the asynchronous compensation process corresponding to the asynchronous compensation task may include: adding a synchronous task lock, and then retrieving the user task whose sync_status=0 and the time difference between the task submission time and the current time exceeds the first preset period (for example, sync_status=0 and now-time>10s). Then, the retrieved user tasks are executed in a loop (foreach execution), that is, the retrieved user tasks are traversed, and the above-mentioned synchronous deployment process is executed for each traversed user task to resynchronize these user tasks. Finally, after the traversal is completed, the lock is unlocked, that is, the synchronous task lock is ended.

[0077] Figure 5 is another processing flow diagram provided by an embodiment of the present invention, such as Figure 5 As shown, the scheduling service can receive user tasks from the HTTP interface. Taking the newly added user tasks as an example, for the received user tasks, the taskId lock operation can be performed, that is, the task_id of the user task is locked first. The user task is then split into device subtasks. Next, a scheduling task identifier is generated for each device subtask, and the device subtask is inserted into the detail table. Then, set task_merged=0 for the device subtask. Next, submit the subtask table to the database and release the lock. Among them, the field in the detail table originally used to store the task identifier, for example, the aid field can be empty, or the field originally used to store the task identifier can be deleted in advance, and the embodiment of the present invention does not limit this.

[0078] After that, when the subtask table including the newly split device subtask is submitted to the database, the preset merge condition is met, and the asynchronous process can be triggered to retrieve the device subtask with task_merged=0 and start the merge process. Specifically, in this merge process, s_task_ids represents the s_task_id of all the device subtasks with task_merged=0. For each s_task_id (i.e. Figure 5 Each s_id in the transaction, the following operations can be performed using the transaction management mechanism: first open the transaction, then determine whether there is a scheduled task, that is, determine whether there is a scheduled task that matches the s_task_id. If so, perform the merge operation directly. If not, you can first create a scheduled task for the s_task_id. Then perform the merge operation. After that, you can set task_merged=1 for these device subtasks, and finally commit the transaction. The scheduling service can also periodically retrieve device subtasks with task_merged=0 through asynchronous compensation and merge them according to the merge process.

[0079] Specifically, the merge operation may include: obtaining all corresponding task details information. Specifically, all corresponding detail information may be obtained based on the info table, that is, obtaining the task details information of the target task whose scheduling task identifier is consistent with the s_task_id from the info table, and obtaining the task details information of all split subtasks corresponding to the scheduling task identifier of the s_task_id. Then, the target task is obtained by merging based on the task details information. Finally, an optimistic lock update is performed, that is, the target task is written into the scheduling task table using the optimistic lock mechanism.

[0080] It should be noted that the HTTP interface may specifically include a new / update interface and a delete interface. In an embodiment of the present invention, it may also include that in response to a user task update or delete request, the API service sends the user task that needs to be updated or the user task that needs to be deleted to the scheduling service. The scheduling service may split these user tasks, and then update the detail table based on the device subtasks obtained by the split, or soft-delete the detail table. Specifically, when the user task is received from the deletion interface, if the scheduling service identifier of the split device subtask is detected in the detail table, the detail table is soft-deleted to avoid unnecessary deletion operations. When the user task is received from the new / update interface, it can be detected whether the detail table contains the scheduling service identifier of the split device subtask. If it exists, it can be determined that the user task is a user task that needs to be updated. Accordingly, the operation of updating the detail table can be performed. If it does not exist, it can be processed according to the aforementioned processing flow for the newly added user task.

[0081] Optionally, in some embodiments, the embodiments of the present invention may further include the following steps:

[0082] Step S41: For any of the target tasks, when the end time of the target task is less than the current time, determine new most recent execution period information for the target task based on the operation cycle configuration information of the target task.

[0083] Step S42: Based on the new most recent execution period information, update the current most recent execution period information of the target task.

[0084] In an embodiment of the present invention, the most recent execution period information can be recalculated when the end time of the target task is less than the current time. A timed task can be used to periodically check whether the end time of each target task is less than the current time according to a preset refresh cycle. If it is less than, the most recent execution period information of the target task is refreshed. Specifically, the refresh operation may include: obtaining the operation cycle configuration information of the target task, and recalculating the most recent execution period information based on the obtained operation cycle configuration information to obtain new most recent execution period information of the target task. Then, the start time and end time in the new most recent execution period information are written into the start_time field and end_time field of the target task, respectively.

[0085] Among them, the operation cycle configuration information of the target task can correspond to the operation cycle configuration information of all device subtasks used when the target task is merged and generated, and the operation cycle configuration information of the target task can be read from the task data of the target task. Specifically, all execution periods that meet the operation cycle configuration information of the target task can be determined based on the operation cycle configuration information of the target task. Then, the execution period whose end time is not greater than the current time and is closest to the current time is used as the new latest execution period information. Exemplarily, it is assumed that all execution periods include: 6:00 to 8:00 on October 1, 2024 and 10:00 to 12:00 on October 1, 2024. The current time is 8:01 on October 1, 2024. Then 10:00 to 12:00 on October 1, 2024 can be determined as the new latest execution period information. In the case where there is no latest execution period information for the target task, the start_time field and the end_time field of the target task can be set to preset values, that is, the end time and the start time are set to preset values. Exemplarily, the preset value may be set as needed, for example, the preset value may be -1.

[0086] Figure 6 is a schematic diagram of the most recent execution period information and the current time provided by an embodiment of the present invention, such as Figure 6As shown, now represents the current time, start and end represent the start time and end time of the most recent execution period information, respectively. Corresponding to the four different relative relationships between the current time and the most recent execution period information, the most recent execution period information can be divided into four types. Among them, type 1 corresponds to scenario 1, type 1 is that the current time is less than the start time, the task has not started, indicating that the start time of the target task has not been reached, and the target task is not started. Type 2 corresponds to scenario 2, type 2 is that the current time is greater than the start time and less than the end time, the task is running, indicating that the target task is being executed. Type 3 corresponds to scenario 3, type 3 is that the current time is greater than the end time, and the task is executed once. Type 4 corresponds to scenario 4, type 4 indicates that the end time and start time of the target task are preset values, start_time=end_time=-1, there is no new most recent execution period information for the target task, the execution period of the target task is all over, and the target task is no longer executed. Among them, type 3 is an instantaneous type. When type 3 is reached, the scheduling service will update the most recent execution period information of the target task based on the scheduled task. Accordingly, the most recent execution period information will be subsequently converted to type 1, type 2 or type 4.

[0087] It should be noted that the latest execution period information can also be recalculated when a task change is detected. Specifically, the operation cycle configuration information of the changed target task can be obtained, and based on the obtained operation cycle configuration information, new latest execution period information can be determined for the target task, which will not be described in detail in the embodiment of the present invention.

[0088] In the embodiment of the present invention, when the end time of the target task is less than the current time, new recent execution period information is determined for the target task based on the operation cycle configuration information of the target task. Based on the new recent execution period information, the current recent execution period information of the target task is updated. In this way, by updating the recent execution period information of the target task, the accuracy of the recent execution period information can be ensured, thereby ensuring the accuracy of subsequent scheduling according to the recent execution period information.

[0089] Optionally, in some embodiments, the embodiments of the present invention may further include:

[0090] Step S51: For any of the target tasks, the task status of the target task is set based on the most recent execution period information of the target task.

[0091] Step S52: for a target task whose task status is suspended, if the time between the start time of the target task and the current time is not greater than the preset time, the task status of the target task is set to the to-be-scheduled state.

[0092] The task status of the target task can be divided into suspended status, scheduling status and stopped status. The suspended status means that the current time is less than the start time of the target task, the scheduling status means that the current time is less than the end time of the target task and greater than the start time of the target task, and the stopped status means that the target task does not have the latest execution period information, that is, the target task is no longer executed and the target task has stopped.

[0093] Optionally, in some embodiments, the step of setting the task status of the target task based on the most recent execution period information of the target task may specifically include:

[0094] When the current time is less than the start time of the target task, the task state of the target task is set to the suspended state.

[0095] When the current time is less than the end time of the target task and greater than the start time of the target task, the task state of the target task is set to a scheduling state.

[0096] When the end time and the start time of the target task are preset values, the task state of the target task is set to a stopped state; the preset value is set when there is no recent execution period information of the target task.

[0097] Specifically, the scheduling service can periodically check each target task according to the scheduled task and the preset period to synchronize the status. If it is detected that the current time is less than the start time of the target task, that is, the target task meets the above type 1, then if the task state of the target task is not in the suspended state, the task state of the target task is set to the suspended state. If it is detected that the current time is less than the end time of the target task and greater than the start time of the target task, that is, the target task meets the above type 2, then if the task state of the target task is not in the scheduled state, the task state of the target task is set to the scheduled state. If it is detected that the end time and start time of the target task are preset values, that is, the target task meets the above type 4, then if the task state of the target task is not in the stopped state, the task state of the target task is set to the stopped state. Exemplarily, Figure 7 is a schematic diagram of a scheduling task state machine provided by an embodiment of the present invention, such as Figure 7As shown, for the suspended target task, when the start time is reached, the task status of the target task is to be scheduled. Correspondingly, after the target task is pulled, the task status of the target task is scheduled. Furthermore, when a heartbeat or alarm is received from the algorithm service, the task status of the target task is running. If the algorithm service loses the heartbeat or the algorithm reports an error, the task status of the target task can be set to abnormal.

[0098] When the end time is reached, based on the aforementioned recent execution period information update logic, if the target task still has new recent execution period information, then after the update, the task state of the target task is set to suspended again. If the target task does not have new recent execution period information, that is, there is no subsequent execution time, the task state of the target task is set to stopped. In an embodiment of the present invention, the scheduling service can modify the task state based on the compensation task, wherein the compensation task can be unlocked when modifying the task state to ensure processing performance.

[0099] Furthermore, for a target task whose task status is suspended, the task status of the target task can be set to pending scheduling if the time between the start time of the target task and the current time is not greater than the preset time. In this way, the target task can be pulled in advance by the algorithm service, thereby reducing the task scheduling delay.

[0100] In an embodiment of the present invention, for the target task obtained by merging, the task state of the target task is further set to ensure the state consistency of the task. Compared with a full scan of the target task, that is, according to the preset scanning cycle, for each target task, all the operation cycle configuration information of the target task is obtained, and based on the current time, it is calculated whether the current time has entered the most recent execution period of the target task. If entered, the task state of the target task is set to a waiting-to-be-scheduled state. Since the operation cycle configuration information is often more complex, when the task scale is large, it will cause the problem of excessive load, and the performance of task scheduling is poor.

[0101] In an embodiment of the present invention, the most recent execution period information will be calculated when the target task is merged and generated, and the task status will be set based on the most recent execution period information. In addition, when the end time of the target task is less than the current time (i.e., the current most recent execution period information ends), the most recent execution period information will be updated accordingly, thereby causing the task status to be updated accordingly. Accordingly, when the scheduling service retrieves the tasks that need to be scheduled, it directly detects the target tasks whose task status is suspended based on the maintained task status, and filters the tasks that need to be set to the to-be-scheduled state, without having to perform a full scan and load the full run cycle configuration information. Therefore, the load can be reduced, the performance of task scheduling can be improved, and the system stability of the platform can be improved. In this way, large-scale and complex run cycle configurations can be handled, and the scheduling of tasks with large-scale and complex run cycle configurations can be achieved.

[0102] Optionally, in some embodiments, the step of adding the task to be scheduled to the local task queue based on the start time and end time of the task to be scheduled may specifically include:

[0103] Step 1021: Generate a key name based on the start time and end time of the task to be scheduled, and generate a key value representing the task identifier of the task to be scheduled;

[0104] Step 1022: Write the key name and the key value into the local task queue in the form of key-value pairs.

[0105] The algorithm service can use key-value pairs to store the tasks to be scheduled locally. Specifically, a corresponding key-value pair is set for a task to be scheduled. A key-value pair is written into the queue as an element in the local task queue, and can be written to the end of the queue. The key-value pair is memory data, and the algorithm service can use a treemap to maintain memory data. The key name can be in seconds, that is, the key name can represent a timestamp with an accuracy of seconds.

[0106] Specifically, the task identifier of the task to be scheduled refers to the scheduling task identifier s_task_id of the task to be scheduled. For a task to be scheduled, the scheduling task identifier of the task to be scheduled can be used as the value in the corresponding key-value pair, that is, value = s_task_id. It should be noted that when writing to the queue, key-value pairs with the same key can be merged. Specifically, the value can be set to a list composed of values ​​in key-value pairs with the same key, and the list includes all scheduling task identifiers in key-value pairs with the same key. By establishing key-value pairs and storing them in the local task queue, an ordered task list mapping can be established. That is, in a key-value pair, the key can correspond to multiple scheduling task identifiers to save queue space and facilitate scheduling. For example, Figure 8 is a partial schematic diagram of a local memory queue provided by an embodiment of the present invention, such as Figure 8 As shown in the figure, key = 1720670015, the corresponding value can include the scheduling task identifiers of the four tasks t1, t2, t3 and t4. key = 1720671058, the corresponding value can include the scheduling task identifiers of the two tasks t3 and t4. The scale of the key can be accurate to seconds.

[0107] In the embodiment of the present invention, a key name is generated based on the start time and end time of the task to be scheduled, and a key value representing the task identifier of the task to be scheduled is generated. The key name and the key value are written into the local task queue in the form of a key-value pair. In this way, the task identifier of the task to be scheduled and its time information can be stored in an orderly manner.

[0108] Optionally, in some embodiments, the step of generating a key name based on the start time and end time of the task to be scheduled may specifically include:

[0109] Step 1021a: when the current time is less than the start time of the task to be scheduled, generate a key name representing the start time of the task to be scheduled, and generate a key name representing the end time of the task to be scheduled.

[0110] Step 1021b: When the current time is less than the end time of the task to be scheduled and greater than the start time, generate a key name representing the end time of the task to be scheduled.

[0111] Correspondingly, the embodiment of the present invention may further include: step S61, when the current time is less than the end time of the task to be scheduled and greater than the start time, directly starting to execute the task to be scheduled.

[0112] Since the algorithm service may have an exception when acquiring the tasks to be scheduled, some tasks in the state of being scheduled may be omitted. Accordingly, when the tasks to be scheduled are acquired later, the current time may be less than the end time of the tasks to be scheduled and greater than the start time, or the current time may be greater than the end time of the tasks to be scheduled. Accordingly, when generating the key name, the judgment can be made based on the start time and end time of the tasks to be scheduled. If the current time is less than the start time of the tasks to be scheduled (now<start_time), the start time and end time of the tasks to be scheduled can be queued. Specifically, a key name representing the start time can be generated for the tasks to be scheduled, and the key name stores the start time of the tasks to be scheduled. And, a key name representing the end time can be generated for the tasks to be scheduled, and the key name stores the end time of the tasks to be scheduled. Accordingly, the corresponding key-value pairs generated for the tasks to be scheduled include: a key-value pair corresponding to the start time and a key-value pair corresponding to the end time. Exemplarily, let key1 be the key name representing the start time, and key2 be the key name representing the end time, then two key-value pairs, key1-value and key2-value, can be generated for the task to be scheduled. Among them, the value in key1-value and key2-value corresponding to the task to be scheduled stores the scheduling task identifier of the task to be scheduled, and key1 and key2 respectively store the start time and end time of the task to be scheduled. Accordingly, when writing to the queue, all key-value pairs with the same key1 can be merged into one key-value pair and written into the local task queue. All key-value pairs with the same key2 can be merged into one key-value pair and written into the local task queue.

[0113] If the current time is less than the end time of the task to be scheduled and greater than the start time of the task to be scheduled (start_time<now<end_time), the current time is already greater than the start time of the task to be scheduled. Putting the start time into the queue and writing the key-value pair representing the start time into the queue will cause unnecessary waste of queue space. The end time of the task to be scheduled can be put into the queue and the task to be scheduled can be directly started. Specifically, a key name representing the end time can be generated for the task to be scheduled, and the key name stores the end time of the task to be scheduled. Accordingly, the corresponding key-value pair generated for the task to be scheduled only includes: the key-value pair corresponding to the end time. Exemplarily, a key-value pair of key2-valu can be generated for the task to be scheduled. Among them, the value in the key2-value corresponding to the task to be scheduled stores the scheduling task identifier of the task to be scheduled, and key2 stores the end time of the task to be scheduled. Accordingly, when writing to the queue, all key-value pairs with the same key2 can be merged into one key-value pair and written into the local task queue. Furthermore, if the current time is greater than the end time of the task to be scheduled (now>end_time), the processing flow of the task to be scheduled can be directly ended.

[0114] In an embodiment of the present invention, a key name is generated based on the relative greater than the current time and the start time and end time of the task to be scheduled. This ensures the accuracy of the generated key-value pairs and avoids writing unnecessary key-value pairs into the queue.

[0115] Optionally, in some embodiments, the above step of detecting the tasks to be scheduled in the local task queue may specifically include:

[0116] Step 1031: Obtain the key name stored in the head of the local task queue according to the second preset period to obtain the head key name.

[0117] Step 1032: When it is detected that the queue head key name indicates the start time and the queue head key name is equal to the current time, the to-be-scheduled task indicated by the key value corresponding to the queue head key name is determined as the first task.

[0118] Step 1033: When it is detected that the queue head key name indicates the end time and the queue head key name is equal to the current time, the to-be-scheduled task indicated by the key value corresponding to the queue head key name is determined as the second task.

[0119] Among them, the second preset period can be set as needed, and illustratively, the second preset period can be 1 second. The elements in the queue are first in, first out, and the time represented by the key name in the key-value pair stored in the head of the local task queue is the smallest, that is, the closest to the current time. Therefore, the head of the team can be detected every time. Specifically, the key name stored in the head of the team can be obtained, that is, the key name in the top element of the treemap in the head element of the team can be obtained as the head key name. If the head key name is key1, it can be determined that the head key name represents the start time. Further, if the content of the head key name is equal to the current time, the task to be scheduled represented by value in the head key-value pair can be determined as the first task. Among them, the specific value of value in the head key-value pair can be read. The tasks to be scheduled represented by each scheduling task identifier in the specific value read are determined as the tasks to be scheduled represented by value in the head key-value pair. Accordingly, the first task can be started. For example, the algorithm instance can be scheduled to start executing the first task. It should be noted that in order to avoid missing the to-be-scheduled tasks whose current time is greater than the start time due to anomalies, when the team head key name represents the start time and the team head key name is not greater than the current time, it can be determined that the start time of the to-be-scheduled task indicated by the key value corresponding to the team head key name reaches the current time, and it is determined as the first task.

[0120] If the team leader key is key2, it can be determined that the team leader key represents the end time. Further, if the content of the team leader key is equal to the current time, the specific value of value in the team leader key-value pair can be read. The task to be scheduled represented by each scheduling task identifier in the specific value read is determined as the second task. Accordingly, the execution of the second task can be terminated. In this way, the task end can be controlled more accurately and the task end delay can be reduced. For example, the algorithm instance that executes the second task can be scheduled to end the execution of the second task. It should be noted that in order to avoid the exception causing the omission of the task to be scheduled whose end time is less than the current time, when the team leader key represents the end time and the team leader key is not greater than the current time, it can be determined that the end time of the task to be scheduled indicated by the key value corresponding to the team leader key reaches the current time, and it is determined as the second task. Further, in an embodiment of the present invention, after determining the first task or the second task based on the team leader key, the team leader key-value pair can be removed from the local task queue to facilitate subsequent detection.

[0121] Fig. 9 is a schematic diagram of an interaction process provided by an embodiment of the present invention, such as Fig. 9As shown, the scheduling service can update the most recent execution period information of the target task, and then set the task status of the target task to the to-be-scheduled state in advance by presetting the duration. Correspondingly, the algorithm service can execute once every 5 seconds to pull the to-be-scheduled tasks from the scheduling service. Then, for the pulled tasks to be scheduled, the key-value pairs corresponding to the tasks to be scheduled can be written into the local memory queue. At the same time, the algorithm service executes once every 1 second to obtain the head key name of the team. Determine whether the head key name of the team is equal to the current time. If they are equal, start / end the execution of the to-be-scheduled task represented by the value in the head key-value pair of the team. If they are not equal, the current process can be ended. Specifically, when the head key name of the team indicates the start time, execution can be started. When the head key name of the team indicates the end time, execution can be stopped.

[0122] In the embodiment of the present invention, the task state of the target task is set to the waiting state by presetting the duration in advance, so that the algorithm service can obtain the waiting task in advance. At the same time, by maintaining the local task queue in the local memory of the algorithm service and asynchronously obtaining the head key name of the local task queue every second for processing, the task delay can be effectively controlled.

[0123] Optionally, the API service, scheduling service, algorithm service and streaming media service in the embodiment of the present invention can be centrally deployed on the same hardware device, for example, all deployed in the same cluster, or, alternatively, deployed in different clusters. For example, the algorithm service can be deployed in a graphics processing unit (GPU) cluster and accessed using a node type (nodetype) method. The API service, scheduling service and streaming media service can be deployed in a central processing unit (CPU) cluster, and the GPU cluster and the CPU cluster can interact based on HTTP. Among them, the scheduling service can include a relational database, for example, a mysql database and a remote dictionary service (Remote DictionaryServer, redis) database, etc., which can be deployed in advance. For the API service, scheduling service and streaming media service, these three services can be CPU services and deployed using a container orchestration platform (Kubernetes, K8s), HTTP interaction can be used between services, and the number of instances specifically included in the API service, scheduling service and streaming media service can be expanded according to the scale. Optionally, the embodiment of the present invention can also be applied to other long-term task scenarios, for example, audio stream algorithm task scenarios, etc., and the embodiment of the present invention does not limit this. Among them, a long-term task refers to a task with multiple execution time periods, corresponding to a periodic task that needs to be executed multiple times.

[0124] Fig.10 : is a structural diagram of a task scheduling device provided by an embodiment of the present invention, and the device may include:

[0125] The first acquisition module 201 is used for the first acquisition module, and is used to acquire a task whose task status is a to-be-scheduled state from the target task as the to-be-scheduled task; the task status of the target task is set to the to-be-scheduled state when the time length between the start time of the target task and the current time is not greater than a preset time length;

[0126] The adding module 202 is used to add the task to be scheduled to the local task queue based on the start time and end time of the task to be scheduled;

[0127] The detection module 203 is used to detect the tasks to be scheduled in the local task queue, and the first execution module 204 is used to start executing the detected first task, and the end execution module 205 is used to end executing the detected second task; the first task is a task to be scheduled whose start time reaches the current time, and the second task is a task to be scheduled whose end time reaches the current time.

[0128] Optionally, the device also includes: a first submission module, used to submit the user task to a preset scheduling service, and a splitting module, used to split the user task into at least one device subtask through the preset scheduling service; the user task is configured with at least one data source device identifier and a data processing algorithm identifier, and one of the device subtasks is configured with a data source device identifier, a data processing algorithm identifier and a merge flag; an acquisition module, used to acquire a device subtask whose merge flag is a first flag value as a task to be merged when a preset merging condition is met, and a merging module, used to merge the tasks to be merged based on the consistent data source device identifier and the data processing algorithm identifier to obtain the target task.

[0129] Optionally, the first submission module 204 is specifically used to: submit the user task to the preset scheduling service after setting the synchronization identifier of the user task to the first specified value; the device also includes: a first update module, used to update the synchronization identifier of the user task to a second specified value if the user task is successfully submitted; a second submission module, used to obtain the user task whose synchronization identifier is the first specified value according to a first preset period, and resubmit the obtained user task to the preset scheduling service. Optionally, the merging module is specifically used to: generate the most recent execution period information based on the operation cycle configuration information of each of the tasks to be merged; the most recent execution period information includes the start time and the end time; generate the target task based on the task detail information of each of the tasks to be merged and the most recent execution period information.

[0130] Optionally, the device further includes: a determination module for determining new most recent execution period information for any of the target tasks based on the operation cycle configuration information of the target task when the end time of the target task is less than the current time; a second update module for updating the current most recent execution period information of the target task based on the new most recent execution period information. Optionally, the device further includes: a first setting module for setting the task status of the target task based on the most recent execution period information of the target task for any of the target tasks; a second setting module for setting the task status of the target task to the to-be-scheduled status for the target task whose task status is suspended when the duration between the start time of the target task and the current time is not greater than the preset duration.

[0131] Optionally, the joining module 202 is specifically used to: generate a key name based on the start time and end time of the task to be scheduled, and generate a key value representing the task identifier of the task to be scheduled; write the key name and the key value into the local task queue in the form of a key-value pair. Optionally, the joining module 202 is also specifically used to: generate a key name representing the start time of the task to be scheduled, and generate a key name representing the end time of the task to be scheduled when the current time is less than the start time of the task to be scheduled; generate a key name representing the end time of the task to be scheduled when the current time is less than the end time of the task to be scheduled and greater than the start time; the device also includes: a second execution module, which is used to directly start executing the task to be scheduled when the current time is less than the end time of the task to be scheduled and greater than the start time.

[0132] Optionally, the detection module 203 is specifically used to: obtain the key name stored in the head of the local task queue according to the second preset period to obtain the head key name; when it is detected that the head key name indicates the start time and the head key name is equal to the current time, determine the to-be-scheduled task indicated by the key value corresponding to the head key name as the first task; when it is detected that the head key name indicates the end time and the head key name is equal to the current time, determine the to-be-scheduled task indicated by the key value corresponding to the head key name as the second task. Optionally, the data source device identifier is used to indicate the device that provides the video stream, the data processing algorithm identifier is used to indicate the video stream algorithm, and the user task is a video stream algorithm task.

[0133] In summary, the task scheduling device provided by the embodiment of the present invention obtains a task whose task status is a to-be-scheduled state from a target task as a to-be-scheduled task; the task status of the target task is set to a to-be-scheduled state when the time length from the start time of the target task to the current time is not greater than the preset time length. Based on the start time and end time of the to-be-scheduled task, the to-be-scheduled task is added to the local task queue. The to-be-scheduled tasks in the local task queue are detected, and the first task detected is started to be executed, and the second task detected is finished to be executed; the first task is a to-be-scheduled task whose start time reaches the current time, and the second task is a to-be-scheduled task whose end time reaches the current time. In this way, task scheduling of the to-be-scheduled tasks can be realized, and the task status of the target task is set to a to-be-scheduled state when the time length from the start time of the target task to the current time is less than the preset time length, that is, the target task is set to a to-be-scheduled state in advance by the preset time length, so that the target task can be pulled in advance as a to-be-scheduled task, and then the target task can be started to be executed more timely to a certain extent, reducing the task startup delay. Furthermore, task scheduling can be achieved by adding the tasks to be scheduled to the local task queue, detecting the first task whose start time reaches the current time and the second task whose end time reaches the current time in the local task queue, and starting and ending the first task respectively, thereby improving the efficiency of task scheduling.

[0134] The present invention also provides an electronic device, see Fig.11 , comprising: a processor 901, a memory 902, and a computer program 9021 stored in the memory and executable on the processor, wherein the processor implements the task scheduling method of the aforementioned embodiment when executing the program. The present invention also provides a readable storage medium, which enables the electronic device to execute the task scheduling method of the aforementioned embodiment when the instructions in the readable storage medium are executed by the processor of the electronic device.

[0135] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment. It should be noted that the various information and data obtained in the embodiments of the present invention are obtained with the authorization of the information / data holder. All actions to obtain signals, information or data in this application are carried out in accordance with the relevant data protection laws and policies of the country where they are located, and with the authorization given by the corresponding device owner.

[0136] The algorithms and displays provided herein are not inherently related to any particular computer, virtual system, or other device. Various general-purpose systems may also be used with the teachings based thereon. Based on the above description, the structure required to construct such a system is obvious. In addition, the present invention is not directed to any particular programming language. It should be understood that the content of the present invention described herein can be implemented using various programming languages, and the above description of a specific language is to disclose the best implementation of the present invention. In the description provided herein, a large number of specific details are described. However, it is understood that embodiments of the present invention can be practiced without these specific details. In some instances, well-known methods, structures, and techniques are not shown in detail so as not to obscure the understanding of this specification. Similarly, it should be understood that in order to simplify the present invention and help understand one or more of the various inventive aspects, in the above description of the exemplary embodiments of the present invention, the various features of the present invention are sometimes grouped together into a single embodiment, figure, or description thereof. However, the disclosed method should not be interpreted as reflecting the following intention: the claimed invention requires more features than the features explicitly stated in each claim. More specifically, as reflected in the following claims, the inventive aspects are less than all the features of the single embodiment disclosed above. Thus, the claims following the Detailed Description are hereby expressly incorporated into this Detailed Description, with each claim standing on its own as a separate embodiment of this invention.

[0137] Those skilled in the art will appreciate that the modules in the devices in the embodiments may be adaptively changed and set in one or more devices different from the embodiments. The modules or units or components in the embodiments may be combined into one module or unit or component, and in addition they may be divided into multiple submodules or subunits or subcomponents. Except that at least some of such features and / or processes or units are mutually exclusive, all features disclosed in this specification (including the accompanying claims, abstracts and drawings) and all processes or units of any method or device disclosed in this manner may be combined in any combination. Unless otherwise expressly stated, each feature disclosed in this specification (including the accompanying claims, abstracts and drawings) may be replaced by an alternative feature that provides the same, equivalent or similar purpose. The various component embodiments of the present invention may be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. Those skilled in the art will appreciate that a microprocessor or digital signal processor (DSP) may be used in practice to implement some or all of the functions of some or all of the components in the sorting device according to the present invention. The present invention may also be implemented as a device or apparatus program for executing part or all of the methods described herein. Such a program for implementing the present invention may be stored on a computer readable medium, or may have the form of one or more signals. Such a signal may be downloaded from an Internet website, or provided on a carrier signal, or provided in any other form.

[0138] It should be noted that the above embodiments illustrate the present invention rather than limit it, and that those skilled in the art may devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference symbol between brackets shall not be construed as a limitation on the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "one" or "an" preceding an element does not exclude the presence of a plurality of such elements. The present invention may be implemented by means of hardware comprising a number of different elements and by means of a suitably programmed computer. In a unit claim enumerating a number of devices, several of these devices may be embodied by the same hardware item. The use of the words first, second, and third, etc., does not indicate any order. These words may be interpreted as names.

[0139] The user information (including but not limited to the user's device information, user personal information, etc.), related data, etc. involved in the present invention are all information authorized by the user or authorized by all parties. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working process of the system, device and unit described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here. The above is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention. The above is only a specific implementation method of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be covered within the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A task scheduling method, characterized in that: The method comprises: Obtaining a task whose task status is to be scheduled from the target task as the task to be scheduled; the task status of the target task is set to the to-be-scheduled state when the time between the start time of the target task and the current time is not greater than a preset time; Based on the start time and end time of the task to be scheduled, add the task to be scheduled to the local task queue; Detect the tasks to be scheduled in the local task queue, start executing the first task detected, and end executing the second task detected; the first task is a task to be scheduled whose start time reaches the current time, and the second task is a task to be scheduled whose end time reaches the current time.

2. The method according to claim 1, characterized in that The method further comprises: Submitting a user task to a preset scheduling service, and splitting the user task into at least one device subtask through the preset scheduling service; the user task is configured with at least one data source device identifier and a data processing algorithm identifier, and one device subtask is configured with a data source device identifier, a data processing algorithm identifier, and a merge flag; When the preset merging condition is met, the device subtask with the merging flag being the first flag value is obtained as the task to be merged, and the tasks to be merged with the same data source device identifier and data processing algorithm identifier are merged to obtain the target task.

3. The method according to claim 2, characterized in that The submitting the user task to the preset scheduling service comprises: after setting the synchronization flag of the user task to a first specified value, submitting the user task to the preset scheduling service; The method further comprises: in case the user task is submitted successfully, updating the synchronization flag of the user task to a second specified value; The user task whose synchronization identifier is the first specified value is acquired according to a first preset period, and the acquired user task is resubmitted to the preset scheduling service.

4. The method according to claim 2, characterized in that: The tasks to be merged based on the consistent data source device identifier and data processing algorithm identifier are merged to obtain the target task, including: Generate the latest execution period information based on the operation cycle configuration information of each task to be merged; the latest execution period information includes the start time and the end time; The target task is generated based on the task detail information of each of the tasks to be merged and the most recent execution period information.

5. The method according to claim 1, characterized in that: The method further comprises: For any of the target tasks, when the end time of the target task is less than the current time, determining new most recent execution period information for the target task based on the operation cycle configuration information of the target task; Based on the new most recent execution period information, the current most recent execution period information of the target task is updated.

6. The method according to any one of claims 1 to 5, characterized in that: The method further comprises: For any of the target tasks, based on the most recent execution period information of the target task, setting the task status of the target task; For a target task whose task status is a suspended status, when the time length between the start time of the target task and the current time is not greater than the preset time length, the task status of the target task is set to the to-be-scheduled status.

7. The method according to any one of claims 1 to 5, characterized in that: The step of adding the task to be scheduled to a local task queue based on the start time and the end time of the task to be scheduled includes: Generate a key name based on the start time and end time of the task to be scheduled, and generate a key value representing the task identifier of the task to be scheduled; The key name and the key value are written into the local task queue in the form of key-value pairs.

8. The method according to claim 7, characterized in that The generating of the key name based on the start time and end time of the task to be scheduled includes: In the case where the current time is less than the start time of the task to be scheduled, generating a key name representing the start time of the task to be scheduled, and generating a key name representing the end time of the task to be scheduled; When the current time is less than the end time of the task to be scheduled and greater than the start time, generate a key name representing the end time of the task to be scheduled; The method further includes: when the current time is less than the end time of the task to be scheduled and greater than the start time, directly starting to execute the task to be scheduled.

9. The method according to claim 7, characterized in that: The detecting the tasks to be scheduled in the local task queue includes: Obtaining the key name stored in the head of the local task queue according to the second preset period to obtain the head key name; When it is detected that the queue head key name indicates the start time and the queue head key name is equal to the current time, the to-be-scheduled task indicated by the key value corresponding to the queue head key name is determined as the first task; When it is detected that the queue head key name indicates the end time and the queue head key name is equal to the current time, the to-be-scheduled task indicated by the key value corresponding to the queue head key name is determined as the second task.

10. The method according to claim 2, characterized in that The data source device identifier is used to indicate a device that provides a video stream, the data processing algorithm identifier is used to indicate a video stream algorithm, and the user task is a video stream algorithm task.

11. A task scheduling device, characterized in that: The device comprises: The first acquisition module is used to acquire a task whose task status is a to-be-scheduled state from the target task as the to-be-scheduled task; the task status of the target task is set to the to-be-scheduled state when the time length between the start time of the target task and the current time is not greater than a preset time length; A adding module, used for adding the task to be scheduled to a local task queue based on the start time and end time of the task to be scheduled; A detection module is used to detect the tasks to be scheduled in the local task queue, and a first execution module is used to start executing the detected first task, and an end execution module is used to end executing the detected second task; the first task is a task to be scheduled whose start time reaches the current time, and the second task is a task to be scheduled whose end time reaches the current time.

12. An electronic device, characterized in that: include: A processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method according to any one of claims 1 to 10 when executing the program.

13. A readable storage medium, characterized in that: When the instructions in the readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute any one of the methods of claims 1-10.

Images